An alternative screening approach for Google Search identifies an accurate and manageable number of results for a systematic review (case study)

BACKGROUND

A challenge when using Google Search to identify studies for a systematic review is managing the high number of results, which can number in the hundreds of thousands or even more. Studies and guidance on web searching suggest limiting the screening process, e.g. to the first 100 results.

OBJECTIVES

Our objective in this case study is to demonstrate an alternative approach to screening the results retrieved by Google Search which is based on our experience that the viewable number of results is often far fewer than the estimated number calculated by the search engine.

METHODS

We screened the results of three searches of Google Search using our approach, which involves increasing the number of results displayed per page from 10 to the maximum of 100. We then calculated the viewable number of results and compared this with the estimated number.

RESULTS

The mean of the estimated number of results for the three searches was 569,454,000. The mean of the viewable number results was 463 (0.00008% of the mean of the estimated number of results).

CONCLUSION

Our findings challenge the commonly reported view that the number of results retrieved when using Google Search is too high to screen in full.

ChatGPT: Game-changer or wildcard for systematic searching?

In this editorial, Anthea Sutton and Veronica Parisi reflect on ChatGPT, how it may contribute to systematic searching, and provide their overview of some recent training they attended on ChatGPT, AI and systematic literature reviews.

Integrating diversity, equity, and inclusion principles into instruction: Case studies from the University of Florida's Health Science Center Libraries

Librarians at the University of Florida Health Science Center Libraries have begun to intentionally incorporate diversity, equity, and inclusion (DEI) principles into teaching during design, implementation, and evaluation. This article uses four case studies to provide an overview of the librarians' approaches to inclusive teaching (1) an annual workshop for Physical Therapy students on the intersection between DEI, health literacy, and patient education; (2) a librarian-taught one-credit course for public health students, which was revised to create a more inviting syllabus and integrate elements of universal design; (3) an annual project for first year medical students highlighting health disparities and community resources; and (4) piloting the application of critical librarianship principles in library standalone sessions on database searching and reference management. Suggestions are provided for other librarians who are interested in developing a culture of inclusive teaching in their own libraries.

Development of a validated search filter for Ovid Embase for degenerative cervical myelopathy

BACKGROUND

Degenerative cervical myelopathy (DCM) is a recently proposed umbrella term for symptomatic cervical spinal cord compression secondary to degeneration of the spine. Currently literature searching for DCM is challenged by the inconsistent uptake of the term 'DCM' with many overlapping keywords and numerous synonyms.

OBJECTIVES

Here, we adapt our previous Ovid medline search filter for the Ovid embase database, to support comprehensive literature searching. Both embase and medline are recommended as a minimum for systematic reviews.

METHODS

References contained within embase identified in our prior study formed a 'development gold standard' reference database (N = 220). The search filter was adapted for embase and checked against the reference database. The filter was then validated against the 'validation gold standard'.

RESULTS

A direct translation was not possible, as medline indexing for DCM and the keywords search field were not available in embase. We also used the 'focus' function to improve precision. The resulting search filter has 100% sensitivity in testing.

DISCUSSION AND CONCLUSION

We have developed a validated search filter capable of retrieving DCM references in embase with high sensitivity. In the absence of consistent terminology and indexing, this will support more efficient and robust evidence synthesis in the field.

Why do researchers co-author evidence syntheses with librarians? A mixed-methods study

Librarians and information specialists are experts in designing comprehensive literature searches, such as those needed for Evidence Syntheses (ES). The contributions of these professionals to ES research teams have several documented benefits, especially when they collaborate on the project. However, librarian co-authorship is relatively rare. This study explores researcher motivations for working with librarians at the co-author level through a mixed methods design. Interviews with researchers identified 20 potential motivations that were then tested through an online questionnaire sent to authors of recently published ES. Consistent with previous findings, most respondents did not have a librarian co-author on their ES, though 16% acknowledged one in their manuscript and 10% consulted one but did not document the contribution. Search expertise was the most common motivation both to and not to co-author with librarians. Those that had or were interested in co-authoring stated that they wanted the librarians' search expertise, while those who had not or were not interested stated that they already had the necessary search expertise. Researchers who were motivated by methodological expertise and availability were more likely to have co-authored their ES with a librarian. No motivations were negatively associated with librarian co-authorship. These findings provide an overview of the motivations that influence researchers to bring a librarian into an ES investigatory team. More research is needed to substantiate the validity of these motivations.

Feasibility and desirability of screening search results from Google Search exhaustively for systematic reviews: A cross-case analysis

A commonly reported challenge of using Google Search to identify studies for a systematic review is the high number of results retrieved. Thus, 'stopping rules' are applied when screening, such as screening only the first 100 results. However, recent evidence shows that Google Search estimates a much higher number of results than the viewable number, raising the possibility of exhaustive screening. This study aimed to provide further evidence on the feasibility of screening search results from Google Search exhaustively, and to assess the desirability of this in terms of identifying studies for a systematic review. We conducted a cross-case analysis of the search results of eight Google Search searches from two systematic reviews. Feasibility of exhaustive screening was ascertained by calculating the viewable number of results. Desirability was ascertained according to: (1) the distribution of studies within the results, irrespective of relevance to a systematic review; (2) the distribution of studies which met the inclusion criteria for the two systematic reviews. The estimated number of results across the eight searches ranged from 342,000 to 72,300,000. The viewable number ranged from 272 to 364. Across the eight searches the distribution of studies was highest in the first 100 results. However, the lowest ranking relevant studies were ranked 227th and 215th for the two systematic reviews. One study per review was identified uniquely from searching Google Search, both within the first 100 results. The findings suggest it is feasible and desirable to screen Google Search results more extensively than commonly reported.

Determining COVID-19's impact on an academic medical library's literature search service

Objective

At many institutions, literature search services are an important aspect of health science librarianship. This exploratory study analyzes how the COVID-19 pandemic impacted the use of an academic hospital medical library's literature search service.

Methods

To evaluate the pandemic's impact on literature searching at The University of Tennessee Medical Center's Preston Medical Library, data were analyzed for changes from the year before the pandemic (March 1, 2019 to February 29, 2020) to the first year during the pandemic (March 1st, 2020 to February 28, 2021). This was accomplished using LibWizard, a library feedback and assessment application, to review literature search data during the two periods. Variables of interest included total searches, purpose of searches, affiliation of the searcher, and searches with a pandemic-related research question.

Results

A 36.6% drop in literature search service usage was reported from the pre-pandemic year to the during-pandemic year. There was a 55.3% decrease in searches intended for research, as well as significant decreases in the number of searches requested by all patron affiliations. After March 2020, 10% of all searches concerned a COVID-related topic.

Conclusion

The overall decrease in literature search requests, decrease in research searches, decrease in searches among all patron affiliations, and increase in searches on a COVID-related topic suggest that healthcare worker and institutional priorities changed during the pandemic. The results revealed research interests during the first year of the pandemic, as well as an overall change in library service functionality.

Landscape of health sciences librarian-mediated search services

BACKGROUND

Health librarians have traditionally provided mediated searches to support patient care, education and research.

OBJECTIVES

This study aims to discover the types of search result formats used by health science libraries, determine current practice among health science libraries (types of requesters served, fees, deduplication, turnaround time and citation manager use) and uncover innovative methods for providing search results.

METHODS

An online survey was distributed to the MEDLIB-L, ExpertSearching, MidContinental Chapter of the Medical Library Association and ICON listservs and through direct email to selected Association of Academic Health Sciences Libraries reference and education librarians.

RESULTS

Librarians affiliated with 127 institutions from 11 countries (including the USS) and 36 USS states and territories responded. One hundred and forty-two of the total 150 analysed responses provided information on full-text access, and 81 of those 142 responses (57%) indicated that the institutions' link-resolver links were included in search results provided to the requester. The survey responses provide information on literature search services regarding turnaround time, use of a citation managers, fees and deduplication.

CONCLUSION

With the developing landscape of citation managers and the tools offered, these data can be used as a benchmark for librarians who are considering evaluating or modifying their search service delivery.

Efficient searching for NICE public health guidelines: Would using fewer sources still find the evidence?

Systematic searches are integral to identifying the evidence that is used in National Institute for Health and Care Excellence (NICE) public health guidelines (PHGs). This study analyses the sources, including bibliographic databases and other techniques, required for PHGs. The aims were to analyse the sources used to identify the publications included in NICE PHGs; and to assess whether fewer sources could have been searched to retrieve these publications. Data showing how the included publications had been identified was collated using search summary tables. Three scenarios were created to test various combinations of sources to determine whether fewer sources could have been used. The sample included 29 evidence reviews, compiled using 13 searches, to support 10 PHG topics. Across the PHGs, 23 databases and six other techniques retrieved included publications. A mean reduction in total results of 6.5% could have been made if the minimum set of sources plus Cochrane Library, Embase, and MEDLINE were searched. On average, Cochrane Library, Embase, and MEDLINE contributed 76.8% of the included publications, with other databases adding 11% and other techniques 12.2%. None of the searches had a minimum set that was comprised entirely of databases. There was not a core set of sources for PHGs. A range of databases and techniques, covering a multi-disciplinary evidence base, was required to identify all included publications. It would be possible to reduce the number of sources searched and make some gains in productivity. It is important to create a tailored set of sources to do an efficient search.

NHS librarians collaborate to develop a search bank peer reviewing and sharing COVID-19 searches - an evaluation

BACKGROUND

Responding to the COVID-19 pandemic, Health Education England (HEE) mobilised a group of expert searchers from NHS libraries in England to develop a platform for librarians to share peer reviewed search strategies and results on the Knowledge for Healthcare website.

OBJECTIVES

(1) To document the origins of the COVID-19 search bank, (2) evaluate attitudes of NHS librarians in England towards the search bank and (3) identify lessons learned and consider whether the initiative might be developed further.

METHODS

Structured interviews with the peer reviewers (n = 10) were conducted, and a questionnaire survey of the NHS library community using the search bank was undertaken.

RESULTS

The interviews confirmed the value of collaboration. Expert searchers worked in pairs to peer review submitted search strategies. The survey (85 responses) indicated that a majority had used the search bank, and approved of the project, with some differences of opinion on functionality and future developments.

DISCUSSION

Collaborative working for the search bank probably saved time for individual NHS librarians. The quality of the searches submitted was variable as were librarians' approaches to presentation and development of search strategies. Peer review benefits from a buddy approach among expert searchers and agreement about feedback provided to contributors.

CONCLUSION

Search strategies are the most useful element of a search bank. Peer review can be challenging and would benefit from a formal structure, but it is professionally rewarding.

Updated generic search filters for finding studies of adverse drug effects in Ovid medline and Embase may retrieve up to 90% of relevant studies

BACKGROUND

The most current objectively derived search filters for adverse drug effects are 15 years old and other strategies have not been developed and tested empirically.

OBJECTIVE

To develop and validate search filters to retrieve evidence on adverse drug effects from Ovid medline and Ovid Embase.

METHODS

We identified systematic reviews of adverse drug effects in Epistemonikos. From these reviews, we collated their included studies which we then randomly divided into three tests and one validation set of records. We constructed a search strategy to maximise relative recall using word frequency analysis with test set one. This search strategy was then refined using test sets two and three and validated on the final set of records.

RESULTS

Of 107 systematic reviews which met our inclusion criteria, 1948 unique included studies were available from medline and 1980 from Embase. Generic adverse drug effects searches in medline and Embase achieved 90% and 89% relative recall, respectively. When specific adverse effects terms were added recall was improved.

CONCLUSION

We have derived and validated search filters that retrieve around 90% of records with adverse drug effects data in medline and Embase. The addition of specific adverse effects terms is required to achieve higher recall.

Databases for Researching Athletic Training Literature

Searching the athletic training literature can be confusing and overwhelming with many possible databases for locating relevant peer-reviewed scholarship. Finding evidence-based literature from respected publications is helpful in clinical decision-making for athletic training practitioners. This column details recommended databases and search tips to help students, staff, clinicians, and faculty in the field of athletic training find the literature they need to help make evidence-based decisions and to stay current with the published literature. Databases discussed include Cochrane, PubMed, SPORTDiscus, CINAHL, PEDro, Sports Medicine, and Education Index (formerly Physical Education Index), and Google Scholar.

Adapting to Changes in Publishing When Searching for Alternatives and Reporting on Animal Research: A Librarian's Perspective

Since the inaugural issue of ATLA, many changes within publishing have occurred, impacting when, where, and how researchers conduct literature searches for non-animal alternatives. Such changes include increased rate of growth in scientific publications, greater number of databases and online resources available to search, opportunities for open and almost immediate dissemination of research outputs such as preprints and method protocols, and the development of reporting guidelines for animal research. Here we offer a librarian's perspective on these changes and advice on how to manage them to enable robust and diverse alternatives to be implemented in future research.

The ScHARR LMIC filter: adapting a low- and middle-income countries geographic search filter to identify studies on preterm birth prevention and management

INTRODUCTION

Search filters are used to find evidence on specific subjects. Performance of filters can be varied and may need adapting to meet the needs of research topics. There are limited geographic search filters available, and only one pertaining to low- and middle-income countries (LMICs). When searching for literature on preterm birth prevention and management in LMICs for a research project at the School of Health and Related Research (ScHARR), we made use of the Cochrane Effective Practice and Organisation of Care (EPOC) LMIC geographic search filter for the databases; Ovid MEDLINE, Ovid Embase, Cochrane Library. During screening following a broad scoping search in Ovid MEDLINE, it was found that the EPOC LMIC filter did not identify a relevant study. Adaptations to the filter were required to maximise retrieval.

METHODS

Adaptations were made to the LMIC geographic search filter to identify the missing study. Institution was included as a search field, and the search terms high burden or countdown countries were added. The filter was translated for the databases; Ovid Embase, Cochrane Library, Ovid PsycINFO, and CINAHL via EBSCO.

RESULTS

The adapted ScHARR LMIC filter increases the sensitivity of the EPOC LMIC search filter.

DISCUSSION

The ScHARR LMIC filter is a non-validated 1st generation filter. Validating the filter would confirm its retrieval performance and benefit information professionals, researchers and health professionals CONCLUSION: We recommend that the ScHARR LMIC filter is used to improve sensitivity of the Cochrane EPOC LMIC filter and reduce the risk of missing relevant studies. This article is protected by copyright. All rights reserved.

Literature searching methods or guidance and their application to public health topics: A narrative review

Background

Information specialists conducting searches for systematic reviews need to consider key questions around which and how many sources to search. This is particularly important for public health topics where evidence may be found in diverse sources.

Objectives

The objective of this review is to give an overview of recent studies on information retrieval guidance and methods that could be applied to public health evidence and used to guide future searches.

Methods

A literature search was performed in core databases and supplemented by browsing health information journals and citation searching. Results were sifted and reviewed.

Results

Seventy‐two papers were found and grouped into themes covering sources and search techniques. Public health topics were poorly covered in this literature.

Discussion

Many researchers follow the recommendations to search multiple databases. The review topic influences decisions about sources. Additional sources covering grey literature eliminate bias but are time‐consuming and difficult to search systematically. Public health searching is complex, often requiring searches in multidisciplinary sources and using additional methods.

Conclusions

Search planning is advisable to enable decisions about which and how many sources to search. This could improve with more work on modelling search scenarios, particularly in public health topics, to examine where publications were found and guide future research.

Search Club: Using peer support to develop search skills and share knowledge in a specialist NHS team

The newly-formed Knowledge Management team at Health Education England (HEE) established an internal "Search Club" for their Knowledge Specialists to share good practice, exchange ideas, and discuss approaches to developing search strategies. The article describes how this was initiated and run online. The sessions improved the Knowledge Management team skills, and gave them an opportunity to share and learn from each other. A further benefit has been the creation of resources such as a "synonyms bank", search strategies, and a bank of grey literature sources. These reduce duplication of effort, save time, and improve consistency across the team's output. D.I.

The NICE OECD countries' geographic search filters: Part 2-validation of the MEDLINE and Embase (Ovid) filters

Objective:

We previously developed draft MEDLINE and Embase (Ovid) geographic search filters for Organisation for Economic Co-operation and Development (OECD) countries to assess their feasibility for finding evidence about the countries. Here, we describe the validation of these search filters.

Methods:

We identified OECD country references from thirty National Institute for Health and Care Excellence (NICE) guidelines to generate gold standard sets for MEDLINE (n=2,065) and Embase (n=2,023). We validated the filters by calculating their recall against these sets. We then applied the filters to existing search strategies for three OECD-focused NICE guideline reviews (NG103 on flu vaccination, NG140 on abortion care, and NG146 on workplace health) to calculate the filters' impact on the number needed to read (NNR) of the searches.

Results:

The filters both achieved 99.95% recall against the gold standard sets. Both filters achieved 100% recall for the three NICE guideline reviews. The MEDLINE filter reduced NNR from 256 to 232 for the NG103 review, from 38 to 27 for the NG140 review, and from 631 to 591 for the NG146 review. The Embase filter reduced NNR from 373 to 341 for the NG103 review, from 101 to 76 for the NG140 review, and from 989 to 925 for the NG146 review.

Conclusion:

The NICE OECD countries' search filters are the first validated filters for the countries. They can save time for research topics about OECD countries by finding the majority of evidence about OECD countries while reducing search result volumes in comparison to no filter use.

Google is goodish: An information literacy course designed to teach users why google may not always be the best place to search for evidence

This article describes a course that was developed in response to health sector and local authority workers being reliant on Google and using it for their information needs regardless of whether it was the best place to search. The methodology for developing and structuring the course is explored, including details of the content included. The author concludes by asserting that teaching users about the effective use of Google is an important part of user education. D.I.

Identifying evidence for five realist reviews in primary health care: A comparison of search methods

The approach to identifying evidence for inclusion in realist reviews differs from that used in 'traditional' systematic reviews. Guidance suggests that realist reviews should be inclusive of diverse data from a range of sources, gathered in iterative searching cycles. Saturation is prioritised over exhaustiveness. Supplementary techniques such as citation snowballing are emphasised as potentially important sources of evidence. This paper describes the processes used to identify evidence in a selection of realist reviews focused on primary health care settings and examines the origin and type of evidence selected for inclusion. Data from five realist reviews were extracted from (a) reviewers' reference management libraries and (b) records kept by review teams. Although all reviews focused on primary health care, they used data from a wide range of document types and research designs, drawing on learning from multiple perspectives and settings, and sourced the documents containing this data in a variety of ways. Systematic searching of academic databases played an important role, supplementary search techniques such as snowballing were used to identify a significant proportion of documents included in the reviews. Our analysis demonstrates the diverse data sources used within realist reviews and the need for flexible, responsive efforts to identify relevant documents. Reviewers and information specialists should devise approaches to data gathering that reflect the individual needs of realist review projects and report these transparently.

The NICE OECD countries' geographic search filters: Part 1-methodology for developing the draft MEDLINE and Embase (Ovid) filters

Objective:

There are no existing validated search filters for the group of 37 Organisation for Economic Co-operation and Development (OECD) countries. This study describes how information specialists from the United Kingdom's National Institute for Health and Care Excellence (NICE) developed and evaluated novel OECD countries’ geographic search filters for MEDLINE and Embase (Ovid) to improve literature search effectiveness for evidence about OECD countries.

Methods:

We created the draft filters using an alternative approach to standard filter construction. They are composed entirely of geographic subject headings and are designed to retain OECD country evidence by excluding non-OECD country evidence using the NOT Boolean operator. To evaluate the draft filters’ effectiveness, we used MEDLINE and Embase literature searches for three NICE guidelines that retrieved >5,000 search results. A 10% sample of the excluded references was screened to check that OECD country evidence was not inadvertently excluded.

Results:

The draft MEDLINE filter reduced results for each NICE guideline by 9.5% to 12.9%. In Embase, search results were reduced by 10.7% to 14%. Of the sample references, 7 of 910 (0.8%) were excluded inadvertently. These references were from a guideline about looked-after minors that concerns both OECD and non-OECD countries.

Conclusion:

The draft filters look promising—they reduced search result volumes while retaining most OECD country evidence from MEDLINE and Embase. However, we advise caution when using them in topics about both non-OECD and OECD countries. We have created final versions of the search filters and will validate them in a future study.

Redundancy of Terms in Search Strategies. Comment on "Searching PubMed to Retrieve Publications on the COVID-19 Pandemic: Comparative Analysis of Search Strings"

Collapse

Overlaps of multiple database retrieval and citation tracking in dementia care research: a methodological study

OBJECTIVE

We aimed to determine overlaps and optimal combination of multiple database retrieval and citation tracking for evidence synthesis, based on a previously conducted scoping review on facilitators and barriers to implementing nurse-led interventions in dementia care.

METHODS

In our 2019 scoping review, we performed a comprehensive literature search in eight databases (CENTRAL, CINAHL, Embase, Emcare, MEDLINE, Ovid Nursing Database, PsycINFO, and Web of Science Core Collection) and used citation tracking. We retrospectively analyzed the coverage and overlap of 10,527 retrieved studies published between 2015 and 2019. To analyze database overlap, we used cross tables and multiple correspondence analysis (MCA).

RESULTS

Of the retrieved studies, 6,944 were duplicates and 3,583 were unique references. Using our search strategies, considerable overlaps can be found in some databases, such as between MEDLINE and Web of Science Core Collection or between CINAHL, Emcare, and PsycINFO. Searching MEDLINE, CINAHL, and Web of Science Core Collection and using citation tracking were necessary to retrieve all included studies of our scoping review.

CONCLUSIONS

Our results can contribute to enhancing future search practice related to database selection in dementia care research. However, due to limited generalizability, researchers and librarians should carefully choose databases based on the research question. More research on optimal database retrieval in dementia care research is required for the development of methodological standards.

Adoption of peer review of literature search strategies in knowledge synthesis from 2009 to 2018: An overview

BACKGROUND

Knowledge synthesis (KS) reviews rely on good quality literature searches to capture a complete set of relevant studies, and peer review of the search strategy is one quality control mechanism that contributes to better quality reviews. Guidelines for peer review of electronic search strategies (PRESS) have been available since 2008.

OBJECTIVES

This overview provides a snapshot of KS indexed in Scopus, published between 2009 and 2018, that reported peer review of the literature search strategy.

METHODS

Articles were identified through citation chasing for PRESS guidance documents and supplementary keyword searches. The characteristics of individual articles and the journals that published them were documented, and descriptive statistics were compiled.

RESULTS

415 articles from 169 journals met inclusion criteria. Approximately half were published in 14 journal titles. Most reviews reported the involvement of an information professional, but PRESS reviewers were rarely acknowledged. An overwhelming majority of review teams were based in Canada.

DISCUSSION

Reported use of PRESS was low during the period examined, but under-reporting may be a factor. Investigation of the barriers and facilitators of PRESS adoption is needed.

CONCLUSION

Despite its value, adoption of PRESS appears low. Advocacy for, and education about, PRESS may be required.

Who is blocking access to PubMed? Educational 'Escape Room' for medical residents

Teaching students how to conduct bibliographic searches in health sciences' databases is essential training. One of the challenges librarians face is how to motivate students during classroom learning. In this article, two hospital libraries, in Spain, used Escape rooms as a method of bringing creativity, teamwork, communication and critical thinking into bibliographic search instruction. Escape rooms are a series of puzzles that must be solved to exit the game. This article explores the methods used for integrating escape rooms into training programmes and evaluates the results. Escape Rooms are a useful tool that can be integrated into residents' training to support their instruction on bibliographic searches. This kind of learning stablishes competences like logical thinking and deductive approaching. These aspects aid participants to make their own decision and to develop social and intellectual skills.

Evaluation of literature searching tools for curation of mismatch repair gene variants in hereditary colon cancer

Pathogenic constitutional genomic variants in the mismatch repair (MMR) genes are the drivers of Lynch syndrome; optimal variant interpretation is required for the management of suspected and confirmed cases. The International Society for Hereditary Gastrointestinal Tumours (InSiGHT) provides expert classifications for MMR variants for the US National Human Genome Research Institute's (NHGRI) ClinGen initiative and interprets variants with discordant classifications and those of uncertain significance (VUSs). Given the onerous nature of extracting information related to variants, literature searching tools which harness artificial intelligence may aid in retrieving information to allow optimum variant classification. In this study, we described the nature of discordance in a sample of 80 variants from a list of variants requiring updating by InSiGHT for ClinGen by comparing their existing InSiGHT classifications with the various submissions for each variant on the US National Centre for Biotechnology Information's (NCBI) ClinVar database. To identify the potential value of a literature searching tool in extracting information related to classification, all variants were searched for using a traditional method (Google Scholar) and literature searching tool (Mastermind) independently. Descriptive statistics were used to compare: the number of articles before and after screening for relevance and the number of relevant articles unique to either method. Relevance was defined as containing the variant in question as well as data informing variant interpretation. A total of 916 articles were returned by both methods and Mastermind averaged four relevant articles per search compared to Google Scholar's three. Of relevant Mastermind articles, 193/308 (62.7%) were unique to it, compared to 87/202, (43.0%) for Google Scholar. For 24 variants, either or both methods found no information. All 6/80 (20%) variants with pathogenic or likely pathogenic InSiGHT classifications have newer VUS assertions on ClinVar. Our study demonstrated that for a sample of variants with varying discordant interpretations, Mastermind was able to return on average, a more relevant and unique literature search. Google Scholar was able to retrieve information that Mastermind did not, which supports a conclusion that Mastermind could play a complementary role in literature searching for classification. This work will aid InSiGHT in its role of classifying MMR variants.

Searching PubMed to Retrieve Publications on the COVID-19 Pandemic: Comparative Analysis of Search Strings

Background

Since it was declared a pandemic on March 11, 2020, COVID-19 has dominated headlines around the world and researchers have generated thousands of scientific articles about the disease. The fast speed of publication has challenged researchers and other stakeholders to keep up with the volume of published articles. To search the literature effectively, researchers use databases such as PubMed.

Objective

The aim of this study is to evaluate the performance of different searches for COVID-19 records in PubMed and to assess the complexity of searches required.

Methods

We tested PubMed searches for COVID-19 to identify which search string performed best according to standard metrics (sensitivity, precision, and F-score). We evaluated the performance of 8 different searches in PubMed during the first 10 weeks of the COVID-19 pandemic to investigate how complex a search string is needed. We also tested omitting hyphens and space characters as well as applying quotation marks.

Results

The two most comprehensive search strings combining several free-text and indexed search terms performed best in terms of sensitivity (98.4%/98.7%) and F-score (96.5%/95.7%), but the single-term search COVID-19 performed best in terms of precision (95.3%) and well in terms of sensitivity (94.4%) and F-score (94.8%). The term Wuhan virus performed the worst: 7.7% for sensitivity, 78.1% for precision, and 14.0% for F-score. We found that deleting a hyphen or space character could omit a substantial number of records, especially when searching with SARS-CoV-2 as a single term.

Conclusions

Comprehensive search strings combining free-text and indexed search terms performed better than single-term searches in PubMed, but not by a large margin compared to the single term COVID-19. For everyday searches, certain single-term searches that are entered correctly are probably sufficient, whereas more comprehensive searches should be used for systematic reviews. Still, we suggest additional measures that the US National Library of Medicine could take to support all PubMed users in searching the COVID-19 literature.

Conduct and reporting of citation searching in Cochrane systematic reviews: A cross-sectional study

Background

The search for studies for a systematic review should be conducted systematically and reported transparently to facilitate reproduction. This study aimed to report on the conduct and reporting of backward citation searching (ie, checking reference lists) and forward citation searching in a cross section of Cochrane reviews. Citation searching uses the citation network surrounding a source study to identify additional studies.

Methods

Cochrane reviews were identified by searching the Cochrane Database of Systematic Reviews using the wildcard symbol and date limiting to the 3‐month period November 2016 to January 2017. Cochrane reviews thus identified were screened for mention of citation searching. Descriptive detail on the conduct and reporting of citation searching was captured in data extraction forms and described and evaluated.

Results

Two hundred fifteen Cochrane reviews were identified. One hundred seventy‐two reviews reported backward citation searching, and 18 reviews reported forward citation searching. Web of Science was the most frequently reported citation index. The studies used for backward citation searching consisted mainly of studies meeting the inclusion criteria. One‐third of reviews that reported forward citation searching used selected studies of importance. Reporting of citation searching was compliant with the Methodological Expectations of Cochrane Intervention Reviews (MECIR) standards, but full transparency requires additional detail that only a minority of reviews reported.

Conclusion

The conduct of backward citation searching was more uniform than forward citation searching. This might be due to lack of MECIR guidance for forward citation searching. Reporting was generally compliant with MECIR, but this is not always sufficient to ensure full transparency.

The development of search filters for adverse effects of medical devices in medline and embase

BACKGROUND

Objectively derived search filters for adverse drug effects and complications in surgery have been developed but not for medical device adverse effects.

OBJECTIVE

To develop and validate search filters to retrieve evidence on medical device adverse effects from ovid medline and embase.

METHODS

We identified systematic reviews from Epistemonikos and the Health Technology Assessment (hta) database. Included studies within these reviews that reported on medical device adverse effects were randomly divided into three test sets and one validation set of records. Using word frequency analysis from one test set, we constructed a sensitivity maximising search strategy. This strategy was refined using two other test sets, then validated.

RESULTS

From 186 systematic reviews which met our inclusion criteria, 1984 unique included studies were available from medline and 1986 from embase. Generic adverse effects searches in medline and embase achieved 84% and 83% sensitivity. Recall was improved to over 90%, however, when specific adverse effects terms were added.

CONCLUSION

We have derived and validated novel search filters that retrieve over 80% of records with medical device adverse effects data in medline and embase. The addition of specific adverse effects terms is required to achieve higher levels of sensitivity.

Topic search filters: a systematic scoping review

BACKGROUND

Searching for topics within large biomedical databases can be challenging, especially when topics are complex, diffuse, emerging or lack definitional clarity. Experimentally derived topic search filters offer a reliable solution to effective retrieval; however, their number and range of subject foci remain unknown.

OBJECTIVES

This systematic scoping review aims to identify and describe available experimentally developed topic search filters.

METHODS

Reports on topic search filter development (1990-) were sought using grey literature sources and 15 databases. Reports describing the conception and prospective development of a database-specific topic search and including an objectively measured estimate of its performance ('sensitivity') were included.

RESULTS

Fifty-four reports met inclusion criteria. Data were extracted and thematically synthesised to describe the characteristics of 58 topic search filters.

DISCUSSION

Topic search filters are proliferating and cover a wide range of subjects. Filter reports, however, often lack clear definitions of concepts and topic scope to guide users. Without standardised terminology, filters are challenging to find. Information specialists may benefit from a centralised topic filter repository and appraisal checklists to facilitate quality assessment.

CONCLUSION

Findings will help information specialists identify existing topic search filters and assist filter developers to build on current knowledge in the field.

Development and validation of a search filter to identify equity-focused studies: reducing the number needed to screen

Background

Health inequalities, worse health associated with social and economic disadvantage, are reported by a minority of research articles. Locating these studies when conducting an equity-focused systematic review is challenging due to a deficit in standardised terminology, indexing, and lack of validated search filters. Current reporting guidelines recommend not applying filters, meaning that increased resources are needed at the screening stage.

Methods

We aimed to design and test search filters to locate studies that reported outcomes by a social determinant of health. We developed and expanded a ‘specific terms strategy’ using keywords and subject headings compiled from recent systematic reviews that applied an equity filter. A ‘non-specific strategy’ was compiled from phrases used to describe equity analyses that were reported in titles and abstracts, and related subject headings. Gold standard evaluation and validation sets were compiled. The filters were developed in MEDLINE, adapted for Embase and tested in both. We set a target of 0.90 sensitivity (95% CI; 0.84, 0.94) in retrieving 150 gold standard validation papers. We noted the reduction in the number needed to screen in a proposed equity-focused systematic review and the proportion of equity-focused reviews we assessed in the project that applied an equity filter to their search strategy.

Results

The specific terms strategy filtered out 93-95% of all records, and retrieved a validation set of articles with a sensitivity of 0.84 in MEDLINE (0.77, 0.89), and 0.87 (0.81, 0.92) in Embase. When combined (Boolean ‘OR’) with the non-specific strategy sensitivity was 0.92 (0.86, 0.96) in MEDLINE (Embase 0.94; 0.89, 0.97). The number needed to screen was reduced by 77% by applying the specific terms strategy, and by 59.7% (MEDLINE) and 63.5% (Embase) by applying the combined strategy. Eighty-one per cent of systematic reviews filtered studies by equity.

Conclusions

A combined approach of using specific and non-specific terms is recommended if systematic reviewers wish to filter studies for reporting outcomes by social determinants. Future research should concentrate on the indexing standardisation for equity studies and further development and testing of both specific and non-specific terms for accurate study retrieval.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0567-x) contains supplementary material, which is available to authorized users.

Design and implementation of a tool for conversion of search strategies between PubMed and Ovid MEDLINE

BACKGROUND

Both PubMed and Ovid MEDLINE contain records from the MEDLINE database. However, there are subtle differences in content, functionality, and search syntax between the two. There are many instances in which researchers may wish to search both interfaces, such as when conducting supplementary searching for a systematic review to retrieve a unique content from PubMed or when using a previously published search strategy from a different interface, but little guidance on how to best conduct these searches. The aim of this project is to describe differences in search functionality between Ovid MEDLINE and PubMed, provide guidance for converting search strategies between the two, and develop an easy-to-use, freely available web-based tool to automate search syntax translations.

CASE PRESENTATION

In this paper, we present a custom-built freely available online tool, Medline Transpose, to streamline the process of converting search strategies between Ovid MEDLINE and PubMed. With this tool, users can paste a strategy formatted for one interface into the search box and immediately retrieve an output formatted for use in the other interface, with recommendations for changes that users can make to the strategy where an exact translation does not exist.

CONCLUSION

This novel approach has the potential to reduce time and errors that database users spend translating search strategies.

Delivering a MOOC for literature searching in health libraries: evaluation of a pilot project

In an era when library budgets are being reduced, Massive Online Open Courses (MOOC's) can offer practical and viable alternatives to the delivery of costly face-to-face training courses. In this study, guest writers Gil Young from Health Care Libraries Unit - North, Lisa McLaren from Brighton and Sussex Medical School and Liverpool University PhD student Michelle Maden describe the outcomes of a funded project they led to develop a MOOC to deliver literature search training for health librarians. Funded by Health Education England, the MOOC was developed by the Library and Information Health Network North West as a pilot project that ran for six weeks. In particular, the MOOC target audience is discussed, how content was developed for the MOOC, promotion and participation, cost-effectiveness, evaluation, the impact of the MOOC and recommendations for future development. H. S.

Finding the Integrated Care Evidence Base in PubMed and Beyond: A Bibliometric Study of the Challenges

Introduction:

Integrated care research evidence should be optimally visible and accessible to stakeholders. This study examines the contribution of specific databases to the discovery of integrated care evidence, and tests the usefulness of Medical Subject Heading (MeSH) indexing of this literature within PubMed.

Methods:

We used bibliometric methods to analyse the integrated care literature indexed within six databases between 2007 and 2016. An international expert advisory group assessed the relevance of citations randomly retrieved from PubMed using MeSH term ‘Delivery of Health Care, Integrated’.

Results:

Integrated care evidence is diffuse, spread across many journals. Between 2007 and 2016, integrated care citations grew substantially, with the rate of increase highest in Embase. PubMed contributes the largest proportion of unique citations (citations not included in any of the other databases analysed), followed by Embase, PsycINFO and CINAHL. On average, expert reviewers rated 42.5% of citations retrieved by MeSH term ‘Delivery of Health Care, Integrated’ as relevant to integrated care. When these citations were dual reviewed, inter-rater agreement was low.

Conclusion:

MeSH terms alone are insufficient to retrieve integrated care content from PubMed. Embase and CINAHL contain unique content not found in PubMed that should not be overlooked. A validated search filter is proposed to simplify the process of finding integrated care research for clinicians, managers and decision-makers.

Professional collaboration in searching the evidence for an ill-defined concept

This paper outlines the inter-professional collaboration of the authors, a PhD student, his supervisor and an information professional, to systematically search the literature for an ill-defined concept. The research question posed for the scoping literature review indicated that the topic, the subjective socio-economic status and health of adults with intellectual disabilities, was rare. The need for a methodological search process was therefore identified and successfully carried out. The paper presents an analysis of the processes and the collaboration involved in developing a successful search strategy. The resulting transformative learning by the researcher of the professional practice of the information specialist illuminates their facilitating and supportive role in advancing health related research. F.J.

The development of search filters for adverse effects of surgical interventions in medline and Embase

Background

Search filter development for adverse effects has tended to focus on retrieving studies of drug interventions. However, a different approach is required for surgical interventions.

Objective

To develop and validate search filters for medline and Embase for the adverse effects of surgical interventions.

Methods

Systematic reviews of surgical interventions where the primary focus was to evaluate adverse effect(s) were sought. The included studies within these reviews were divided randomly into a development set, evaluation set and validation set. Using word frequency analysis we constructed a sensitivity maximising search strategy and this was tested in the evaluation and validation set.

Results

Three hundred and fifty eight papers were included from 19 surgical intervention reviews. Three hundred and fifty two papers were available on medline and 348 were available on Embase. Generic adverse effects search strategies in medline and Embase could achieve approximately 90% relative recall. Recall could be further improved with the addition of specific adverse effects terms to the search strategies.

Conclusion

We have derived and validated a novel search filter that has reasonable performance for identifying adverse effects of surgical interventions in medline and Embase. However, we appreciate the limitations of our methods, and recommend further research on larger sample sizes and prospective systematic reviews.

A Clinical Librarian Embedded in Medical Education: Patient-Centered Encounters for Preclinical Medical Students

Adding patient encounters and simulation to the preclinical years of medical school is becoming increasingly popular. This article describes the creation of active learning opportunities by a clinical librarian that are aimed at training preclinical students through the use of simulated patient scenarios. Scenarios for second-year students walk them through the evidence-based resources needed in clinical years and beyond through a standardized patient encounter. Scenarios for first-year students involve role-play of cases where the patient and physician bring contrasting ideas to the outpatient interaction. All scenarios are carried out under the guidance of a clinician and librarian.

Comparison of a full systematic review versus rapid review approaches to assess a newborn screening test for tyrosinemia type 1

BACKGROUND

Rapid reviews are increasingly used to replace/complement systematic reviews to support evidence-based decision-making. Little is known about how this expedited process affects results.

OBJECTIVES

To assess differences between rapid and systematic review approaches for a case study of test accuracy of succinylacetone for detecting tyrosinemia type 1.

METHODS

Two reviewers conducted an "enhanced" rapid review then a systematic review. The enhanced rapid review involved narrower searches, a single reviewer checking 20% of titles/abstracts and data extraction, and quality assessment using an unadjusted QUADAS-2. Two reviewers performed the systematic review with a tailored QUADAS-2. Post hoc analysis examined rapid reviewing with a single reviewer (basic rapid review).

RESULTS

Ten papers were included. Basic rapid reviews would have missed 1 or 4 of these (dependent on which reviewer). Enhanced rapid and systematic reviews identified all 10 papers; one paper was only identified in the rapid review through reference checking. Two thousand one hundred seventy-six fewer title/abstracts and 129 fewer full texts were screened during the enhanced rapid review than the systematic review. The unadjusted QUADAS-2 generated more "unclear" ratings than the adjusted QUADAS-2 [29/70 (41.4%) versus 16/70 (22.9%)], and fewer "high" ratings [22/70 (31.4%) versus 42/70 (60.0%)]. Basic rapid reviews contained important inaccuracies in data extraction, which were detected by a second reviewer in the enhanced rapid and systematic reviews.

CONCLUSIONS

Enhanced rapid reviews with 20% checking by a second reviewer may be an appropriate tool for policymakers to expeditiously assess evidence. Basic rapid reviews (single reviewer) have higher risks of important inaccuracies and omissions.

The medline UK filter: development and validation of a geographic search filter to retrieve research about the UK from OVID medline

BACKGROUND

A validated geographic search filter for the retrieval of research about the United Kingdom (UK) from bibliographic databases had not previously been published.

OBJECTIVES

To develop and validate a geographic search filter to retrieve research about the UK from OVID medline with high recall and precision.

METHODS

Three gold standard sets of references were generated using the relative recall method. The sets contained references to studies about the UK which had informed National Institute for Health and Care Excellence (NICE) guidance. The first and second sets were used to develop and refine the medline UK filter. The third set was used to validate the filter. Recall, precision and number-needed-to-read (NNR) were calculated using a case study.

RESULTS

The validated medline UK filter demonstrated 87.6% relative recall against the third gold standard set. In the case study, the medline UK filter demonstrated 100% recall, 11.4% precision and a NNR of nine.

CONCLUSION

A validated geographic search filter to retrieve research about the UK with high recall and precision has been developed. The medline UK filter can be applied to systematic literature searches in OVID medline for topics with a UK focus.

Asking More of Qualitative Synthesis: A Response to Sally Thorne

We continue the conversation initiated by Sally Thorne's observations about "metasynthetic madness." We note that the variety of labels used to describe qualitative syntheses often reflect authors' disciplines and geographical locations. The purpose of systematic literature searching is to redress authors' lack of citation of relevant earlier work and to reassure policy makers that qualitative syntheses are systematic and transparent. There is clearly a need to develop other methods of searching to supplement electronic searches. If searches produce large numbers of articles, sampling strategies may be needed to choose which articles to synthesize. The quality of any synthesis is dependent on the quality of the primary articles; both primary research and qualitative synthesis need to move beyond description and toward theory and explanation. Synthesizers need to pay attention to those articles which do not seem to fit their emerging analysis if they are to avoid stifling new ideas.

Teaching Evidence-Based Veterinary Medicine in the US and Canada

There is no comprehensive review of the extent to which evidence-based veterinary medicine (EBVM) is taught in AVMA-accredited colleges of veterinary medicine in the US and Canada. We surveyed teaching faculty and librarians at these institutions to determine what EBVM skills are currently included in curricula, how they are taught, and to what extent librarians are involved in this process. Librarians appear to be an underused resource, as 59% of respondents did not use librarians/library resources in teaching EBVM. We discovered that there is no standard teaching methodology nor are there common learning activities for EBVM among our survey respondents, who represent 22 institutions. Respondents reported major barriers to inclusion such as a perceived shortage of time in an already-crowded course of study and a lack of high-quality evidence and point-of-care tools. Suggestions for overcoming these barriers include collaborating with librarians and using new EBVM online teaching resources.

A checklist to assess database-hosting platforms for designing and running searches for systematic reviews

BACKGROUND

Systematic reviews require literature searches that are precise, sensitive and often complex. Database-hosting platforms need to facilitate this type of searching in order to minimise errors and the risk of bias in the results.

OBJECTIVES

The main objective of the study was to create a generic checklist of criteria to assess the ability of host platforms to cope with complex searching, for example, for systematic reviews, and to test the checklist against three host platforms (EBSCOhost, OvidSP and ProQuest).

METHOD

The checklist was developed as usual review work was carried out and through discussion between the two authors. Attributes on the checklist were designated as 'desirable' or 'essential'. The authors tested the checklist independently against three host platforms and graded their performance from 1 (insufficient) to 3 (performs well).

RESULTS

Fifty-five desirable or essential attributes were identified for the checklist. None of the platforms performed well for all of the attributes on the checklist.

CONCLUSIONS

Not all database-hosting platforms are designed for complex searching. Librarians and other decision-makers who work in health research settings need to be aware of the different limitations of host platforms for complex searching when they are making purchasing decisions or training others.

Identifying evidence for public health guidance: a comparison of citation searching with Web of Science and Google Scholar

AIM

To examine how effectively forwards citation searching with Web of Science (WOS) or Google Scholar (GS) identified evidence to support public health guidance published by the National Institute for Health and Care Excellence.

METHOD

Forwards citation searching was performed using GS on a base set of 46 publications and replicated using WOS.

OUTCOMES

WOS and GS were compared in terms of recall; precision; number needed to read (NNR); administrative time and costs; and screening time and costs. Outcomes for all publications were compared with those for a subset of highly important publications.

RESULTS

The searches identified 43 relevant publications. The WOS process had 86.05% recall and 1.58% precision. The GS process had 90.7% recall and 1.62% precision. The NNR to identify one relevant publication was 63.3 with WOS and 61.72 with GS. There were nine highly important publications. WOS had 100% recall, 0.38% precision and NNR of 260.22. GS had 88.89% recall, 0.33% precision and NNR of 300.88. Administering the WOS results took 4 h and cost £88-£136, compared with 75 h and £1650-£2550 with GS.

CONCLUSION

WOS is recommended over GS, as citation searching was more effective, while the administrative and screening times and costs were lower.

Extending Treatment Networks in Health Technology Assessment: How Far Should We Go?

BACKGROUND

Network meta-analysis may require substantially more resources than does a standard systematic review. One frequently asked question is "how far should I extend the network and which treatments should I include?"

OBJECTIVE

To explore the increase in precision from including additional evidence.

METHODS

We assessed the benefit of extending treatment networks in terms of precision of effect estimates and examined how this depends on network structure and relative strength of additional evidence. We introduced a "star"-shaped network. Network complexity is increased by adding more evidence connecting treatments under five evidence scenarios. We also examined the impact of heterogeneity and absence of evidence facilitating a "first-order" indirect comparison.

RESULTS

In all scenarios, extending the network increased the precision of the A versus B treatment effect. Under a fixed-effect model, the increase in precision was modest when the existing direct A versus B evidence was already strong and was substantial when the direct evidence was weak. Under a random-effects model, the gain in precision was lower when heterogeneity was high. When evidence is available for all "first-order" indirect comparisons, including second-order evidence has limited benefit for the precision of the A versus B estimate. This is interpreted as a "ceiling effect."

CONCLUSIONS

Including additional evidence increases the precision of a "focal" treatment comparison of interest. Once the comparison of interest is connected to all others via "first-order" indirect evidence, there is no additional benefit in including higher order comparisons. This conclusion is generalizable to any number of treatment comparisons, which would then all be considered "focal." The increase in precision is modest when direct evidence is already strong, or there is a high degree of heterogeneity.

Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking

BACKGROUND

Increasing numbers of systematic reviews evaluating the diagnostic test accuracy of technologies are being published. Currently, review teams tend to apply conventional systematic review standards to identify relevant studies for inclusion, for example sensitive searches of multiple bibliographic databases. There has been little evaluation of the efficiency of searching only one or two such databases for this type of review. The aim of this study was to assess the viability of an approach that restricted searches to MEDLINE, EMBASE and the reference lists of included studies.

METHODS

A convenience sample of nine Health Technology Assessment (HTA) systematic reviews of diagnostic test accuracy, with 302 included citations, was analysed to determine the number and proportion of included citations that were indexed in and retrieved from MEDLINE and EMBASE. An assessment was also made of the number and proportion of citations not retrieved from these databases but that could have been identified from the reference lists of included citations.

RESULTS

287/302 (95 %) of the included citations in the nine reviews were indexed across MEDLINE and EMBASE. The reviews' searches of MEDLINE and EMBASE accounted for 85 % of the included citations (256/302). Of the forty-six (15 %) included citations not retrieved by the published searches, 24 (8 %) could be found in the reference lists of included citations. Only 22/302 (7 %) of the included citations were not found by the proposed, more efficient approach.

CONCLUSIONS

The proposed approach would have accounted for 280/302 (93 %) of included citations in this sample of nine systematic reviews. This exploratory study suggests that there might be a case for restricting searches for systematic reviews of diagnostic test accuracy studies to MEDLINE, EMBASE and the reference lists of included citations. The conduct of such reviews might be rendered more efficient by using this approach.

Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable?

BACKGROUND

In recognising the potential value of theory in understanding how interventions work comes a challenge - how to make identification of theory less haphazard?

OBJECTIVES

To explore the feasibility of systematic identification of theory.

METHOD

We searched PubMed for published reviews (1998-2012) that had explicitly sought to identify theory. Systematic searching may be characterised by a structured question, methodological filters and an itemised search procedure. We constructed a template (BeHEMoTh - Behaviour of interest; Health context; Exclusions; Models or Theories) for use when systematically identifying theory. The authors tested the template within two systematic reviews.

RESULTS

Of 34 systematic reviews, only 12 reviews (35%) reported a method for identifying theory. Nineteen did not specify how they identified studies containing theory. Data were unavailable for three reviews. Candidate terms include concept(s)/conceptual, framework(s), model(s), and theory/theories/theoretical. Information professionals must overcome inadequate reporting and the use of theory out of context. The review team faces an additional concern in lack of 'theory fidelity'.

CONCLUSIONS

Based on experience with two systematic reviews, the BeHEMoTh template and procedure offers a feasible and useful approach for identification of theory. Applications include realist synthesis, framework synthesis or review of complex interventions. The procedure requires rigorous evaluation.

Just how wide should 'wide reading' be?

Educationalists introduce students to literature search strategies that, with rare exceptions, focus chiefly on the location of primary research reports and systematic reviews of those reports. These sources are, however, unlikely to adequately address the normative and/or metaphysical questions that nurses frequently and legitimately interest themselves in. To meet these interests, non-research texts exploring normative and/or metaphysical topics might and perhaps should, in some situations, be deemed suitable search targets. This seems plausible and, moreover, students are encouraged to 'read widely'. Yet accepting this proposition creates significant difficulties. Specifically, if non-research scholarly sources and artistic or literary (humanities) products dealing with normative/metaphysical issues were included in what are, at present, scientifically orientated searches, it is difficult to draw boundaries around what--if anything--is to be excluded. Engaging with this issue highlights problems with qualitative scholarship's designation as 'evidence'. Thus, absurdly, if qualitative scholarship's findings are labelled evidence because they generate practice-relevant understanding/insight, then any literary or artistic artefact (e.g. a throwaway lifestyle magazine) that generates kindred understandings/insights is presumably also evidence? This conclusion is rejected and it is instead proposed that while artistic, literary, and qualitative inquiries can provide practitioners with powerful and stimulating non-evidential understanding, these sources are not evidence as commonly conceived.

Clinical librarian support for rapid review of clinical utility of cancer molecular biomarkers

The clinical librarian used a restricted literature searching and quality-filtering approach to provide relevant clinical evidence for the use of cancer molecular biomarkers by institutional policy makers and clinicians in the rapid review process. The librarian-provided evidence was compared with the cited references in the institutional molecular biomarker algorithm. The overall incorporation rate of the librarian-provided references into the algorithm was above 80%. This study suggests the usefulness of clinical librarian expertise for clinical practice. The searching and filtering methods for high-level evidence can be adopted by information professionals who are involved in the rapid literature review.

Conducting systematic reviews of intervention questions I: Writing the review protocol, formulating the question and searching the literature

This article is the fourth of six articles addressing systematic reviews in animal agriculture and veterinary medicine. Previous articles in the series have introduced systematic reviews, discussed study designs and hierarchies of evidence, and provided details on conducting randomized controlled trials, a common design for use in systematic reviews. This article describes development of a review protocol and the first two steps in a systematic review: formulating a review question, and searching the literature for relevant research. The emphasis is on systematic reviews of questions related to interventions. The review protocol is developed prior to conducting the review and specifies the plan for the conduct of the review, identifies the roles and responsibilities of the review team and provides structured definitions related to the review question. For intervention questions, the review question should be defined by the PICO components: population, intervention, comparison and outcome(s). The literature search is designed to identify all potentially relevant original research that may address the question. Search terms related to some or all of the PICO components are entered into literature databases, and searches for unpublished literature also are conducted. All steps of the literature search are documented to provide transparent reporting of the process.

Efficient strategies to find diagnostic test accuracy studies in kidney journals

AIM

Nephrologists looking for quick answers to diagnostic clinical questions in MEDLINE can use a range of published search strategies or Clinical Query limits to improve the precision of their searches. We aimed to evaluate existing search strategies for finding diagnostic test accuracy studies in nephrology journals.

METHODS

We assessed the accuracy of 14 search strategies for retrieving diagnostic test accuracy studies from three nephrology journals indexed in MEDLINE. Two investigators hand searched the same journals to create a reference set of diagnostic test accuracy studies to compare search strategy results against.

RESULTS

We identified 103 diagnostic test accuracy studies, accounting for 2.1% of all studies published. The most specific search strategy was the Narrow Clinical Queries limit (sensitivity: 0.20, 95% CI 0.13-0.29; specificity: 0.99, 95% CI 0.99-0.99). Using the Narrow Clinical Queries limit, a searcher would need to screen three (95% CI 2-6) articles to find one diagnostic study. The most sensitive search strategy was van der Weijden 1999 Extended (sensitivity: 0.95; 95% CI 0.89-0.98; specificity 0.55, 95% CI 0.53-0.56) but required a searcher to screen 24 (95% CI 23-26) articles to find one diagnostic study. Bachmann 2002 was the best balanced search strategy, which was sensitive (0.88, 95% CI 0.81-0.94), but also specific (0.74, 95% CI 0.73-0.75), with a number needed to screen of 15 (95% CI 14-17).

CONCLUSION

Diagnostic studies are infrequently published in nephrology journals. The addition of a strategy for diagnostic studies to a subject search strategy in MEDLINE may reduce the records needed to screen while preserving adequate search sensitivity for routine clinical use.