Study filters for non-randomized studies of interventions consistently lacked sensitivity upon external validation

BACKGROUND

Little evidence is available on searches for non-randomized studies (NRS) in bibliographic databases within the framework of systematic reviews. For instance, it is currently unclear whether, when searching for NRS, effective restriction of the search strategy to certain study types is possible. The following challenges need to be considered: 1) For non-randomized controlled trials (NRCTs): whether they can be identified by established filters for randomized controlled trials (RCTs). 2) For other NRS types (such as cohort studies): whether study filters exist for each study type and, if so, which performance measures they have. The aims of the present analysis were to identify and validate existing NRS filters in MEDLINE as well as to evaluate established RCT filters using a set of MEDLINE citations.

METHODS

Our analysis is a retrospective analysis of study filters based on MEDLINE citations of NRS from Cochrane reviews. In a first step we identified existing NRS filters. For the generation of the reference set, we screened Cochrane reviews evaluating NRS, which covered a broad range of study types. The citations of the studies included in the Cochrane reviews were identified via the reviews' bibliographies and the corresponding PubMed identification numbers (PMIDs) were extracted from PubMed. Random samples comprising up to 200 citations (i.e. 200 PMIDs) each were created for each study type to generate the test sets.

RESULTS

A total of 271 Cochrane reviews from 41 different Cochrane groups were eligible for data extraction. We identified 14 NRS filters published since 2001. The study filters generated between 660,000 and 9.5 million hits in MEDLINE. Most filters covered several study types. The reference set included 2890 publications classified as NRS for the generation of the test sets. Twelve test sets were generated (one for each study type), of which 8 included 200 citations each. None of the study filters achieved sufficient sensitivity (≥ 92%) for all of the study types targeted.

CONCLUSIONS

The performance of current NRS filters is insufficient for effective use in daily practice. It is therefore necessary to develop new strategies (e.g. new NRS filters in combination with other search techniques). The challenges related to NRS should be taken into account.

Study filters for non-randomized studies of interventions consistently lacked sensitivity upon external validation

BACKGROUND

Little evidence is available on searches for non-randomized studies (NRS) in bibliographic databases within the framework of systematic reviews. For instance, it is currently unclear whether, when searching for NRS, effective restriction of the search strategy to certain study types is possible. The following challenges need to be considered: 1) For non-randomized controlled trials (NRCTs): whether they can be identified by established filters for randomized controlled trials (RCTs). 2) For other NRS types (such as cohort studies): whether study filters exist for each study type and, if so, which performance measures they have. The aims of the present analysis were to identify and validate existing NRS filters in MEDLINE as well as to evaluate established RCT filters using a set of MEDLINE citations.

METHODS

Our analysis is a retrospective analysis of study filters based on MEDLINE citations of NRS from Cochrane reviews. In a first step we identified existing NRS filters. For the generation of the reference set, we screened Cochrane reviews evaluating NRS, which covered a broad range of study types. The citations of the studies included in the Cochrane reviews were identified via the reviews' bibliographies and the corresponding PubMed identification numbers (PMIDs) were extracted from PubMed. Random samples comprising up to 200 citations (i.e. 200 PMIDs) each were created for each study type to generate the test sets.

RESULTS

A total of 271 Cochrane reviews from 41 different Cochrane groups were eligible for data extraction. We identified 14 NRS filters published since 2001. The study filters generated between 660,000 and 9.5 million hits in MEDLINE. Most filters covered several study types. The reference set included 2890 publications classified as NRS for the generation of the test sets. Twelve test sets were generated (one for each study type), of which 8 included 200 citations each. None of the study filters achieved sufficient sensitivity (≥ 92%) for all of the study types targeted.

CONCLUSIONS

The performance of current NRS filters is insufficient for effective use in daily practice. It is therefore necessary to develop new strategies (e.g. new NRS filters in combination with other search techniques). The challenges related to NRS should be taken into account.

Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study

BACKGROUND

Within systematic reviews, when searching for relevant references, it is advisable to use multiple databases. However, searching databases is laborious and time-consuming, as syntax of search strategies are database specific. We aimed to determine the optimal combination of databases needed to conduct efficient searches in systematic reviews and whether the current practice in published reviews is appropriate. While previous studies determined the coverage of databases, we analyzed the actual retrieval from the original searches for systematic reviews.

METHODS

Since May 2013, the first author prospectively recorded results from systematic review searches that he performed at his institution. PubMed was used to identify systematic reviews published using our search strategy results. For each published systematic review, we extracted the references of the included studies. Using the prospectively recorded results and the studies included in the publications, we calculated recall, precision, and number needed to read for single databases and databases in combination. We assessed the frequency at which databases and combinations would achieve varying levels of recall (i.e., 95%). For a sample of 200 recently published systematic reviews, we calculated how many had used enough databases to ensure 95% recall.

RESULTS

A total of 58 published systematic reviews were included, totaling 1746 relevant references identified by our database searches, while 84 included references had been retrieved by other search methods. Sixteen percent of the included references (291 articles) were only found in a single database; Embase produced the most unique references (n = 132). The combination of Embase, MEDLINE, Web of Science Core Collection, and Google Scholar performed best, achieving an overall recall of 98.3 and 100% recall in 72% of systematic reviews. We estimate that 60% of published systematic reviews do not retrieve 95% of all available relevant references as many fail to search important databases. Other specialized databases, such as CINAHL or PsycINFO, add unique references to some reviews where the topic of the review is related to the focus of the database.

CONCLUSIONS

Optimal searches in systematic reviews should search at least Embase, MEDLINE, Web of Science, and Google Scholar as a minimum requirement to guarantee adequate and efficient coverage.

Information Resources for the Exotic Animal Practitioner

An essential component of evidence-based practice is finding the best available evidence to answer a clinical question. Finding evidence is difficult for veterinarians in general, and exotic animal clinicians in particular, owing to the lack of studies that provide a high level of clinically relevant evidence and limited access to resources. Knowing where and how to search for evidence can facilitate evidence-based practice.

No suitable precise or optimized epidemiologic search filters were available for bibliographic databases

OBJECTIVES

To determine a suitable approach to a systematic search for epidemiologic publications in bibliographic databases. For this purpose, suitable sensitive, precise, and optimized filters were to be selected for MEDLINE searches. In addition, the relevance of bibliographic databases was determined.

STUDY DESIGN AND SETTING

Epidemiologic systematic reviews (SRs) retrieved in a systematic search and company dossiers were screened to identify epidemiologic publications (primary studies and SRs) published since 2007. These publications were used to generate a test and validation set. Furthermore, each SR's search strategy was reviewed, and epidemiologic filters were extracted. The search syntaxes were validated using the relative recall method.

RESULTS

The test set comprises 729 relevant epidemiologic publications, of which 566 were MEDLINE-indexed. About 27 epidemiologic filters were extracted. One suitable sensitive filter was identified (Larney et al. 2013: 95.94% sensitivity). Precision was presumably underestimated so that no precise or optimized filters can be recommended. About 77.64% of the publications were found in MEDLINE.

CONCLUSION

There is currently no suitable approach to conducting efficient systematic searches for epidemiologic publications in bibliographic databases. The filter by Larney et al. (2013) can be used for sensitive MEDLINE searches. No robust conclusions can be drawn on precise or optimized filters. Additional search approaches should be considered.

All Information Is Not Equal: Using the Literature Databases PubMed and The Cochrane Library for Identifying the Evidence on Granulocyte Transfusion Therapy

To be able to take well-informed decisions or carry out sound research, clinicians and researchers alike require specific information seeking skills matching their respective information needs. Biomedical information is traditionally available via different literature databases. This article gives an introduction to two diverging sources, PubMed (23 million references) and The Cochrane Library (800,000 references), both of which offer sophisticated instruments for searching an increasing amount of medical publications of varied quality and ambition. Whereas PubMed as an unfiltered source of primary literature comprises all different kinds of publication types occurring in academic journals, The Cochrane Library is a pre-filtered source which offers access to either synthesized publication types or critically appraised and carefully selected references. A search approach has to be carried out deliberately and requires a good knowledge on the scope and features of the databases as well as on the ability to build a search strategy in a structured way. We present a specific and a sensitive search approach, making use of both databases within two application case scenarios in order to identify the evidence on granulocyte transfusions for infections in adult patients with neutropenia.

Journal metrics-based position of diabetes & metabolism journal after the change of its text language to english

After changing its language from Korean or English to English only in 2010, the journal metrics of Diabetes & Metabolism Journal (DMJ) were analyzed to assess whether this change in the journal policy was successful. The journal metric items that were analyzed were the following: impact factor; total citations; countries of authors; proportion of the articles funded out of the total number of original articles; and Hirsch-index (H-index). A retrospective, descriptive analysis was carried out using various databases, such as KoreaMed, Korean Medical Citation Index (KoMCI), KoreaMed Synapse, Web of Science, and Journal Citation Ranking. The journal's impact factor was 2.054, which corresponds to 83/122 (68.0%) out of the 2012 JCR endocrinology and metabolism category. The number of the journal's total citations was 330 in 2013. In addition to Korean authors, authors from 13 other countries published papers in the journal from 2010 to 2013. The number of funded papers from 2010 to 2013 was 65 out of 148 original articles (43.9%). The journal's H-index from KoreaMed Synapse was 7, and that from Web of Science was 9. It can be concluded that changing the journal's language to English was successful based on journal metrics. DMJ is currently positioned as an international journal based on the international diversity of authors and editors, its sufficiently high proportion of funded articles, its relatively high impact factor, and the number of total citations.