1
|
Wafford QE, Miller CH, Wescott AB, Kubilius RK. Meeting a need: development and validation of PubMed search filters for immigrant populations. J Med Libr Assoc 2024; 112:22-32. [PMID: 38911528 PMCID: PMC11189137 DOI: 10.5195/jmla.2024.1716] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/25/2024] Open
Abstract
Objective There is a need for additional comprehensive and validated filters to find relevant references more efficiently in the growing body of research on immigrant populations. Our goal was to create reliable search filters that direct librarians and researchers to pertinent studies indexed in PubMed about health topics specific to immigrant populations. Methods We applied a systematic and multi-step process that combined information from expert input, authoritative sources, automation, and manual review of sources. We established a focused scope and eligibility criteria, which we used to create the development and validation sets. We formed a term ranking system that resulted in the creation of two filters: an immigrant-specific and an immigrant-sensitive search filter. Results When tested against the validation set, the specific filter sensitivity was 88.09%, specificity 97.26%, precision 97.88%, and the NNR 1.02. The sensitive filter sensitivity was 97.76%when tested against the development set. The sensitive filter had a sensitivity of 97.14%, specificity of 82.05%, precision of 88.59%, accuracy of 90.94%, and NNR [See Table 1] of 1.13 when tested against the validation set. Conclusion We accomplished our goal of developing PubMed search filters to help researchers retrieve studies about immigrants. The specific and sensitive PubMed search filters give information professionals and researchers options to maximize the specificity and precision or increase the sensitivity of their search for relevant studies in PubMed. Both search filters generated strong performance measurements and can be used as-is, to capture a subset of immigrant-related literature, or adapted and revised to fit the unique research needs of specific project teams (e.g. remove US-centric language, add location-specific terminology, or expand the search strategy to include terms for the topic/s being investigated in the immigrant population identified by the filter). There is also a potential for teams to employ the search filter development process described here for their own topics and use.
Collapse
Affiliation(s)
- Q Eileen Wafford
- , Research Librarian, Galter Health Sciences Library & Learning Center, Northwestern University Feinberg School of Medicine, Chicago, IL
| | - Corinne H Miller
- , Clinical Informationist, Galter Health Sciences Library & Learning Center, Northwestern University Feinberg School of Medicine, Chicago, IL
| | - Annie B Wescott
- , Research Librarian, Galter Health Sciences Library & Learning Center, Northwestern University Feinberg School of Medicine, Chicago, IL
| | - Ramune K Kubilius
- , Collection Development/Special Projects Librarian, Galter Health Sciences Library & Learning Center, Northwestern University Feinberg School of Medicine, Chicago, IL
| |
Collapse
|
2
|
Toews I, Anglemyer A, Nyirenda JL, Alsaid D, Balduzzi S, Grummich K, Schwingshackl L, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study. Cochrane Database Syst Rev 2024; 1:MR000034. [PMID: 38174786 PMCID: PMC10765475 DOI: 10.1002/14651858.mr000034.pub3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/05/2024]
Abstract
BACKGROUND Researchers and decision-makers often use evidence from randomised controlled trials (RCTs) to determine the efficacy or effectiveness of a treatment or intervention. Studies with observational designs are often used to measure the effectiveness of an intervention in 'real world' scenarios. Numerous study designs and their modifications (including both randomised and observational designs) are used for comparative effectiveness research in an attempt to give an unbiased estimate of whether one treatment is more effective or safer than another for a particular population. An up-to-date systematic analysis is needed to identify differences in effect estimates from RCTs and observational studies. This updated review summarises the results of methodological reviews that compared the effect estimates of observational studies with RCTs from evidence syntheses that addressed the same health research question. OBJECTIVES To assess and compare synthesised effect estimates by study type, contrasting RCTs with observational studies. To explore factors that might explain differences in synthesised effect estimates from RCTs versus observational studies (e.g. heterogeneity, type of observational study design, type of intervention, and use of propensity score adjustment). To identify gaps in the existing research comparing effect estimates across different study types. SEARCH METHODS We searched MEDLINE, the Cochrane Database of Systematic Reviews, Web of Science databases, and Epistemonikos to May 2022. We checked references, conducted citation searches, and contacted review authors to identify additional reviews. SELECTION CRITERIA We included systematic methodological reviews that compared quantitative effect estimates measuring the efficacy or effectiveness of interventions tested in RCTs versus in observational studies. The included reviews compared RCTs to observational studies (including retrospective and prospective cohort, case-control and cross-sectional designs). Reviews were not eligible if they compared RCTs with studies that had used some form of concurrent allocation. DATA COLLECTION AND ANALYSIS Using results from observational studies as the reference group, we examined the relative summary effect estimates (risk ratios (RRs), odds ratios (ORs), hazard ratios (HRs), mean differences (MDs), and standardised mean differences (SMDs)) to evaluate whether there was a relatively larger or smaller effect in the ratio of odds ratios (ROR) or ratio of risk ratios (RRR), ratio of hazard ratios (RHR), and difference in (standardised) mean differences (D(S)MD). If an included review did not provide an estimate comparing results from RCTs with observational studies, we generated one by pooling the estimates for observational studies and RCTs, respectively. Across all reviews, we synthesised these ratios to produce a pooled ratio of ratios comparing effect estimates from RCTs with those from observational studies. In overviews of reviews, we estimated the ROR or RRR for each overview using observational studies as the reference category. We appraised the risk of bias in the included reviews (using nine criteria in total). To receive an overall low risk of bias rating, an included review needed: explicit criteria for study selection, a complete sample of studies, and to have controlled for study methodological differences and study heterogeneity. We assessed reviews/overviews not meeting these four criteria as having an overall high risk of bias. We assessed the certainty of the evidence, consisting of multiple evidence syntheses, with the GRADE approach. MAIN RESULTS We included 39 systematic reviews and eight overviews of reviews, for a total of 47. Thirty-four of these contributed data to our primary analysis. Based on the available data, we found that the reviews/overviews included 2869 RCTs involving 3,882,115 participants, and 3924 observational studies with 19,499,970 participants. We rated 11 reviews/overviews as having an overall low risk of bias, and 36 as having an unclear or high risk of bias. Our main concerns with the included reviews/overviews were that some did not assess the quality of their included studies, and some failed to account appropriately for differences between study designs - for example, they conducted aggregate analyses of all observational studies rather than separate analyses of cohort and case-control studies. When pooling RORs and RRRs, the ratio of ratios indicated no difference or a very small difference between the effect estimates from RCTs versus from observational studies (ratio of ratios 1.08, 95% confidence interval (CI) 1.01 to 1.15). We rated the certainty of the evidence as low. Twenty-three of 34 reviews reported effect estimates of RCTs and observational studies that were on average in agreement. In a number of subgroup analyses, small differences in the effect estimates were detected: - pharmaceutical interventions only (ratio of ratios 1.12, 95% CI 1.04 to 1.21); - RCTs and observational studies with substantial or high heterogeneity; that is, I2 ≥ 50% (ratio of ratios 1.11, 95% CI 1.04 to 1.18); - no use (ratio of ratios 1.07, 95% CI 1.03 to 1.11) or unclear use (ratio of ratios 1.13, 95% CI 1.03 to 1.25) of propensity score adjustment in observational studies; and - observational studies without further specification of the study design (ratio of ratios 1.06, 95% CI 0.96 to 1.18). We detected no clear difference in other subgroup analyses. AUTHORS' CONCLUSIONS We found no difference or a very small difference between effect estimates from RCTs and observational studies. These findings are largely consistent with findings from recently published research. Factors other than study design need to be considered when exploring reasons for a lack of agreement between results of RCTs and observational studies, such as differences in the population, intervention, comparator, and outcomes investigated in the respective studies. Our results underscore that it is important for review authors to consider not only study design, but the level of heterogeneity in meta-analyses of RCTs or observational studies. A better understanding is needed of how these factors might yield estimates reflective of true effectiveness.
Collapse
Affiliation(s)
- Ingrid Toews
- Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center - University of Freiburg, Freiburg, Germany
| | - Andrew Anglemyer
- Department of Preventive and Social Medicine, Dunedin School of Medicine, University of Otago, Dunedin, New Zealand
| | - John Lz Nyirenda
- Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center - University of Freiburg, Freiburg, Germany
| | - Dima Alsaid
- Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center - University of Freiburg, Freiburg, Germany
| | - Sara Balduzzi
- Biometrics Department, Antoni van Leeuwenhoek - Netherlands Cancer Institute, Amsterdam, Netherlands
| | - Kathrin Grummich
- Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center - University of Freiburg, Freiburg, Germany
| | - Lukas Schwingshackl
- Institute for Evidence in Medicine (for Cochrane Germany Foundation), Medical Center - University of Freiburg, Freiburg, Germany
| | - Lisa Bero
- Charles Perkins Centre and School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Camperdown, Sydney, Australia
| |
Collapse
|
3
|
Escobar Liquitay CM, Garegnani L, Garrote V, Solà I, Franco JV. Search strategies (filters) to identify systematic reviews in MEDLINE and Embase. Cochrane Database Syst Rev 2023; 9:MR000054. [PMID: 37681507 PMCID: PMC10485899 DOI: 10.1002/14651858.mr000054.pub2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 09/09/2023]
Abstract
BACKGROUND Bibliographic databases provide access to an international body of scientific literature in health and medical sciences. Systematic reviews are an important source of evidence for clinicians, researchers, consumers, and policymakers as they address a specific health-related question and use explicit methods to identify, appraise and synthesize evidence from which conclusions can be drawn and decisions made. Methodological search filters help database end-users search the literature effectively with different levels of sensitivity and specificity. These filters have been developed for various study designs and have been found to be particularly useful for intervention studies. Other filters have been developed for finding systematic reviews. Considering the variety and number of available search filters for systematic reviews, there is a need for a review of them in order to provide evidence about their retrieval properties at the time they were developed. OBJECTIVES To review systematically empirical studies that report the development, evaluation, or comparison of search filters to retrieve reports of systematic reviews in MEDLINE and Embase. SEARCH METHODS We searched the following databases from inception to January 2023: MEDLINE, Embase, PsycINFO; Library, Information Science & Technology Abstracts (LISTA) and Science Citation Index (Web of Science). SELECTION CRITERIA We included studies if one of their primary objectives is the development, evaluation, or comparison of a search filter that could be used to retrieve systematic reviews on MEDLINE, Embase, or both. DATA COLLECTION AND ANALYSIS Two review authors independently extracted data using a pre-specified and piloted data extraction form using InterTASC Information Specialist Subgroup (ISSG) Search Filter Evaluation Checklist. MAIN RESULTS We identified eight studies that developed filters for MEDLINE and three studies that developed filters for Embase. Most studies are very old and some were limited to systematic reviews in specific clinical areas. Six included studies reported the sensitivity of their developed filter. Seven studies reported precision and six studies reported specificity. Only one study reported the number needed to read and positive predictive value. None of the filters were designed to differentiate systematic reviews on the basis of their methodological quality. For MEDLINE, all filters showed similar sensitivity and precision, and one filter showed higher levels of specificity. For Embase, filters showed variable sensitivity and precision, with limited study reports that may affect accuracy assessments. The report of these studies had some limitations, and the assessments of their accuracy may suffer from indirectness, considering that they were mostly developed before the release of the PRISMA 2009 statement or due to their limited scope in the selection of systematic review topics. Search filters for MEDLINE Three studies produced filters with sensitivity > 90% with variable degrees of precision, and only one of them was developed and validated in a gold-standard database, which allowed the calculation of specificity. The other two search filters had lower levels of sensitivity. One of these produced a filter with higher levels of specificity (> 90%). All filters showed similar sensitivity and precision in the external validation, except for one which was not externally validated and another one which was conceptually derived and only externally validated. Search filters for Embase We identified three studies that developed filters for this database. One of these studies developed filters with variable sensitivity and precision, including highly sensitive strategies (> 90%); however, it was not externally validated. The other study produced a filter with a lower sensitivity (72.7%) but high specificity (99.1%) with a similar performance in the external validation. AUTHORS' CONCLUSIONS Studies reporting the development, evaluation, or comparison of search filters to retrieve reports of systematic reviews in MEDLINE showed similar sensitivity and precision, with one filter showing higher levels of specificity. For Embase, filters showed variable sensitivity and precision, with limited information about how the filter was produced, which leaves us uncertain about their performance assessments. Newer filters had limitations in their methods or scope, including very focused subject topics for their gold standards, limiting their applicability across other topics. Our findings highlight that consensus guidance on the conduct of search filters and standardized reporting of search filters are needed, as we found highly heterogeneous development methods, accuracy assessments and outcome selection. New strategies adaptable across interfaces could enhance their usability. Moreover, the performance of existing filters needs to be evaluated in light of the impact of reporting guidelines, including the PRISMA 2009, on how systematic reviews are reported. Finally, future filter developments should also consider comparing the filters against a common reference set to establish comparative performance and assess the quality of systematic reviews retrieved by strategies.
Collapse
Affiliation(s)
| | - Luis Garegnani
- Research Department, Instituto Universitario Hospital Italiano de Buenos Aires, Buenos Aires, Argentina
| | - Virginia Garrote
- Central Library, Instituto Universitario Hospital Italiano de Buenos Aires, Buenos Aires, Argentina
| | - Ivan Solà
- Iberoamerican Cochrane Centre, Biomedical Research Institute Sant Pau (IIB Sant Pau), CIBER Epidemiología y Salud Pública (CIBERESP), Barcelona, Spain
| | - Juan Va Franco
- Institute of General Practice, Medical Faculty of the Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany
| |
Collapse
|
4
|
Chuachamsai S, Acharya A, Fischer K, Nibali L, Ho D, Pelekos G. The effectiveness of adjunctive measures in managing peri-implant mucositis: an umbrella review. Int J Implant Dent 2022; 8:26. [PMID: 35674882 PMCID: PMC9177933 DOI: 10.1186/s40729-022-00426-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Accepted: 05/30/2022] [Indexed: 11/28/2022] Open
Abstract
OBJECTIVES The purpose of this umbrella review was to gather and summarize the data from published systematic reviews (SRs) that compared non-surgical mechanical debridement (NSMD) with and without the use of adjunctive treatments on the management of peri-implant mucositis (PIM). MATERIALS AND METHODS A protocol was developed and registered in PROSPERO (CRD42021254350) before the systematic search for the SRs. Seven electronic databases, including Cochrane Library, Embase (via Ovid), MEDLINE (via Pubmed), Proquest, Prospero, Scopus and Web of Science, were searched for published reviews. The search for unpublished and informally published reviews was further attempted in the last four databases. The methodological quality of the included reviews was assessed using AMSTAR 2. RESULTS Twelve included SRs assessed clinical studies published between 2014 and 2020, including a total of seventeen primary clinical trials. All SRs summarized data from individual studies and provided a narrative conclusion regarding the effectiveness of the adjunctive treatments. Only six SRs performed a meta-analysis (MA) of additional benefits of the adjunctive therapy for PIM, with results indicating no significant difference between the different treatment modalities. The overall confidence was adjudged ranging from critically low to low using AMSTAR 2 and significant additional benefits of any adjunctive treatments in comparison with NSMD were not apparent. CONCLUSION Overall, the reviewed evidence did not support the use of adjunctive treatments for improvement of clinical outcomes in PM management as compared to NSMD alone.
Collapse
Affiliation(s)
- Sompol Chuachamsai
- Division of Periodontology and Implant Dentistry Faculty of Dentistry, The University of Hong Kong, Hong Kong, SAR, China
| | - Aneesha Acharya
- Dr D. Y. Patil Dental College and Hospital, Dr D. Y. Patil Vidyapeeth, Pune, India
- Division of Periodontology and Implant Dentistry, Faculty of Dentistry, The University of Hong Kong, Hong Kong, SAR, China
| | - Kai Fischer
- Clinic of Conservative and Preventive Dentistry, Division of Periodontology and Peri-Implant Diseases, University of Zurich, Zurich, Switzerland
| | - Luigi Nibali
- Periodontology Unit, Faculty of Dentistry, Oral and Craniofacial Sciences, Centre for Host-Microbiome Interactions, King's College London, London, UK
| | - Dominic Ho
- Division of Periodontology and Implant Dentistry, Faculty of Dentistry, The University of Hong Kong, Hong Kong, SAR, China
| | - Georgios Pelekos
- Division of Periodontology and Implant Dentistry, Faculty of Dentistry, The University of Hong Kong, Hong Kong, SAR, China.
- Department of Periodontology, Faculty of Dentistry, The University of Hong Kong, Prince Philip Dental Hospital, 34 Hospital Road, Sai Ying Pun, Hong Kong, 999077, SAR, China.
| |
Collapse
|
5
|
Salvador-Oliván JA, Marco-Cuenca G, Arquero-Avilés R. Development of an efficient search filter to retrieve systematic reviews from PubMed. J Med Libr Assoc 2021; 109:561-574. [PMID: 34858085 PMCID: PMC8608217 DOI: 10.5195/jmla.2021.1223] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022] Open
Abstract
Objective: Locating systematic reviews is essential for clinicians and researchers when creating or updating reviews and for decision-making in health care. This study aimed to develop a search filter for retrieving systematic reviews that improves upon the performance of the PubMed systematic review search filter. Methods: Search terms were identified from abstracts of reviews published in Cochrane Database of Systematic Reviews and the titles of articles indexed as systematic reviews in PubMed. Both the precision of the candidate terms and the number of systematic reviews retrieved from PubMed were evaluated after excluding the subset of articles retrieved by the PubMed systematic review filter. Terms that achieved a precision greater than 70% and relevant publication types indexed with MeSH terms were included in the filter search strategy. Results: The search strategy used in our filter added specific terms not included in PubMed's systematic review filter and achieved a 61.3% increase in the number of retrieved articles that are potential systematic reviews. Moreover, it achieved an average precision that is likely greater than 80%. Conclusions: The developed search filter will enable users to identify more systematic reviews from PubMed than the PubMed systematic review filter with high precision.
Collapse
Affiliation(s)
| | - Gonzalo Marco-Cuenca
- , Professor, School of Medicine, Department of Library and Information Science, University of Zaragoza, Spain
| | - Rosario Arquero-Avilés
- , Professor, Department of Library and Information Science, Complutense University of Madrid, Madrid, Spain
| |
Collapse
|
6
|
Ayiku L, Hudson T, Williams C, Levay P, Jacob C. The NICE OECD countries' geographic search filters: Part 2-validation of the MEDLINE and Embase (Ovid) filters. J Med Libr Assoc 2021; 109:583-589. [PMID: 34858087 PMCID: PMC8608218 DOI: 10.5195/jmla.2021.1224] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
Objective: We previously developed draft MEDLINE and Embase (Ovid) geographic search filters for Organisation for Economic Co-operation and Development (OECD) countries to assess their feasibility for finding evidence about the countries. Here, we describe the validation of these search filters. Methods: We identified OECD country references from thirty National Institute for Health and Care Excellence (NICE) guidelines to generate gold standard sets for MEDLINE (n=2,065) and Embase (n=2,023). We validated the filters by calculating their recall against these sets. We then applied the filters to existing search strategies for three OECD-focused NICE guideline reviews (NG103 on flu vaccination, NG140 on abortion care, and NG146 on workplace health) to calculate the filters' impact on the number needed to read (NNR) of the searches. Results: The filters both achieved 99.95% recall against the gold standard sets. Both filters achieved 100% recall for the three NICE guideline reviews. The MEDLINE filter reduced NNR from 256 to 232 for the NG103 review, from 38 to 27 for the NG140 review, and from 631 to 591 for the NG146 review. The Embase filter reduced NNR from 373 to 341 for the NG103 review, from 101 to 76 for the NG140 review, and from 989 to 925 for the NG146 review. Conclusion: The NICE OECD countries' search filters are the first validated filters for the countries. They can save time for research topics about OECD countries by finding the majority of evidence about OECD countries while reducing search result volumes in comparison to no filter use.
Collapse
Affiliation(s)
- Lynda Ayiku
- , Information Specialist, NICE Information Services team, National Institute for Health and Care Excellence, Manchester, UK
| | - Thomas Hudson
- , Information Specialist, NICE Information Services team, National Institute for Health and Care Excellence, Manchester, UK
| | - Ceri Williams
- , Information Specialist, NICE Information Services team, National Institute for Health and Care Excellence, Manchester, UK
| | - Paul Levay
- , Information Specialist, NICE Information Services team, National Institute for Health and Care Excellence, Manchester, UK
| | - Catherine Jacob
- , Information Specialist, NICE Information Services team, National Institute for Health and Care Excellence, Manchester, UK
| |
Collapse
|