1
|
Morel T, Nguyen-Soenen J, Thompson W, Fournier JP. Development and validation of search filters to retrieve medication discontinuation articles in Medline and Embase. Health Info Libr J 2024; 41:156-165. [PMID: 38013506 DOI: 10.1111/hir.12516] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Revised: 08/08/2023] [Accepted: 11/10/2023] [Indexed: 11/29/2023]
Abstract
BACKGROUND Medication discontinuation studies explore the outcomes of stopping a medication compared to continuing it. Comprehensively identifying medication discontinuation articles in bibliographic databases remains challenging due to variability in terminology. OBJECTIVES To develop and validate search filters to retrieve medication discontinuation articles in Medline and Embase. METHODS We identified medication discontinuation articles in a convenience sample of systematic reviews. We used primary articles to create two reference sets for Medline and Embase, respectively. The reference sets were equally divided by randomization in development sets and validation sets. Terms relevant for discontinuation were identified by term frequency analysis in development sets and combined to develop two search filters that maximized relative recalls. The filters were validated against validation sets. Relative recalls were calculated with their 95% confidences intervals (95% CI). RESULTS We included 316 articles for Medline and 407 articles for Embase, from 15 systematic reviews. The Medline optimized search filter combined 7 terms. The Embase optimized search filter combined 8 terms. The relative recalls were respectively 92% (95% CI: 87-96) and 91% (95% CI: 86-94). CONCLUSIONS We developed two search filters for retrieving medication discontinuation articles in Medline and Embase. Further research is needed to estimate precision and specificity of the filters.
Collapse
Affiliation(s)
- Thomas Morel
- Département de Médecine Générale, Nantes Université, Nantes, France
- SPHERE-UMR INSERM 1246, Nantes Université, Université de Tours, Nantes, France
| | - Jérôme Nguyen-Soenen
- Département de Médecine Générale, Nantes Université, Nantes, France
- SPHERE-UMR INSERM 1246, Nantes Université, Université de Tours, Nantes, France
| | - Wade Thompson
- Department of Anesthesiology, Pharmacology, and Therapeutics, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| | - Jean-Pascal Fournier
- Département de Médecine Générale, Nantes Université, Nantes, France
- SPHERE-UMR INSERM 1246, Nantes Université, Université de Tours, Nantes, France
| |
Collapse
|
2
|
Morel T, Heinrich CH, Zerah L, Hurley E, Christiaens A, Fournier JP. Use of deprescribing search filters in systematic review search strategies: A case study. Basic Clin Pharmacol Toxicol 2024; 134:116-120. [PMID: 37264997 DOI: 10.1111/bcpt.13908] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Revised: 05/29/2023] [Accepted: 05/30/2023] [Indexed: 06/03/2023]
Abstract
Two deprescribing search filters for MEDLINE and one deprescribing search filter for Embase have been recently developed, including objectively developed search filters. The objective of this case study was to implement these three deprescribing search filters in systematic review (SR) search strategies and to assess their effect on performances. SR that independently developed original search strategies (OSS) were selected. The deprescribing filters were implemented in each OSS, generating two implemented search strategies (ISS1 and ISS2) in MEDLINE and one ISS (ISS3) in Embase. OSS were re-run on the same date as ISS. The performances of ISS and OSS were calculated and compared. Two SR were included (SR1 and SR2). For MEDLINE, SR1 included 12 articles. The sensitivity was 50% for OSS, 58% for ISS1 and 42% for ISS2. SR2 included four articles. The sensitivity of OSS, ISS 1 and 2 was 25%. For Embase, SR1 included 12 articles. The sensitivity was 33% for OSS and 58% for ISS3. SR2 included four articles. None of the four included articles were retrieved with OSS or ISS3. While sensitivity of OSS was moderate, the objectively developed deprescribing filters maintained or slightly improved this sensitivity when implementing.
Collapse
Affiliation(s)
- Thomas Morel
- Département de Médecine Générale, Nantes Université, Nantes, France
- SPHERE, UMR INSERM 1246, Nantes Université, Université de Tours, Nantes, France
| | - Clara H Heinrich
- Pharmaceutical Care Research Group, University College Cork, Cork, Ireland
| | - Lorène Zerah
- Département de Gériatrie, Assistance Publique-Hôpitaux de Paris (AP-HP), Hôpital Pitié-Salpêtrière, Paris, France
- INSERM, Institut Pierre Louis d'Épidémiologie et de Santé Publique, Sorbonne Université, Paris, France
| | - Eoin Hurley
- Pharmaceutical Care Research Group, University College Cork, Cork, Ireland
| | - Antoine Christiaens
- INSERM, Institut Pierre Louis d'Épidémiologie et de Santé Publique, Sorbonne Université, Paris, France
- Fund for Scientific Research - FNRS, Brussels, Belgium
- Clinical pharmacy research group (CLIP), Louvain Drug Research Institute (LDRI), Université catholique de Louvain, Brussels, Belgium
| | - Jean-Pascal Fournier
- Département de Médecine Générale, Nantes Université, Nantes, France
- SPHERE, UMR INSERM 1246, Nantes Université, Université de Tours, Nantes, France
| |
Collapse
|
3
|
Escobar Liquitay CM, Garegnani L, Garrote V, Solà I, Franco JV. Search strategies (filters) to identify systematic reviews in MEDLINE and Embase. Cochrane Database Syst Rev 2023; 9:MR000054. [PMID: 37681507 PMCID: PMC10485899 DOI: 10.1002/14651858.mr000054.pub2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 09/09/2023]
Abstract
BACKGROUND Bibliographic databases provide access to an international body of scientific literature in health and medical sciences. Systematic reviews are an important source of evidence for clinicians, researchers, consumers, and policymakers as they address a specific health-related question and use explicit methods to identify, appraise and synthesize evidence from which conclusions can be drawn and decisions made. Methodological search filters help database end-users search the literature effectively with different levels of sensitivity and specificity. These filters have been developed for various study designs and have been found to be particularly useful for intervention studies. Other filters have been developed for finding systematic reviews. Considering the variety and number of available search filters for systematic reviews, there is a need for a review of them in order to provide evidence about their retrieval properties at the time they were developed. OBJECTIVES To review systematically empirical studies that report the development, evaluation, or comparison of search filters to retrieve reports of systematic reviews in MEDLINE and Embase. SEARCH METHODS We searched the following databases from inception to January 2023: MEDLINE, Embase, PsycINFO; Library, Information Science & Technology Abstracts (LISTA) and Science Citation Index (Web of Science). SELECTION CRITERIA We included studies if one of their primary objectives is the development, evaluation, or comparison of a search filter that could be used to retrieve systematic reviews on MEDLINE, Embase, or both. DATA COLLECTION AND ANALYSIS Two review authors independently extracted data using a pre-specified and piloted data extraction form using InterTASC Information Specialist Subgroup (ISSG) Search Filter Evaluation Checklist. MAIN RESULTS We identified eight studies that developed filters for MEDLINE and three studies that developed filters for Embase. Most studies are very old and some were limited to systematic reviews in specific clinical areas. Six included studies reported the sensitivity of their developed filter. Seven studies reported precision and six studies reported specificity. Only one study reported the number needed to read and positive predictive value. None of the filters were designed to differentiate systematic reviews on the basis of their methodological quality. For MEDLINE, all filters showed similar sensitivity and precision, and one filter showed higher levels of specificity. For Embase, filters showed variable sensitivity and precision, with limited study reports that may affect accuracy assessments. The report of these studies had some limitations, and the assessments of their accuracy may suffer from indirectness, considering that they were mostly developed before the release of the PRISMA 2009 statement or due to their limited scope in the selection of systematic review topics. Search filters for MEDLINE Three studies produced filters with sensitivity > 90% with variable degrees of precision, and only one of them was developed and validated in a gold-standard database, which allowed the calculation of specificity. The other two search filters had lower levels of sensitivity. One of these produced a filter with higher levels of specificity (> 90%). All filters showed similar sensitivity and precision in the external validation, except for one which was not externally validated and another one which was conceptually derived and only externally validated. Search filters for Embase We identified three studies that developed filters for this database. One of these studies developed filters with variable sensitivity and precision, including highly sensitive strategies (> 90%); however, it was not externally validated. The other study produced a filter with a lower sensitivity (72.7%) but high specificity (99.1%) with a similar performance in the external validation. AUTHORS' CONCLUSIONS Studies reporting the development, evaluation, or comparison of search filters to retrieve reports of systematic reviews in MEDLINE showed similar sensitivity and precision, with one filter showing higher levels of specificity. For Embase, filters showed variable sensitivity and precision, with limited information about how the filter was produced, which leaves us uncertain about their performance assessments. Newer filters had limitations in their methods or scope, including very focused subject topics for their gold standards, limiting their applicability across other topics. Our findings highlight that consensus guidance on the conduct of search filters and standardized reporting of search filters are needed, as we found highly heterogeneous development methods, accuracy assessments and outcome selection. New strategies adaptable across interfaces could enhance their usability. Moreover, the performance of existing filters needs to be evaluated in light of the impact of reporting guidelines, including the PRISMA 2009, on how systematic reviews are reported. Finally, future filter developments should also consider comparing the filters against a common reference set to establish comparative performance and assess the quality of systematic reviews retrieved by strategies.
Collapse
Affiliation(s)
| | - Luis Garegnani
- Research Department, Instituto Universitario Hospital Italiano de Buenos Aires, Buenos Aires, Argentina
| | - Virginia Garrote
- Central Library, Instituto Universitario Hospital Italiano de Buenos Aires, Buenos Aires, Argentina
| | - Ivan Solà
- Iberoamerican Cochrane Centre, Biomedical Research Institute Sant Pau (IIB Sant Pau), CIBER Epidemiología y Salud Pública (CIBERESP), Barcelona, Spain
| | - Juan Va Franco
- Institute of General Practice, Medical Faculty of the Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany
| |
Collapse
|
4
|
Méndez-Sánchez L, Clark P, Winzenberg TM, Tugwell P, Correa-Burrows P, Costello R. Calcium and vitamin D for increasing bone mineral density in premenopausal women. Cochrane Database Syst Rev 2023; 1:CD012664. [PMID: 36705288 PMCID: PMC9881395 DOI: 10.1002/14651858.cd012664.pub2] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/28/2023]
Abstract
BACKGROUND Osteoporosis is a condition where bones become fragile due to low bone density and impaired bone quality. This results in fractures that lead to higher morbidity and reduced quality of life. Osteoporosis is considered a major public health concern worldwide. For this reason, preventive measurements need to be addressed throughout the life course. Exercise and a healthy diet are among the lifestyle factors that can help prevent the disease, the latter including intake of key micronutrients for bone, such as calcium and vitamin D. The evidence on whether supplementation with calcium and vitamin D improves bone mineral density (BMD) in premenopausal women is still inconclusive. In this age group, bone accrual is considered to be the goal of supplementation, so BMD is relevant for the future stages of life. OBJECTIVES To evaluate the benefits and harms of calcium and vitamin D supplementation, alone or in combination, to increase the BMD, reduce fractures, and report the potential adverse events in healthy premenopausal women compared to placebo. SEARCH METHODS We used standard, extensive Cochrane search methods. The latest search was 12 April 2022. SELECTION CRITERIA We included randomised controlled trials in healthy premenopausal women (with or without calcium or vitamin D deficiency) comparing supplementation of calcium or vitamin D (or both) at any dose and by any route of administration versus placebo for at least three months. Vitamin D could have been administered as cholecalciferol (vitamin D3) or ergocalciferol (vitamin D2). DATA COLLECTION AND ANALYSIS We used standard Cochrane methods. Outcomes included total hip bone mineral density (BMD), lumbar spine BMD, quality of life, new symptomatic vertebral fractures, new symptomatic non-vertebral fractures, withdrawals due to adverse events, serious adverse events, all reported adverse events and additional withdrawals for any reason. MAIN RESULTS We included seven RCTs with 941 participants, of whom 138 were randomised to calcium supplementation, 110 to vitamin D supplementation, 271 to vitamin D plus calcium supplementation, and 422 to placebo. Mean age ranged from 18.1 to 42.1 years. Studies reported results for total hip or lumbar spine BMD (or both) and withdrawals for various reasons, but none reported fractures or withdrawals for adverse events or serious adverse events. Results for the reported outcomes are presented for the three comparisons: calcium versus placebo, vitamin D versus placebo, and calcium plus vitamin D versus placebo. In all comparisons, there was no clinical difference in outcomes, and the certainty of the evidence was moderate to low. Most studies were at risk of selection, performance, detection, and reporting biases. Calcium versus placebo Four studies compared calcium versus placebo (138 participants in the calcium group and 123 in the placebo group) with mean ages from 18.0 to 47.3 years. Calcium supplementation may have little to no effect on total hip or lumbar spine BMD after 12 months in three studies and after six months in one study (total hip BMD: mean difference (MD) -0.04 g/cm2, 95% confidence interval (CI) -0.11 to 0.03; I2 = 71%; 3 studies, 174 participants; low-certainty evidence; lumbar spine BMD: MD 0 g/cm2, 95% CI -0.06 to 0.06; I2 = 71%; 4 studies, 202 participants; low-certainty evidence). Calcium alone supplementation does not reduce or increase the withdrawals in the trials (risk ratio (RR) 0.78, 95% CI 0.52 to 1.16; I2 = 0%; 4 studies, 261 participants: moderate-certainty evidence). Vitamin D versus placebo Two studies compared vitamin D versus placebo (110 participants in the vitamin D group and 79 in the placebo group), with mean ages from 18.0 to 32.7 years. These studies reported lumbar spine BMD as a mixture of MDs and percent of change and we were unable to pool the results. In the original studies, there were no differences in lumbar BMD between groups. Vitamin D alone supplementation does not reduce or increase withdrawals for any reason between groups (RR 0.74, 95% CI 0.46 to 1.19; moderate-certainty evidence). Calcium plus vitamin D versus placebo Two studies compared calcium plus vitamin D versus placebo (271 participants in the calcium plus vitamin D group and 270 in the placebo group; 220 participants from Woo 2007 and 50 participants from Islam 2010). The mean age range was 18.0 to 36 years. These studies measured different anatomic areas, one study reported total hip BMD and the other study reported lumbar spine BMD; therefore, data were not pooled for this outcome. The individual studies found no difference between groups in percent of change on total hip BMD (-0.03, 95% CI -0.06 to 0; moderate-certainty evidence), and lumbar spine BMD (MD 0.01, 95% CI -0.01 to 0.03; moderate-certainty evidence). Calcium plus vitamin D supplementation may not reduce or increase withdrawals for any reason (RR 0.82, 95% CI 0.29 to 2.35; I2 = 72%; 2 studies, 541 participants; low-certainty evidence). AUTHORS' CONCLUSIONS Our results do not support the isolated or combined use of calcium and vitamin D supplementation in healthy premenopausal women as a public health intervention to improve BMD in the total hip or lumbar spine, and therefore it is unlikely to have a benefit for the prevention of fractures (vertebral and non-vertebral). The evidence found suggests that there is no need for future studies in the general population of premenopausal women; however, studies focused on populations with a predisposition to diseases related to bone metabolism, or with low bone mass or osteoporosis diagnosed BMD would be useful.
Collapse
Affiliation(s)
- Lucía Méndez-Sánchez
- Clinical Epidemiology Unit, Children's Hospital of Mexico Federico Gomez-Faculty of Medicine UNAM, Mexico City, Mexico
- Cochrane Mexico UNAM (Universidad Nacional Autónoma de México), Cochrane Mexico, Mexico City, Mexico
| | - Patricia Clark
- Clinical Epidemiology Unit, Children's Hospital of Mexico Federico Gomez-Faculty of Medicine UNAM, Mexico City, Mexico
- Cochrane Mexico UNAM (Universidad Nacional Autónoma de México), Cochrane Mexico, Mexico City, Mexico
| | - Tania M Winzenberg
- Menzies Institute for Medical Research and Faculty of Health, University of Tasmania, Hobart, Australia
| | - Peter Tugwell
- Department of Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada
| | - Paulina Correa-Burrows
- Department of Human Nutrition, Institute of Nutrition & Food Technology, University of Chile, Santiago de Chile, Chile
| | - Rebecca Costello
- Office of Dietary Supplements, National Institutes of Health, Bethesda, Maryland, USA
| |
Collapse
|
5
|
Morel T, Nguyen-Soenen J, Thompson W, Fournier JP. Development and validation of search filters to identify articles on deprescribing in Medline and Embase. BMC Med Res Methodol 2022; 22:79. [PMID: 35337283 PMCID: PMC8953136 DOI: 10.1186/s12874-022-01515-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Accepted: 12/20/2021] [Indexed: 11/23/2022] Open
Abstract
Background Deprescribing literature has been increasing rapidly. Our aim was to develop and validate search filters to identify articles on deprescribing in Medline via PubMed and in Embase via Embase.com. Methods Articles published from 2011 to 2020 in a core set of eight journals (covering fields of interest for deprescribing, such as geriatrics, pharmacology and primary care) formed a reference set. Each article was screened independently in duplicate and classified as relevant or non-relevant to deprescribing. Relevant terms were identified by term frequency analysis in a 70% subset of the reference set. Selected title and abstract terms, MeSH terms and Emtree terms were combined to develop two highly sensitive filters for Medline via Pubmed and Embase via Embase.com. The filters were validated against the remaining 30% of the reference set. Sensitivity, specificity and precision were calculated with their 95% confidence intervals (95% CI). Results A total of 23,741 articles were aggregated in the reference set, and 224 were classified as relevant to deprescribing. A total of 34 terms and 4 MeSH terms were identified to develop the Medline search filter. A total of 27 terms and 6 Emtree terms were identified to develop the Embase search filter. The sensitivity was 92% (95% CI: 83–97%) in Medline via Pubmed and 91% (95% CI: 82–96%) in Embase via Embase.com. Conclusions These are the first deprescribing search filters that have been developed objectively and validated. These filters can be used in search strategies for future deprescribing reviews. Further prospective studies are needed to assess their effectiveness and efficiency when used in systematic reviews. Supplementary Information The online version contains supplementary material available at 10.1186/s12874-022-01515-x.
Collapse
Affiliation(s)
- Thomas Morel
- Département de Médecine Générale, Faculté de Médecine, Nantes Université, Nantes, France.,Inserm-University of Tours-University of Nantes, UMR U1246 Sphere "Methods in Patient-Centered Outcomes and Health Research", 37000, Tours, France
| | - Jérôme Nguyen-Soenen
- Département de Médecine Générale, Faculté de Médecine, Nantes Université, Nantes, France
| | - Wade Thompson
- Women's College Hospital Research Institute, Toronto, ON, Canada.,Research Unit of General Practice, University of Southern Denmark, Odense, Denmark
| | - Jean-Pascal Fournier
- Département de Médecine Générale, Faculté de Médecine, Nantes Université, Nantes, France. .,Inserm-University of Tours-University of Nantes, UMR U1246 Sphere "Methods in Patient-Centered Outcomes and Health Research", 37000, Tours, France.
| |
Collapse
|
6
|
Salvador-Oliván JA, Marco-Cuenca G, Arquero-Avilés R. Development of an efficient search filter to retrieve systematic reviews from PubMed. J Med Libr Assoc 2021; 109:561-574. [PMID: 34858085 PMCID: PMC8608217 DOI: 10.5195/jmla.2021.1223] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022] Open
Abstract
Objective: Locating systematic reviews is essential for clinicians and researchers when creating or updating reviews and for decision-making in health care. This study aimed to develop a search filter for retrieving systematic reviews that improves upon the performance of the PubMed systematic review search filter. Methods: Search terms were identified from abstracts of reviews published in Cochrane Database of Systematic Reviews and the titles of articles indexed as systematic reviews in PubMed. Both the precision of the candidate terms and the number of systematic reviews retrieved from PubMed were evaluated after excluding the subset of articles retrieved by the PubMed systematic review filter. Terms that achieved a precision greater than 70% and relevant publication types indexed with MeSH terms were included in the filter search strategy. Results: The search strategy used in our filter added specific terms not included in PubMed's systematic review filter and achieved a 61.3% increase in the number of retrieved articles that are potential systematic reviews. Moreover, it achieved an average precision that is likely greater than 80%. Conclusions: The developed search filter will enable users to identify more systematic reviews from PubMed than the PubMed systematic review filter with high precision.
Collapse
Affiliation(s)
| | - Gonzalo Marco-Cuenca
- , Professor, School of Medicine, Department of Library and Information Science, University of Zaragoza, Spain
| | - Rosario Arquero-Avilés
- , Professor, Department of Library and Information Science, Complutense University of Madrid, Madrid, Spain
| |
Collapse
|
7
|
Avau B, Van Remoortel H, De Buck E. Translation and validation of PubMed and Embase search filters for identification of systematic reviews, intervention studies, and observational studies in the field of first aid. J Med Libr Assoc 2021; 109:599-608. [PMID: 34858089 PMCID: PMC8608173 DOI: 10.5195/jmla.2021.1219] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
Objective The aim of this project was to validate search filters for systematic reviews, intervention studies, and observational studies translated from Ovid MEDLINE and Embase syntax and used for searches in PubMed and Embase.com during the development of evidence summaries supporting first aid guidelines. We aimed to achieve a balance among recall, specificity, precision, and number needed to read (NNR). Methods Reference gold standards were constructed per study type derived from existing evidence summaries. Search filter performance was assessed through retrospective searches and measurement of relative recall, specificity, precision, and NNR when using the translated search filters. Where necessary, search filters were optimized. Adapted filters were validated in separate validation gold standards. Results Search filters for systematic reviews and observational studies reached recall of ≥85% in both PubMed and Embase. Corresponding specificities for systematic review filters were ≥96% in both databases, with a precision of 9.7% (NNR 10) in PubMed and 5.4% (NNR 19) in Embase. For observational study filters, specificity, precision, and NNR were 68%, 2%, and 51 in PubMed and 47%, 0.8%, and 123 in Embase, respectively. These filters were considered sufficiently effective. Search filters for intervention studies reached a recall of 85% and 83% in PubMed and Embase, respectively. Optimization led to recall of ≥95% with specificity, precision, and NNR of 49%, 1.3%, and 79 in PubMed and 56%, 0.74%, and 136 in Embase, respectively. Conclusions We report validated filters to search for systematic reviews, observational studies, and intervention studies in guideline projects in PubMed and Embase.com.
Collapse
Affiliation(s)
- Bert Avau
- , Centre for Evidence-Based Practice, Belgian Red Cross, Mechelen, Belgium
| | - Hans Van Remoortel
- , Centre for Evidence-Based Practice, Belgian Red Cross, Mechelen, Belgium
| | - Emmy De Buck
- , Centre for Evidence-Based Practice, Belgian Red Cross, Mechelen, Belgium; Department of Public Health and Primary Care, Faculty of Medicine, KU Leuven, Leuven, Belgium; Cochrane First Aid, Mechelen, Belgium
| |
Collapse
|
8
|
Lyrio AO, da Cruz SS, Gomes-Filho IS, Silveira VSS, Souza ES, Batista JET, Figueiredo ACMG, Pereira MG. Validation of a search strategy for randomized clinical trials related to periodontitis. ACTA ACUST UNITED AC 2021; 79:43. [PMID: 33812387 PMCID: PMC8019511 DOI: 10.1186/s13690-021-00560-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2020] [Accepted: 03/09/2021] [Indexed: 11/13/2022]
Abstract
Background Systematic reviews, considered the gold standard for the assessment of scientific evidence, may present conflicting findings for the same clinical issue, and such dissent may be justified by the forms of elaboration of the electronic search strategy. This paper aims to validate a search strategy to identify randomized clinical trials related to periodontitis. A gold standard reference set was developed to validate the identified clinical trials using the relative recall method. The choice of periodontitis is due to the fact that this disease has a high prevalence among chronic non-communicable diseases, is considered the second most common oral disease in the world, is associated with several health problems, such as cardiovascular diseases and diabetes, and, principally, has not been investigated sufficiently to prevent possible damages resulting from it. Methods A validation study was developed in MEDLINE/PubMed. In Stage 1, a methodological filter recommended by the Cochrane Collaboration to identify randomized clinical trials was applied. Stage 2 identified articles related only to periodontitis (gold standard reference set) from among the articles retrieved using the eligibility criteria. In Stage 3, a search statement for the retrieval of periodontitis-related articles was elaborated by experts. Stage 4 defined the proposed search strategy comprising of the combination of the search statement developed with the aforementioned methodological filter and subsequent application in MEDLINE/PubMed. The obtained data were analyzed using the set of articles identified in Stage 2, as the gold standard reference set. The following performance values were calculated - sensitivity, specificity, accuracy, and number needed to read - with their respective 95% confidence interval (95%CI). Results The search strategy under evaluation compared to the gold-standard showed a sensitivity of 93.2% (95%CI, 83.8–97.3), specificity of 99.9% (95%CI 99.8–99.9), and a precision of 77.5% (95%CI, 66.48–85.63). In addition, the number needed to read was 1.3. Conclusion According to the proposed methodological approach, the search strategy under evaluation performed well in the identification of randomized clinical trials related to periodontitis. Supplementary Information The online version contains supplementary material available at 10.1186/s13690-021-00560-0.
Collapse
Affiliation(s)
- Amanda Oliveira Lyrio
- Faculty of Health Sciences, University of Brasilia, Brasília, Distrito Federal, Brazil
| | - Simone Seixas da Cruz
- Health Sciences Center, Federal University of Recôncavo da Bahia, Rua Fonte do Céu, 1521, Andaiá, Santo Antônio de Jesus, Bahia, 44572-560, Brazil.
| | | | | | - Elivan Silva Souza
- Faculty of Collective Health, University of Brasilia, Brasília, Distrito Federal, Brazil
| | | | | | | |
Collapse
|
9
|
Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, Koffel JB. PRISMA-S: an extension to the PRISMA statement for reporting literature searches in systematic reviews . J Med Libr Assoc 2021; 109:174-200. [PMID: 34285662 PMCID: PMC8270366 DOI: 10.5195/jmla.2021.962] [Citation(s) in RCA: 61] [Impact Index Per Article: 20.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022] Open
Abstract
BACKGROUND Literature searches underlie the foundations of systematic reviews and related review types. Yet, the literature searching component of systematic reviews and related review types is often poorly reported. Guidance for literature search reporting has been diverse and, in many cases, does not offer enough detail to authors who need more specific information about reporting search methods and information sources in a clear, reproducible way. This document presents the PRISMA-S (Preferred Reporting Items for Systematic reviews and Meta-Analyses literature search extension) checklist, and explanation and elaboration. METHODS The checklist was developed using a three-stage Delphi survey process, followed by a consensus conference and public review process. RESULTS The final checklist includes sixteen reporting items, each of which is detailed with exemplar reporting and rationale. CONCLUSIONS The intent of PRISMA-S is to complement the PRISMA Statement and its extensions by providing a checklist that could be used by interdisciplinary authors, editors, and peer reviewers to verify that each component of a search is completely reported and, therefore, reproducible.
Collapse
Affiliation(s)
- Melissa L. Rethlefsen
- , Executive Director and Professor, Health Sciences Library & Informatics Center, University of New Mexico
| | - Shona Kirtley
- , Senior Research Information Specialist, UK EQUATOR Centre, Centre for Statistics in Medicine (CSM), Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences (NDORMS), University of Oxford, Botnar Research Centre, Oxford, United Kingdom
| | - Siw Waffenschmidt
- , Head of the Information Management Unit, Institute for Quality and Efficiency in Health Care, Cologne, Germany
| | - Ana Patricia Ayala
- , Research Services Librarian, Gerstein Science Information Centre, University of Toronto, Toronto, ON, Canada
| | - David Moher
- , Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, Centre for Practice Changing Research Building, Ottawa, ON, Canada
| | - Matthew J. Page
- , Research Fellow, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
| | - Jonathan B. Koffel
- , Emerging Technology and Innovation Strategist, University of Minnesota, Minneapolis, MN
| | - PRISMA-S Group
- , Executive Director and Professor, Health Sciences Library & Informatics Center, University of New Mexico
- , Senior Research Information Specialist, UK EQUATOR Centre, Centre for Statistics in Medicine (CSM), Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences (NDORMS), University of Oxford, Botnar Research Centre, Oxford, United Kingdom
- , Head of the Information Management Unit, Institute for Quality and Efficiency in Health Care, Cologne, Germany
- , Research Services Librarian, Gerstein Science Information Centre, University of Toronto, Toronto, ON, Canada
- , Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, Centre for Practice Changing Research Building, Ottawa, ON, Canada
- , Research Fellow, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
- , Emerging Technology and Innovation Strategist, University of Minnesota, Minneapolis, MN
| |
Collapse
|
10
|
Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, Koffel JB. PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Syst Rev 2021; 10:39. [PMID: 33499930 PMCID: PMC7839230 DOI: 10.1186/s13643-020-01542-z] [Citation(s) in RCA: 828] [Impact Index Per Article: 276.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/28/2020] [Accepted: 11/23/2020] [Indexed: 12/13/2022] Open
Abstract
BACKGROUND Literature searches underlie the foundations of systematic reviews and related review types. Yet, the literature searching component of systematic reviews and related review types is often poorly reported. Guidance for literature search reporting has been diverse, and, in many cases, does not offer enough detail to authors who need more specific information about reporting search methods and information sources in a clear, reproducible way. This document presents the PRISMA-S (Preferred Reporting Items for Systematic reviews and Meta-Analyses literature search extension) checklist, and explanation and elaboration. METHODS The checklist was developed using a 3-stage Delphi survey process, followed by a consensus conference and public review process. RESULTS The final checklist includes 16 reporting items, each of which is detailed with exemplar reporting and rationale. CONCLUSIONS The intent of PRISMA-S is to complement the PRISMA Statement and its extensions by providing a checklist that could be used by interdisciplinary authors, editors, and peer reviewers to verify that each component of a search is completely reported and therefore reproducible.
Collapse
Affiliation(s)
- Melissa L. Rethlefsen
- Health Science Center Libraries, George A. Smathers Libraries, University of Florida, Gainesville, USA
| | - Shona Kirtley
- UK EQUATOR Centre, Centre for Statistics in Medicine (CSM), Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences (NDORMS), Botnar Research Centre, University of Oxford, Windmill Road, Oxford, OX3 7LD UK
| | - Siw Waffenschmidt
- Institute for Quality and Efficiency in Health Care, Cologne, Germany
| | - Ana Patricia Ayala
- Gerstein Science Information Centre, University of Toronto, Toronto, Canada
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, Centre for Practice Changing Research Building, 501 Smyth Road, PO BOX 201B, Ottawa, Ontario K1H 8L6 Canada
| | - Matthew J. Page
- School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
| | | |
Collapse
|
11
|
The NICE MEDLINE and Embase (Ovid) health apps search filters: development of validated filters to retrieve evidence about health apps. Int J Technol Assess Health Care 2020; 37:e16. [PMID: 33107420 DOI: 10.1017/s026646232000080x] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
OBJECTIVES Health apps are software programs that are designed to prevent, diagnose, monitor, or manage conditions. Inconsistent terminology for apps is used in research literature and bibliographic database subject headings. It can therefore be challenging to retrieve evidence about them in literature searches. Information specialists at the United Kingdom's National Institute for Health and Care Excellence (NICE) have developed novel validated search filters to retrieve evidence about apps from MEDLINE and Embase (Ovid). METHODS A selection of medical informatics journals was hand searched to identify a "gold standard" (GS) set of references about apps. The GS set was divided into a development and validation set. The filters' search terms were derived from and tested against the development set. An external development set containing app references from published NICE products was also used to inform the development of the filters. The filters were then validated using the validation set. Target recall was >90 percent. The filters' overall recall, specificity, and precision were calculated using all the references identified from the hand search. RESULTS Both filters achieved 98.6 percent recall against their validation sets. Overall, the MEDLINE filter had 98.8 percent recall, 71.3 percent specificity, and 22.6 percent precision. The Embase filter had 98.6 percent recall, 74.9 percent specificity, and 24.5 percent precision. CONCLUSIONS The NICE health apps search filters retrieve evidence about apps from MEDLINE and Embase with high recall. They can be applied to literature searches to retrieve evidence about the interventions by information professionals, researchers, and clinicians.
Collapse
|
12
|
Glanville J, Kotas E, Featherstone R, Dooley G. Which are the most sensitive search filters to identify randomized controlled trials in MEDLINE? J Med Libr Assoc 2020; 108:556-563. [PMID: 33013212 PMCID: PMC7524635 DOI: 10.5195/jmla.2020.912] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022] Open
Abstract
OBJECTIVE The Cochrane Handbook of Systematic Reviews contains search filters to find randomized controlled trials (RCTs) in Ovid MEDLINE: one maximizing sensitivity and another balancing sensitivity and precision. These filters were originally published in 1994 and were adapted and updated in 2008. To determine the performance of these filters, the authors tested them and thirty-six other MEDLINE filters against a large new gold standard set of relevant records. METHODS We identified a gold standard set of RCT reports published in 2016 from the Cochrane CENTRAL database of controlled clinical trials. We retrieved the records in Ovid MEDLINE and combined these with each RCT filter. We calculated their sensitivity, relative precision, and f-scores. RESULTS The gold standard comprised 27,617 records. MEDLINE searches were run on July 16, 2019. The most sensitive RCT filter was Duggan et al. (sensitivity=0.99). The Cochrane sensitivity-maximizing RCT filter had a sensitivity of 0.96 but was more precise than Duggan et al. (0.14 compared to 0.04 for Duggan). The most precise RCT filters had 0.97 relative precision and 0.83 sensitivity. CONCLUSIONS The Cochrane Ovid MEDLINE sensitivity-maximizing RCT filter can continue to be used by Cochrane reviewers and to populate CENTRAL, as it has very high sensitivity and a slightly better precision relative to more sensitive filters. The results of this study, which used a very large gold standard to compare the performance of all known RCT filters, allows searchers to make better informed decisions about which filters to use for their work.
Collapse
Affiliation(s)
- Julie Glanville
- , York Health Economics Consortium, University of York, York, United Kingdom
| | - Eleanor Kotas
- , York Health Economics Consortium, University of York, York, United Kingdom
| | | | | |
Collapse
|
13
|
Rosumeck S, Wagner M, Wallraf S, Euler U. A validation study revealed differences in design and performance of search filters for qualitative research in PsycINFO and CINAHL. J Clin Epidemiol 2020; 128:101-108. [PMID: 32987157 DOI: 10.1016/j.jclinepi.2020.09.031] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2020] [Revised: 08/21/2020] [Accepted: 09/03/2020] [Indexed: 10/23/2022]
Abstract
OBJECTIVES Search filters can support qualitative evidence of information retrieval. Various search filters are available for the bibliographic databases PsycINFO and CINAHL. To date, no comparative overview of validation results of search filters verified with an independent gold standard exists. STUDY DESIGN AND SETTING Identified search filters for PsycINFO and CINAHL were tested for plausibility. Gold standards were generated according to the relative recall approach using references included in an overview of systematic reviews of qualitative studies. All included references were collected and checked for indexing in PsycINFO and CINAHL. Validation tests for each search filter were conducted in both databases to determine whether the references of the gold standards could be retrieved or not. RESULTS Twelve search filters for PsycINFO and fifteen for CINAHL were validated. The complexity and design of these search filters vary, as well as the validation results for the databases. When locating primary studies of qualitative research, the best sensitivity and precision ratio (among filters with a sensitivity of >80%) was achieved with a filter by McKibbon et al. for PsycINFO and a filter by Wilczynski et al. for CINAHL. CONCLUSION Project-specific requirements and resources influence the choice of a specific search filter for PsycINFO and CINAHL.
Collapse
Affiliation(s)
- Stefanie Rosumeck
- Institute for Quality Assurance and Transparency in Healthcare (IQTIG), Katharina-Heinroth-Ufer 1, 10787 Berlin, Germany.
| | - Mandy Wagner
- Institute for Quality Assurance and Transparency in Healthcare (IQTIG), Katharina-Heinroth-Ufer 1, 10787 Berlin, Germany
| | - Simon Wallraf
- Institute for Quality Assurance and Transparency in Healthcare (IQTIG), Katharina-Heinroth-Ufer 1, 10787 Berlin, Germany
| | - Ulrike Euler
- Institute for Quality Assurance and Transparency in Healthcare (IQTIG), Katharina-Heinroth-Ufer 1, 10787 Berlin, Germany
| |
Collapse
|
14
|
Escrivá L, Hessel E, Gustafsson S, van Spronsen R, Svanberg M, Beronius A. A validated search filter for the identification of endocrine disruptors based on the ECHA/EFSA guidance recommendations. ENVIRONMENT INTERNATIONAL 2020; 142:105828. [PMID: 32502797 DOI: 10.1016/j.envint.2020.105828] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/14/2020] [Revised: 05/08/2020] [Accepted: 05/19/2020] [Indexed: 06/11/2023]
Abstract
A guidance document for the identification of endocrine disruptors (EDs) in the regulatory assessment of plant protection products (PPP) and biocidal products (BP) has been published by the European Chemical Agency (ECHA) and the European Food Safety Authority (EFSA). The ECHA/EFSA guidance, mainly addressing EATS (estrogen, androgen, thyroid, steroidogenesis) modalities, is intended to guide applicants and assessors of the competent regulatory authorities on the implementation of the scientific criteria for the determination of ED properties pursuant to the recently implemented PPP (EU 2018/605) and BP (EU 2017/2100) EU Regulations. In this study, a search filter for targeted literature search in context of assessing if a substance can be identified as an ED relevant for human health was developed and validated. Development of the search filter was based on the search strategy presented in the ECHA/EFSA guidance and using the estrogenic chemical Bisphenol AF (BPAF) as a model substance. Information specialists from two independent institutions developed refined search filters based on the suggested original search strategy published (ECHA/EFSA guidance - Appendix F). Articles identified by a systematic literature search for BPAF were screened for relevance with inclusion and exclusion criteria by two independent reviewers obtaining positive (relevant) and negative (irrelevant) controls. The developed search filter was quantitatively evaluated in terms of sensitivity, specificity and precision based on the positive and negative controls. The developed filter was then validated for T modality by its application to the known thyroid-disruptor perchlorate. The result is a sensitive search filter with sufficient specificity, which can be applied for all chemicals where a targeted literature search is needed to assess and identify ED properties of chemicals with relevance for humans. Future application of the filter to a broader range of chemicals may identify further points of improvement.
Collapse
Affiliation(s)
- Laura Escrivá
- Karolinska Institutet, Institute of Environmental Medicine, Stockholm, Sweden; Laboratory of Food Chemistry and Toxicology, Faculty of Pharmacy, University of Valencia, Burjassot, Spain.
| | - Ellen Hessel
- National Institute for Public Health and the Environment (RIVM), Utrecht, the Netherlands.
| | | | - Rob van Spronsen
- National Institute for Public Health and the Environment (RIVM), Utrecht, the Netherlands.
| | | | - Anna Beronius
- Karolinska Institutet, Institute of Environmental Medicine, Stockholm, Sweden.
| |
Collapse
|
15
|
Cooper C, Garside R, Varley-Campbell J, Talens-Bou J, Booth A, Britten N. "It has no meaning to me." How do researchers understand the effectiveness of literature searches? A qualitative analysis and preliminary typology of understandings. Res Synth Methods 2020; 11:627-640. [PMID: 32495989 DOI: 10.1002/jrsm.1426] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2019] [Revised: 05/27/2020] [Accepted: 05/28/2020] [Indexed: 12/28/2022]
Abstract
This study aimed to address the question: what does "effectiveness" mean to researchers in the context of literature searching for systematic reviews? We conducted a thematic analysis of responses to an e-mail survey. Eighty-nine study authors, whose studies met inclusion in a recent review (2018), were contacted via e-mail and asked three questions; one directly asking the question: in literature searching, what does effective (or effectiveness in) literature searching mean to you? Thirty-eight (46%) responses were received from diverse professional groups, including: literature searchers, systematic reviewers, clinicians and researchers. A shared understanding of what effectiveness means was not identified. Instead, five themes were developed from data: (a) effectiveness is described as a metric; (b) effectiveness is a balance between metrics; (c) effectiveness can be categorized by search purpose; (d) effectiveness is an outcome; and, (e) effectiveness is an experimental concept. We propose that these themes constitute a preliminary typology of understandings. No single definition of effectiveness was identified. The proposed typology suggests that different researchers have differing understandings of effectiveness. This could lead to uncertainty as to the aim and the purpose of literature searches and confusion about the outcomes. The typology offers a potential route for further exploration.
Collapse
Affiliation(s)
- Chris Cooper
- Department of Clinical, Educational and Health Psychology, University College London, London, UK
| | - Ruth Garside
- European Centre for Environment and Human Health, University of Exeter Medical School, Truro, UK
| | - Joanna Varley-Campbell
- Department of Clinical, Educational and Health Psychology, University College London, London, UK
| | | | - Andrew Booth
- School of Health & Related Research, University of Sheffield, Sheffield, UK
| | - Nicky Britten
- Institute of Health Research, University of Exeter Medical School, Truro, UK
| |
Collapse
|
16
|
Waffenschmidt S, Navarro-Ruan T, Hobson N, Hausner E, Sauerland S, Haynes RB. Development and validation of study filters for identifying controlled non-randomized studies in PubMed and Ovid MEDLINE. Res Synth Methods 2020; 11:617-626. [PMID: 32472632 DOI: 10.1002/jrsm.1425] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2019] [Revised: 03/19/2020] [Accepted: 05/21/2020] [Indexed: 01/08/2023]
Abstract
A retrospective analysis published by the German Institute for Quality and Efficiency in Health Care (IQWiG) in 2018 concluded that no filter for non-randomized studies (NRS) achieved sufficient sensitivity (≥92%), a precondition for comprehensive information retrieval. New NRS filters are therefore required, taking into account the challenges related to this study type. Our evaluation focused on the development of study filters for NRS with a control group ("controlled NRS"), as this study type allows the calculation of an effect size. In addition, we assumed that due to the more explicit search syntax, controlled NRS are easier to identify than non-controlled ones, potentially resulting in better performance measures of study filters for controlled NRS. Our aim was to develop study filters for identifying controlled NRS in PubMed and Ovid MEDLINE. We developed two new search filters that can assist clinicians and researchers in identifying controlled NRS in PubMed and Ovid MEDLINE. The reference set was based on 2110 publications in Medline extracted from 271 Cochrane reviews and on 4333 irrelevant references. The first filter maximizes sensitivity (92.42%; specificity 79.67%, precision 68.49%) and should be used when a comprehensive search is needed. The second filter maximizes specificity (92.06%; precision 82.98%, sensitivity 80.94%) and should be used when a more focused search is sufficient.
Collapse
Affiliation(s)
- Siw Waffenschmidt
- Institute for Quality and Efficiency in Health Care, Cologne, Germany.,Institute for Health Economics and Clinical Epidemiology, The University Hospital of Cologne, Cologne, Germany
| | - Tamara Navarro-Ruan
- Health Information Research Unit, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Canada
| | - Nick Hobson
- Health Information Research Unit, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Canada
| | - Elke Hausner
- Institute for Quality and Efficiency in Health Care, Cologne, Germany
| | - Stefan Sauerland
- Institute for Quality and Efficiency in Health Care, Cologne, Germany
| | - R Brian Haynes
- Health Information Research Unit, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Canada
| |
Collapse
|
17
|
Cooper C, Kaunelis D, Varley Campbell J, Carter P. Letter in response to Thompson and Scott Authors’ letter reestablished search filters may miss studies when identifying randomized controlled trials. Language for trial phase necessary when searching for RCT. J Clin Epidemiol 2020; 117:154-156. [DOI: 10.1016/j.jclinepi.2019.09.008] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2019] [Accepted: 09/10/2019] [Indexed: 02/06/2023]
|
18
|
Zhang J, Han L, Shields L, Tian J, Wang J. A PRISMA assessment of the reporting quality of systematic reviews of nursing published in the Cochrane Library and paper-based journals. Medicine (Baltimore) 2019; 98:e18099. [PMID: 31804319 PMCID: PMC6919387 DOI: 10.1097/md.0000000000018099] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/10/2023] Open
Abstract
OBJECTIVE The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) was released as a standard of reporting systematic reviewers (SRs). However, not all SRs adhere completely to this standard. This study aimed to evaluate the reporting quality of SRs published in the Cochrane Library and paper-based journals. METHODS The SRs which evaluate the effectiveness of nursing interventions in 2016 were identified via PubMed. The reporting quality of selected articles was evaluated using the PRISMA checklist. For comparison, we divided these articles into Cochrane review (CR) and non-Cochrane review (NCR). Based on the satisfaction of the applicable criteria, each article is assigned an accumulated score and a total percentage score. RESULTS Overall, 41.7% articles were concentrated in 19.0 to 22.5 points which represent the moderate quality, 22% articles were high quality. There were still 36.5% articles with low quality. The mean PRISMA score was 20.54 ± 2.367 for CRs, and 18.81 ± 2.536 for NCRs. Although no significant difference was exit between overall CR and NCR scores, there were differences between items 1, 5, 8, 16, 23. Analysis indicated that CR was significantly associated with the overall PRISMA score. CONCLUSION Compliance of CR and NCR with PRISMA checklist exhibited different strengths and weaknesses. Our study underscores that nursing researchers should pay more attention to comprehensive reporting of SRs in nursing to follow the PRISMA statement. IMPLICATIONS FOR NURSING AND/OR HEALTH POLICY Nursing researchers who participate in SRs should follow the latest Cochrane Handbook to prepare such study. Meanwhile, the PRISMA statement should be followed strictly to report SRs, so as to improve the quality of SRs.
Collapse
Affiliation(s)
- Juxia Zhang
- Nursing Department, Gansu Provincial Hospital
| | - Lin Han
- Nursing Department, Gansu Provincial Hospital
- School of Nursing, Lanzhou University, Lanzhou, China
| | - Linda Shields
- School of Nursing, Midwifery and Indigenous Health, Charles Sturt University, Bathurst, NSW
- School of Medicine, The University of Queensland, Brisbane, QLD, Australia
| | - Jinhui Tian
- Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University
| | - Jiancheng Wang
- Elder Department, Gansu Provincial Hospital, Lanzhou, China
| |
Collapse
|
19
|
Cooper C, Varley-Campbell J, Carter P. Established search filters may miss studies when identifying randomized controlled trials. J Clin Epidemiol 2019; 112:12-19. [DOI: 10.1016/j.jclinepi.2019.04.002] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2019] [Revised: 03/20/2019] [Accepted: 04/08/2019] [Indexed: 10/27/2022]
|
20
|
Rastegarfar B, Ardalan A, Nejat S, Keshtkar A, Moradian MJ. A Productive Proposed Search Syntax for Health Disaster Preparedness Research. Bull Emerg Trauma 2019; 7:93-98. [PMID: 31198795 PMCID: PMC6555207 DOI: 10.29252/beat-070201] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022] Open
Abstract
Objective: To find a proper search strategy to do a systematic review related to preparedness for disasters. Methods: MeSH and Emtree terms were searched to detect synonyms for two main search terms “disaster” and “preparedness”. Expert opinion on the synonyms was examined applying a Google form. The adopted syntax was searched in PubMed and results were sifted. Hand searching in two top key journals was done and sensitivity was calculated. Results: Out of 1120 articles, 122 were included. In PDM journal, 10 articles were included by hand searching, out of which 5 were not spotted in PubMed search with the proposed syntax. In DMPHP journal, 13 publications were included, with 5 not found in PubMed search. Because of human error in hand searching 2 articles were added. Conclusion: The proposed syntax in this study achieves a sensitivity of search of 0.6 in PubMed which could be quite applicable for researchers. Moreover, in case only MeSH or Emtree terms were applied in search strategy or where hand searching was not performed, there were a number of articles missed.
Collapse
Affiliation(s)
- Behnaz Rastegarfar
- Department of Disaster Public Health, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
| | - Ali Ardalan
- Department of Disaster Public Health, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
| | - Saharnaz Nejat
- Department of Epidemiology and Biostatistics, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
| | - Abbasali Keshtkar
- Department of Health Sciences Education Development, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
| | - Mohammad Javad Moradian
- Department of Disaster Public Health, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran.,Trauma Research Center, Shahid Rajaee (Emtiaz) Trauma Hospital, Shiraz University of Medical Sciences, Shiraz, Iran
| |
Collapse
|
21
|
Rastegarfar B, Ardalan A, Nejat S, Keshtkar A, Moradian MJ. A Productive Proposed Search Syntax for Health Disaster Preparedness Research. Bull Emerg Trauma 2019. [PMID: 31198795 DOI: 10.29252/beat-070201.] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/26/2022] Open
Abstract
Objective To find a proper search strategy to do a systematic review related to preparedness for disasters. Methods MeSH and Emtree terms were searched to detect synonyms for two main search terms "disaster" and "preparedness". Expert opinion on the synonyms was examined applying a Google form. The adopted syntax was searched in PubMed and results were sifted. Hand searching in two top key journals was done and sensitivity was calculated. Results Out of 1120 articles, 122 were included. In PDM journal, 10 articles were included by hand searching, out of which 5 were not spotted in PubMed search with the proposed syntax. In DMPHP journal, 13 publications were included, with 5 not found in PubMed search. Because of human error in hand searching 2 articles were added. Conclusion The proposed syntax in this study achieves a sensitivity of search of 0.6 in PubMed which could be quite applicable for researchers. Moreover, in case only MeSH or Emtree terms were applied in search strategy or where hand searching was not performed, there were a number of articles missed.
Collapse
Affiliation(s)
- Behnaz Rastegarfar
- Department of Disaster Public Health, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
| | - Ali Ardalan
- Department of Disaster Public Health, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
| | - Saharnaz Nejat
- Department of Epidemiology and Biostatistics, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
| | - Abbasali Keshtkar
- Department of Health Sciences Education Development, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran
| | - Mohammad Javad Moradian
- Department of Disaster Public Health, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran.,Trauma Research Center, Shahid Rajaee (Emtiaz) Trauma Hospital, Shiraz University of Medical Sciences, Shiraz, Iran
| |
Collapse
|
22
|
Li L, Smith HE, Atun R, Tudor Car L. Search strategies to identify observational studies in MEDLINE and Embase. Cochrane Database Syst Rev 2019; 3:MR000041. [PMID: 30860595 PMCID: PMC8103566 DOI: 10.1002/14651858.mr000041.pub2] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
BACKGROUND Systematic reviews are essential for decision-making. Systematic reviews on observational studies help answer research questions on aetiology, risk, prognosis, and frequency of rare outcomes or complications. However, identifying observational studies as part of systematic reviews efficiently is challenging due to poor and inconsistent indexing in literature databases. Search strategies that include a methodological filter focusing on study design of observational studies might be useful for improving the precision of the search performance. OBJECTIVES To assess the sensitivity and precision of a search strategy with a methodological filter to identify observational studies in MEDLINE and Embase. SEARCH METHODS We searched MEDLINE (1946 to April 2018), Embase (1974 to April 2018), CINAHL (1937 to April 2018), the Cochrane Library (1992 to April 2018), Google Scholar and Open Grey in April 2018, and scanned reference lists of articles. SELECTION CRITERIA Studies using a relative recall approach, i.e. comparing sensitivity or precision of a search strategy containing a methodological filter to identify observational studies in MEDLINE and Embase against a reference standard, or studies that compared two or more methodological filters. DATA COLLECTION AND ANALYSIS Two review authors independently screened articles, extracted relevant information and assessed the quality of the search strategies using the InterTASC Information Specialists' Sub-Group (ISSG) Search Filter Appraisal Checklist. MAIN RESULTS We identified two eligible studies reporting 18 methodological filters. All methodological filters in these two studies were developed using terms from the reference standard records.The first study evaluated six filters for retrieving observational studies of surgical interventions. The study reported on six filters: one Precision Terms Filter (comprising terms with higher precision while maximum sensitivity was maintained) and one Specificity Terms Filter (comprising terms with higher specificity while maximum sensitivity was maintained), both of which were adapted for MEDLINE, for Embase, and for combined MEDLINE/Embase searches. The study reported one reference standard consisting of 217 articles from one systematic review of which 83.9% of the included studies were case seriesThe second study reported on 12 filters for retrieving comparative non-randomised studies (cNRSs) including cohort, case-control, and cross-sectional studies. This study reported on 12 filters using four different approaches: Fixed method A (comprising of a fixed set of controlled vocabulary (CV) words), Fixed method B (comprising a fixed set of CV words and text words (TW)), Progressive method (CV) (a random choice of study design-related CV terms), and Progressive method (CV or TW) (a random choice of study design-related CV terms, and title and abstracts-based TWs). The study reported four reference standards consisting of 89 cNRSs from four systematic reviews.The six methodological filters developed from the first study reported sensitivity of 99.5% to 100% and precision of 16.7% to 21.1%. The Specificity Terms Filter for combined MEDLINE/Embase was preferred because it had higher precision and equal sensitivity to the Precision Terms Filter. The 12 filters from the second study reported lower sensitivity (48% to 100%) and much lower precision (0.09% to 4.47%). The Progressive method (CV or TW) had the highest sensitivity.There were methodological limitations in both included studies. The first study used one surgical intervention-focused systematic review thus limiting the generalizability of findings. The second study used four systematic reviews but with less than 100 studies. The external validation was performed only on Specificity Terms Filter from the first study Both studies were published 10 years ago and labelling and indexing of observational studies has changed since then. AUTHORS' CONCLUSIONS We found 18 methodological filters across two eligible studies. Search strategies from the first study had higher sensitivity and precision, underwent external validation and targeted observational studies. Search strategies from the second study had lower sensitivity and precision, focused on cNRSs, and were not validated externally. Given this limited and heterogeneous evidence, and its methodological limitations, further research and better indexation are needed.
Collapse
Affiliation(s)
- Li Li
- Nanyang Technological UniversityLee Kong Chian School of MedicineSingaporeSingapore
| | - Helen E Smith
- Nanyang Technological UniversityFamily Medicine and Primary Care, Lee Kong Chian School of Medicine11 Mandalay RoadLevel 18‐08 Clinical Sciences BuildingSingaporeSingapore
| | - Rifat Atun
- Harvard School of Public HealthDepartment of Global Health and Population665 Huntington AvenueBostonMAUSA02115
| | - Lorainne Tudor Car
- Nanyang Technological UniversityFamily Medicine and Primary Care, Lee Kong Chian School of Medicine11 Mandalay RoadLevel 18‐08 Clinical Sciences BuildingSingaporeSingapore
| | | |
Collapse
|
23
|
Damarell RA, May N, Hammond S, Sladek RM, Tieman JJ. Topic search filters: a systematic scoping review. Health Info Libr J 2018; 36:4-40. [PMID: 30578606 DOI: 10.1111/hir.12244] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2018] [Accepted: 11/21/2018] [Indexed: 12/21/2022]
Abstract
BACKGROUND Searching for topics within large biomedical databases can be challenging, especially when topics are complex, diffuse, emerging or lack definitional clarity. Experimentally derived topic search filters offer a reliable solution to effective retrieval; however, their number and range of subject foci remain unknown. OBJECTIVES This systematic scoping review aims to identify and describe available experimentally developed topic search filters. METHODS Reports on topic search filter development (1990-) were sought using grey literature sources and 15 databases. Reports describing the conception and prospective development of a database-specific topic search and including an objectively measured estimate of its performance ('sensitivity') were included. RESULTS Fifty-four reports met inclusion criteria. Data were extracted and thematically synthesised to describe the characteristics of 58 topic search filters. DISCUSSION Topic search filters are proliferating and cover a wide range of subjects. Filter reports, however, often lack clear definitions of concepts and topic scope to guide users. Without standardised terminology, filters are challenging to find. Information specialists may benefit from a centralised topic filter repository and appraisal checklists to facilitate quality assessment. CONCLUSION Findings will help information specialists identify existing topic search filters and assist filter developers to build on current knowledge in the field.
Collapse
Affiliation(s)
- Raechel A Damarell
- College of Nursing and Health Sciences, Flinders University, Bedford Park, SA, Australia
| | - Nikki May
- South Australian Health Library Service, Flinders Medical Centre, Bedford Park, SA, Australia
| | - Sue Hammond
- Flinders University Library, Flinders University, Bedford Park, SA, Australia
| | - Ruth M Sladek
- College of Medicine and Public Health, Flinders University, Bedford Park, SA, Australia
| | - Jennifer J Tieman
- College of Nursing and Health Sciences, Flinders University, Bedford Park, SA, Australia
| |
Collapse
|
24
|
Stengel D, Leisterer J, Ferrada P, Ekkernkamp A, Mutze S, Hoenning A. Point-of-care ultrasonography for diagnosing thoracoabdominal injuries in patients with blunt trauma. Cochrane Database Syst Rev 2018; 12:CD012669. [PMID: 30548249 PMCID: PMC6517180 DOI: 10.1002/14651858.cd012669.pub2] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
BACKGROUND Point-of-care sonography (POCS) has emerged as the screening modality of choice for suspected body trauma in many emergency departments worldwide. Its best known application is FAST (focused abdominal sonography for trauma). The technology is almost ubiquitously available, can be performed during resuscitation, and does not expose patients or staff to radiation. While many authors have stressed the high specificity of POCS, its sensitivity varied markedly across studies. This review aimed to compile the current best evidence about the diagnostic accuracy of POCS imaging protocols in the setting of blunt thoracoabdominal trauma. OBJECTIVES To determine the diagnostic accuracy of POCS for detecting and excluding free fluid, organ injuries, vascular lesions, and other injuries (e.g. pneumothorax) compared to a diagnostic reference standard (i.e. computed tomography (CT), magnetic resonance imaging (MRI), thoracoscopy or thoracotomy, laparoscopy or laparotomy, autopsy, or any combination of these) in patients with blunt trauma. SEARCH METHODS We searched Ovid MEDLINE (1946 to July 2017) and Ovid Embase (1974 to July 2017), as well as PubMed (1947 to July 2017), employing a prospectively defined literature and data retrieval strategy. We also screened the Cochrane Library, Google Scholar, and BIOSIS for potentially relevant citations, and scanned the reference lists of full-text papers for articles missed by the electronic search. We performed a top-up search on 6 December 2018, and identified eight new studies which may be incorporated into the first update of this review. SELECTION CRITERIA We assessed studies for eligibility using predefined inclusion and exclusion criteria. We included either prospective or retrospective diagnostic cohort studies that enrolled patients of any age and gender who sustained any type of blunt injury in a civilian scenario. Eligible studies had to provide sufficient information to construct a 2 x 2 table of diagnostic accuracy to allow for calculating sensitivity, specificity, and other indices of diagnostic test accuracy. DATA COLLECTION AND ANALYSIS Two review authors independently screened titles, abstracts, and full texts of reports using a prespecified data extraction form. Methodological quality of individual studies was rated by the QUADAS-2 instrument (the revised and updated version of the original Quality Assessment of Diagnostic Accuracy Studies list of items). We calculated sensitivity and specificity with 95% confidence intervals (CI), tabulated the pairs of sensitivity and specificity with CI, and depicted these estimates by coupled forest plots using Review Manager 5 (RevMan 5). For pooling summary estimates of sensitivity and specificity, and investigating heterogeneity across studies, we fitted a bivariate model using Stata 14.0. MAIN RESULTS We included 34 studies with 8635 participants in this review. Summary estimates of sensitivity and specificity were 0.74 (95% CI 0.65 to 0.81) and 0.96 (95% CI 0.94 to 0.98). Pooled positive and negative likelihood ratios were estimated at 18.5 (95% CI 10.8 to 40.5) and 0.27 (95% CI 0.19 to 0.37), respectively. There was substantial heterogeneity across studies, and the reported accuracy of POCS strongly depended on the population and affected body area. In children, pooled sensitivity of POCS was 0.63 (95% CI 0.46 to 0.77), as compared to 0.78 (95% CI 0.69 to 0.84) in an adult or mixed population. Associated specificity in children was 0.91 (95% CI 0.81 to 0.96) and in an adult or mixed population 0.97 (95% CI 0.96 to 0.99). For abdominal trauma, POCS had a sensitivity of 0.68 (95% CI 0.59 to 0.75) and a specificity of 0.95 (95% CI 0.92 to 0.97). For chest injuries, sensitivity and specificity were calculated at 0.96 (95% CI 0.88 to 0.99) and 0.99 (95% CI 0.97 to 1.00). If we consider the results of all 34 included studies in a virtual population of 1000 patients, based on the observed median prevalence (pretest probability) of thoracoabdominal trauma of 28%, POCS would miss 73 patients with injuries and falsely suggest the presence of injuries in another 29 patients. Furthermore, in a virtual population of 1000 children, based on the observed median prevalence (pretest probability) of thoracoabdominal trauma of 31%, POCS would miss 118 children with injuries and falsely suggest the presence of injuries in another 62 children. AUTHORS' CONCLUSIONS In patients with suspected blunt thoracoabdominal trauma, positive POCS findings are helpful for guiding treatment decisions. However, with regard to abdominal trauma, a negative POCS exam does not rule out injuries and must be verified by a reference test such as CT. This is of particular importance in paediatric trauma, where the sensitivity of POCS is poor. Based on a small number of studies in a mixed population, POCS may have a higher sensitivity in chest injuries. This warrants larger, confirmatory trials to affirm the accuracy of POCS for diagnosing thoracic trauma.
Collapse
Affiliation(s)
- Dirk Stengel
- Unfallkrankenhaus BerlinCentre for Clinical Research, Department of Trauma and Orthopaedic SurgeryBerlinGermany12683
| | | | - Paula Ferrada
- Virginia Commonwealth UniversityDepartment of SurgeryRichmondVAUSA
| | - Axel Ekkernkamp
- University HospitalDepartment of Trauma and Reconstructive SurgeryGreifswaldGermany17475
| | - Sven Mutze
- Unfallkrankenhaus BerlinDepartment of Diagnostic and Interventional RadiologyWarener Str 7BerlinGermany12683
| | - Alexander Hoenning
- Unfallkrankenhaus BerlinCentre for Clinical Research, Department of Trauma and Orthopaedic SurgeryBerlinGermany12683
| | | |
Collapse
|
25
|
Prady SL, Uphoff EP, Power M, Golder S. Development and validation of a search filter to identify equity-focused studies: reducing the number needed to screen. BMC Med Res Methodol 2018; 18:106. [PMID: 30314471 PMCID: PMC6186133 DOI: 10.1186/s12874-018-0567-x] [Citation(s) in RCA: 30] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2018] [Accepted: 10/01/2018] [Indexed: 01/23/2023] Open
Abstract
Background Health inequalities, worse health associated with social and economic disadvantage, are reported by a minority of research articles. Locating these studies when conducting an equity-focused systematic review is challenging due to a deficit in standardised terminology, indexing, and lack of validated search filters. Current reporting guidelines recommend not applying filters, meaning that increased resources are needed at the screening stage. Methods We aimed to design and test search filters to locate studies that reported outcomes by a social determinant of health. We developed and expanded a ‘specific terms strategy’ using keywords and subject headings compiled from recent systematic reviews that applied an equity filter. A ‘non-specific strategy’ was compiled from phrases used to describe equity analyses that were reported in titles and abstracts, and related subject headings. Gold standard evaluation and validation sets were compiled. The filters were developed in MEDLINE, adapted for Embase and tested in both. We set a target of 0.90 sensitivity (95% CI; 0.84, 0.94) in retrieving 150 gold standard validation papers. We noted the reduction in the number needed to screen in a proposed equity-focused systematic review and the proportion of equity-focused reviews we assessed in the project that applied an equity filter to their search strategy. Results The specific terms strategy filtered out 93-95% of all records, and retrieved a validation set of articles with a sensitivity of 0.84 in MEDLINE (0.77, 0.89), and 0.87 (0.81, 0.92) in Embase. When combined (Boolean ‘OR’) with the non-specific strategy sensitivity was 0.92 (0.86, 0.96) in MEDLINE (Embase 0.94; 0.89, 0.97). The number needed to screen was reduced by 77% by applying the specific terms strategy, and by 59.7% (MEDLINE) and 63.5% (Embase) by applying the combined strategy. Eighty-one per cent of systematic reviews filtered studies by equity. Conclusions A combined approach of using specific and non-specific terms is recommended if systematic reviewers wish to filter studies for reporting outcomes by social determinants. Future research should concentrate on the indexing standardisation for equity studies and further development and testing of both specific and non-specific terms for accurate study retrieval. Electronic supplementary material The online version of this article (10.1186/s12874-018-0567-x) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Stephanie L Prady
- Department of Health Sciences, University of York, York, YO10 5DD, UK.
| | - Eleonora P Uphoff
- Department of Health Sciences, University of York, York, YO10 5DD, UK
| | - Madeleine Power
- Department of Health Sciences, University of York, York, YO10 5DD, UK
| | - Su Golder
- Department of Health Sciences, University of York, York, YO10 5DD, UK
| |
Collapse
|