1
|
Stevens ER, Laynor G. Enhancing the quality and efficiency of regulatory science literature reviews through innovation and collaboration with library and information science experts. Front Med (Lausanne) 2024; 11:1434427. [PMID: 39021816 PMCID: PMC11251899 DOI: 10.3389/fmed.2024.1434427] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2024] [Accepted: 06/24/2024] [Indexed: 07/20/2024] Open
Affiliation(s)
- Elizabeth R. Stevens
- Department of Population Health, New York University Grossman School of Medicine, New York, NY, United States
| | - Gregory Laynor
- Health Sciences Library, New York University Grossman School of Medicine, New York, NY, United States
| |
Collapse
|
2
|
Escaldelai FMD, Escaldelai L, Bergamaschi DP. Systematic Review Support software system: web-based solution for managing duplicates and screening eligible studies. REVISTA BRASILEIRA DE EPIDEMIOLOGIA 2022; 25:e220030. [PMID: 36259890 DOI: 10.1590/1980-549720220030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2022] [Accepted: 08/16/2022] [Indexed: 06/16/2023] Open
Abstract
OBJECTIVE To describe the main functions of the "Systematic Review Support" web-based system for removing duplicate articles and aiding eligibility analysis during the process of conducting systematic review studies. METHODS The system was developed based on the incremental build model using the Agile methodology. The software is proprietary source code and was published on a proprietary platform. The architecture of the production environment allows the infrastructure used to increase or decrease according to demand. The system functions are presented with insertion of screenshots of the interfaces of the version for personal computers during the simulation of a systematic review. RESULTS After importing the files containing the abstracts retrieved from the Pubmed, Embase, and Web of Science databases, the system identifies and removes duplicates for later reading and analysis of title and abstract, a stage which can be performed by one or more reviewers independently. After unblinding of reviewers, the decisions on the eligibility of the studies are compared automatically to help the researchers reach a consensus on any disagreements. Results can be filtered and a PDF produced containing the eligible studies. CONCLUSION Version 1.0 of the system is available on the web (sysrev.azurewebsites.net) to assist researchers in the initial stages of systematic reviews.
Collapse
|
3
|
Schneider J, Hoang L, Kansara Y, Cohen AM, Smalheiser NR. Evaluation of publication type tagging as a strategy to screen randomized controlled trial articles in preparing systematic reviews. JAMIA Open 2022; 5:ooac015. [PMID: 35571360 PMCID: PMC9097760 DOI: 10.1093/jamiaopen/ooac015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2020] [Revised: 02/06/2021] [Accepted: 03/24/2021] [Indexed: 11/29/2022] Open
Abstract
Objectives To produce a systematic review (SR), reviewers typically screen thousands of titles and abstracts of articles manually to find a small number which are read in full text to find relevant articles included in the final SR. Here, we evaluate a proposed automated probabilistic publication type screening strategy applied to the randomized controlled trial (RCT) articles (i.e., those which present clinical outcome results of RCT studies) included in a corpus of previously published Cochrane reviews. Materials and Methods We selected a random subset of 558 published Cochrane reviews that specified RCT study only inclusion criteria, containing 7113 included articles which could be matched to PubMed identifiers. These were processed by our automated RCT Tagger tool to estimate the probability that each article reports clinical outcomes of a RCT. Results Removing articles with low predictive scores P < 0.01 eliminated 288 included articles, of which only 22 were actually typical RCT articles, and only 18 were actually typical RCT articles that MEDLINE indexed as such. Based on our sample set, this screening strategy led to fewer than 0.05 relevant RCT articles being missed on average per Cochrane SR. Discussion This scenario, based on real SRs, demonstrates that automated tagging can identify RCT articles accurately while maintaining very high recall. However, we also found that even SRs whose inclusion criteria are restricted to RCT studies include not only clinical outcome articles per se, but a variety of ancillary article types as well. Conclusions This encourages further studies learning how best to incorporate automated tagging of additional publication types into SR triage workflows.
Collapse
Affiliation(s)
- Jodi Schneider
- School of Information Sciences, University of Illinois
Urbana-Champaign, Champaign, Illinois, USA,Corresponding Author: Jodi Schneider, School of Information
Sciences, University of Illinois Urbana-Champaign, 501 E. Daniel St., MC-493, Champaign,
IL 61820, USA;
| | - Linh Hoang
- School of Information Sciences, University of Illinois
Urbana-Champaign, Champaign, Illinois, USA
| | - Yogeshwar Kansara
- School of Information Sciences, University of Illinois
Urbana-Champaign, Champaign, Illinois, USA
| | - Aaron M Cohen
- Department of Medical Informatics and Clinical Epidemiology (DMICE), School of
Medicine, Oregon Health & Science University, Portland, Oregon, USA
| | - Neil R Smalheiser
- Department of Psychiatry, College of Medicine, University of Illinois
Chicago, Chicago, Illinois, USA
| |
Collapse
|
4
|
Escaldelai FMD, Escaldelai L, Bergamaschi DP. Sistema “Apoio à Revisão Sistemática”: solução web para gerenciamento de duplicatas e seleção de artigos elegíveis. REVISTA BRASILEIRA DE EPIDEMIOLOGIA 2022. [DOI: 10.1590/1980-549720220030.2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
Abstract
RESUMO Objetivo: Descrever as principais funcionalidades do sistema “Apoio à Revisão Sistemática” na identificação e exclusão de artigos duplicados e no auxílio na análise de elegibilidade durante a condução de estudo de revisão sistemática. Métodos: O sistema foi desenvolvido com base em um modelo de processo incremental, utilizando-se metodologia Ágil. É de código fechado e foi publicado em plataforma proprietária. O ambiente de produção onde o sistema foi implantado possui arquitetura que permite que a infraestrutura utilizada aumente ou diminua conforme a demanda. As funcionalidades foram apresentadas com inserção de imagens das interfaces da versão para computadores, simulando uma revisão sistemática. Resultados: Após a importação dos resumos recuperados nas bases de dados PubMed, Embase e Web of Science, o sistema permite a identificação e eliminação de duplicatas para posterior leitura e análise de título e resumo, etapa que pode ser realizada por mais de um revisor de maneira independente. Após a quebra do cegamento entre os revisores, as respostas sobre a elegibilidade dos estudos podem ser comparadas automaticamente para facilitar a resolução de divergências pelos pesquisadores. É possível filtrar os resultados e gerar um arquivo PDF com os estudos elegíveis. Conclusão: A versão 1.0 do sistema “Apoio à Revisão Sistemática” encontra-se disponível na web (sysrev.azurewebsites.net) para auxiliar pesquisadores nas etapas iniciais de um estudo de revisão sistemática.
Collapse
|
5
|
Artificial Intelligence in Evidence-Based Medicine. Artif Intell Med 2022. [DOI: 10.1007/978-3-030-64573-1_43] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
6
|
Harrington L. COVID-19 Technology-Enabled Living Systematic Reviews to Enhance Knowledge Translation. AACN Adv Crit Care 2021; 32:133-136. [PMID: 34161967 DOI: 10.4037/aacnacc2021948] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/01/2022]
Affiliation(s)
- Linda Harrington
- Linda Harrington is an Independent Consultant, Health Informatics and Digital Strategy, and Adjunct Faculty at Texas Christian University, 2800 South University Drive, Fort Worth, TX 76109
| |
Collapse
|
7
|
Artificial Intelligence in Evidence-Based Medicine. Artif Intell Med 2021. [DOI: 10.1007/978-3-030-58080-3_43-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
8
|
Whaley P, Edwards SW, Kraft A, Nyhan K, Shapiro A, Watford S, Wattam S, Wolffe T, Angrish M. Knowledge Organization Systems for Systematic Chemical Assessments. ENVIRONMENTAL HEALTH PERSPECTIVES 2020; 128:125001. [PMID: 33356525 PMCID: PMC7759237 DOI: 10.1289/ehp6994] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/03/2020] [Revised: 12/02/2020] [Accepted: 12/04/2020] [Indexed: 05/04/2023]
Abstract
BACKGROUND Although the implementation of systematic review and evidence mapping methods stands to improve the transparency and accuracy of chemical assessments, they also accentuate the challenges that assessors face in ensuring they have located and included all the evidence that is relevant to evaluating the potential health effects an exposure might be causing. This challenge of information retrieval can be characterized in terms of "semantic" and "conceptual" factors that render chemical assessments vulnerable to the streetlight effect. OBJECTIVES This commentary presents how controlled vocabularies, thesauruses, and ontologies contribute to overcoming the streetlight effect in information retrieval, making up the key components of Knowledge Organization Systems (KOSs) that enable more systematic access to assessment-relevant information than is currently achievable. The concept of Adverse Outcome Pathways is used to illustrate what a general KOS for use in chemical assessment could look like. DISCUSSION Ontologies are an underexploited element of effective knowledge organization in the environmental health sciences. Agreeing on and implementing ontologies in chemical assessment is a complex but tractable process with four fundamental steps. Successful implementation of ontologies would not only make currently fragmented information about health risks from chemical exposures vastly more accessible, it could ultimately enable computational methods for chemical assessment that can take advantage of the full richness of data described in natural language in primary studies. https://doi.org/10.1289/EHP6994.
Collapse
Affiliation(s)
- Paul Whaley
- Evidence Based Toxicology Collaboration, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA
- Lancaster Environment Centre, Lancaster University, Lancaster, UK
| | - Stephen W. Edwards
- GenOmics, Bioinformatics, and Translational Research Center, RTI International, Research Triangle Park, North Carolina, USA
| | - Andrew Kraft
- Chemical Pollutant Assessment Division, Center for Public Health and Environmental Assessment, U.S. Environmental Protection Agency (U.S. EPA), Washington, DC, USA
| | - Kate Nyhan
- Environmental Health Sciences, Yale School of Public Health and Harvey Cushing/John Hay Whitney Medical Library, Yale University, New Haven, Connecticut, USA
| | - Andrew Shapiro
- Chemical Pollutant Assessment Division, Center for Public Health and Environmental Assessment, U.S. Environmental Protection Agency (U.S. EPA), Washington, DC, USA
| | - Sean Watford
- National Center for Computational Toxicology, U.S. EPA, Durham, North Carolina, USA
| | - Steve Wattam
- WAP Academy Consultancy Ltd, Thirsk, Yorkshire, UK
| | - Taylor Wolffe
- Lancaster Environment Centre, Lancaster University, Lancaster, UK
| | - Michelle Angrish
- Chemical Pollutant Assessment Division, Center for Public Health and Environmental Assessment, U.S. EPA, Durham, North Carolina, USA
| |
Collapse
|
9
|
Hirt J, Nordhausen T, Appenzeller-Herzog C, Ewald H. Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and an expert survey. F1000Res 2020; 9:1386. [PMID: 34631036 PMCID: PMC8474097 DOI: 10.12688/f1000research.27337.1] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 11/23/2020] [Indexed: 09/22/2023] Open
Abstract
Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed. Objective: We intend to develop recommendations for the use of citation tracking in health-related systematic literature searching. Our study will be guided by the following research questions: What are the benefits of citation tracking for health-related systematic literature searching? Which perspectives and experiences do experts in the field of literature retrieval methods have with regard to citation tracking in health-related systematic literature searching? Methods: Our study will have two parts: a scoping review and an expert survey. The scoping review aims at identifying methodological studies on benefits or problems of citation tracking in health-related systematic literature searching with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE, The Cumulative Index to Nursing and Allied Health Literature (CINAHL), Web of Science Core Collection, two information science databases, and free web searching. Two reviewers will independently assess full texts of selected abstracts. We will conduct direct backward and forward citation tracking on included articles. The results of the scoping review will inform our expert survey through which we aim to learn about experts΄ perspectives and experiences. We will narratively synthesize the results and derive recommendations for performing health-related systematic reviews.
Collapse
Affiliation(s)
- Julian Hirt
- Institute of Applied Nursing Science, Department of Health, Eastern Switzerland University of Applied Sciences (formerly FHS St.Gallen), St.Gallen, Switzerland
- International Graduate Academy, Institute of Health and Nursing Science, Medical Faculty, Martin Luther University Halle-Wittenberg, Halle (Saale), Germany
| | - Thomas Nordhausen
- International Graduate Academy, Institute of Health and Nursing Science, Medical Faculty, Martin Luther University Halle-Wittenberg, Halle (Saale), Germany
| | | | - Hannah Ewald
- University Medical Library, University of Basel, Basel, Switzerland
| |
Collapse
|
10
|
Büchter RB, Weise A, Pieper D. Development, testing and use of data extraction forms in systematic reviews: a review of methodological guidance. BMC Med Res Methodol 2020; 20:259. [PMID: 33076832 PMCID: PMC7574308 DOI: 10.1186/s12874-020-01143-3] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2020] [Accepted: 10/07/2020] [Indexed: 01/08/2023] Open
Abstract
Background Data extraction forms link systematic reviews with primary research and provide the foundation for appraising, analysing, summarising and interpreting a body of evidence. This makes their development, pilot testing and use a crucial part of the systematic reviews process. Several studies have shown that data extraction errors are frequent in systematic reviews, especially regarding outcome data. Methods We reviewed guidance on the development and pilot testing of data extraction forms and the data extraction process. We reviewed four types of sources: 1) methodological handbooks of systematic review organisations (SRO); 2) textbooks on conducting systematic reviews; 3) method documents from health technology assessment (HTA) agencies and 4) journal articles. HTA documents were retrieved in February 2019 and database searches conducted in December 2019. One author extracted the recommendations and a second author checked them for accuracy. Results are presented descriptively. Results Our analysis includes recommendations from 25 documents: 4 SRO handbooks, 11 textbooks, 5 HTA method documents and 5 journal articles. Across these sources the most common recommendations on form development are to use customized or adapted standardised extraction forms (14/25); provide detailed instructions on their use (10/25); ensure clear and consistent coding and response options (9/25); plan in advance which data are needed (9/25); obtain additional data if required (8/25); and link multiple reports of the same study (8/25). The most frequent recommendations on piloting extractions forms are that forms should be piloted on a sample of studies (18/25); and that data extractors should be trained in the use of the forms (7/25). The most frequent recommendations on data extraction are that extraction should be conducted by at least two people (17/25); that independent parallel extraction should be used (11/25); and that procedures to resolve disagreements between data extractors should be in place (14/25). Conclusions Overall, our results suggest a lack of comprehensiveness of recommendations. This may be particularly problematic for less experienced reviewers. Limitations of our method are the scoping nature of the review and that we did not analyse internal documents of health technology agencies.
Collapse
Affiliation(s)
- Roland Brian Büchter
- Institute for Research in Operative Medicine (IFOM), Faculty of Health - School of Medicine, Witten/Herdecke University, Ostmerheimer Str. 200, 51109, Cologne, Germany.
| | - Alina Weise
- Institute for Research in Operative Medicine (IFOM), Faculty of Health - School of Medicine, Witten/Herdecke University, Ostmerheimer Str. 200, 51109, Cologne, Germany
| | - Dawid Pieper
- Institute for Research in Operative Medicine (IFOM), Faculty of Health - School of Medicine, Witten/Herdecke University, Ostmerheimer Str. 200, 51109, Cologne, Germany
| |
Collapse
|
11
|
Hamel C, Michaud A, Thuku M, Affengruber L, Skidmore B, Nussbaumer-Streit B, Stevens A, Garritty C. Few evaluative studies exist examining rapid review methodology across stages of conduct: a systematic scoping review. J Clin Epidemiol 2020; 126:131-140. [DOI: 10.1016/j.jclinepi.2020.06.027] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2020] [Revised: 06/18/2020] [Accepted: 06/23/2020] [Indexed: 10/24/2022]
|