1
|
Tóth B, Berek L, Gulácsi L, Péntek M, Zrubka Z. Automation of systematic reviews of biomedical literature: a scoping review of studies indexed in PubMed. Syst Rev 2024; 13:174. [PMID: 38978132 PMCID: PMC11229257 DOI: 10.1186/s13643-024-02592-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Accepted: 06/20/2024] [Indexed: 07/10/2024] Open
Abstract
BACKGROUND The demand for high-quality systematic literature reviews (SRs) for evidence-based medical decision-making is growing. SRs are costly and require the scarce resource of highly skilled reviewers. Automation technology has been proposed to save workload and expedite the SR workflow. We aimed to provide a comprehensive overview of SR automation studies indexed in PubMed, focusing on the applicability of these technologies in real world practice. METHODS In November 2022, we extracted, combined, and ran an integrated PubMed search for SRs on SR automation. Full-text English peer-reviewed articles were included if they reported studies on SR automation methods (SSAM), or automated SRs (ASR). Bibliographic analyses and knowledge-discovery studies were excluded. Record screening was performed by single reviewers, and the selection of full text papers was performed in duplicate. We summarized the publication details, automated review stages, automation goals, applied tools, data sources, methods, results, and Google Scholar citations of SR automation studies. RESULTS From 5321 records screened by title and abstract, we included 123 full text articles, of which 108 were SSAM and 15 ASR. Automation was applied for search (19/123, 15.4%), record screening (89/123, 72.4%), full-text selection (6/123, 4.9%), data extraction (13/123, 10.6%), risk of bias assessment (9/123, 7.3%), evidence synthesis (2/123, 1.6%), assessment of evidence quality (2/123, 1.6%), and reporting (2/123, 1.6%). Multiple SR stages were automated by 11 (8.9%) studies. The performance of automated record screening varied largely across SR topics. In published ASR, we found examples of automated search, record screening, full-text selection, and data extraction. In some ASRs, automation fully complemented manual reviews to increase sensitivity rather than to save workload. Reporting of automation details was often incomplete in ASRs. CONCLUSIONS Automation techniques are being developed for all SR stages, but with limited real-world adoption. Most SR automation tools target single SR stages, with modest time savings for the entire SR process and varying sensitivity and specificity across studies. Therefore, the real-world benefits of SR automation remain uncertain. Standardizing the terminology, reporting, and metrics of study reports could enhance the adoption of SR automation techniques in real-world practice.
Collapse
Affiliation(s)
- Barbara Tóth
- Doctoral School of Innovation Management, Óbuda University, Bécsi út 96/B, Budapest, 1034, Hungary
| | - László Berek
- Doctoral School for Safety and Security, Óbuda University, Bécsi út 96/B, Budapest, 1034, Hungary
- University Library, Óbuda University, Bécsi út 96/B, Budapest, 1034, Hungary
| | - László Gulácsi
- HECON Health Economics Research Center, University Research, and Innovation Center, Óbuda University, Bécsi út 96/B, Budapest, 1034, Hungary
| | - Márta Péntek
- HECON Health Economics Research Center, University Research, and Innovation Center, Óbuda University, Bécsi út 96/B, Budapest, 1034, Hungary
| | - Zsombor Zrubka
- HECON Health Economics Research Center, University Research, and Innovation Center, Óbuda University, Bécsi út 96/B, Budapest, 1034, Hungary.
| |
Collapse
|
2
|
Becker RA, Bianchi E, LaRocca J, Marty MS, Mehta V. Identifying the landscape of developmental toxicity new approach methodologies. Birth Defects Res 2022; 114:1123-1137. [PMID: 36205106 DOI: 10.1002/bdr2.2075] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 07/08/2022] [Accepted: 07/21/2022] [Indexed: 01/07/2023]
Abstract
BACKGROUND The dynamics and complexities of in utero fetal development create significant challenges in transitioning from lab animal-centric developmental toxicity testing methods to assessment strategies based on new approach methodologies (NAMs). Nevertheless, considerable progress is being made, stimulated by increased research investments and scientific advances, such as induced pluripotent stem cell-derived models. To help identify developmental toxicity NAMs for toxicity screening and potential funding through the American Chemistry Council's Long-Range Research Initiative, a systematic literature review was conducted to better understand the current landscape of developmental toxicity NAMs. METHODS Scoping review tools were used to systematically survey the literature (2010-2021; ~18,000 references identified), results and metadata were then extracted, and a user-friendly interactive dashboard was created. RESULTS The data visualization dashboard, developed using Tableau® software, is provided as a free, open-access web tool. This dashboard enables straightforward interactive queries and visualizations to identify trends and to distinguish and understand areas or NAMs where research has been most, or least focused. CONCLUSIONS Herein, we describe the approach and methods used, summarize the benefits and challenges of applying the systematic-review techniques, and highlight the types of questions and answers for which the dashboard can be used to explore the many different facets of developmental toxicity NAMs.
Collapse
Affiliation(s)
- Richard A Becker
- American Chemistry Council, Washington, District of Columbia, USA
| | | | | | | | | |
Collapse
|
3
|
A Narrative Literature Review of Natural Language Processing Applied to the Occupational Exposome. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19148544. [PMID: 35886395 PMCID: PMC9316260 DOI: 10.3390/ijerph19148544] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 07/07/2022] [Accepted: 07/11/2022] [Indexed: 02/05/2023]
Abstract
The evolution of the Exposome concept revolutionised the research in exposure assessment and epidemiology by introducing the need for a more holistic approach on the exploration of the relationship between the environment and disease. At the same time, further and more dramatic changes have also occurred on the working environment, adding to the already existing dynamic nature of it. Natural Language Processing (NLP) refers to a collection of methods for identifying, reading, extracting and untimely transforming large collections of language. In this work, we aim to give an overview of how NLP has successfully been applied thus far in Exposome research. Methods: We conduct a literature search on PubMed, Scopus and Web of Science for scientific articles published between 2011 and 2021. We use both quantitative and qualitative methods to screen papers and provide insights into the inclusion and exclusion criteria. We outline our approach for article selection and provide an overview of our findings. This is followed by a more detailed insight into selected articles. Results: Overall, 6420 articles were screened for the suitability of this review, where we review 37 articles in depth. Finally, we discuss future avenues of research and outline challenges in existing work. Conclusions: Our results show that (i) there has been an increase in articles published that focus on applying NLP to exposure and epidemiology research, (ii) most work uses existing NLP tools and (iii) traditional machine learning is the most popular approach.
Collapse
|
4
|
Stenzl A, Sternberg CN, Ghith J, Serfass L, Schijvenaars BJA, Sboner A. Application of Artificial Intelligence to Overcome Clinical Information Overload in Urologic Cancer. BJU Int 2021; 130:291-300. [PMID: 34846775 DOI: 10.1111/bju.15662] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
OBJECTIVE To describe the use of artificial intelligence (AI) in medical literature and trial data extraction, and its applications in uro-oncology. This bridging review, which consolidates information from the diverse applications of AI, highlights how AI users can investigate more sophisticated queries than with traditional methods, leading to synthesis of raw data and complex outputs into more actionable and personalized results, particularly in the field of uro-oncology. METHODS Literature and clinical trial searches were performed in PubMed, Dimensions, Embase and Google (1999-2020). The searches focused on the use of AI and its various forms to facilitate literature searches, clinical guidelines development, and clinical trial data extraction in uro-oncology. To illustrate how AI can be applied toaddress questions about optimizing therapeutic decision making and individualizing treatment regimens, the Dimensions-linked information platform was searched for "prostate cancer" keywords (76 publications were identified; 48 were included). RESULTS AI offers the promise of transforming raw data and complex outputs into actionable insights. Literature and clinical trial searches can be automated, enabling clinicians to develop and analyze publications expeditiously on complex issues such as therapeutic sequencing and to obtain updates on documents that evolve at the pace and scope of the landscape. An AI-based platform inclusive of 12 trial databases and >100 scientific literature sources enabled the creation of an interactive visualization. CONCLUSION As the literature and clinical trial landscape continues to grow in complexity and with increasing speed, the ability to pull the right information at the right time from different search engines and resources while excluding social media bias becomes more challenging. This review demonstrates that by applying natural language processing and machine learning algorithms, validated and optimized AI leads to a speedier, more personalized, efficient and focused search compared with traditional methods.
Collapse
Affiliation(s)
- Arnulf Stenzl
- Department of Urology, University of Tübingen, Tübingen, Germany
| | - Cora N Sternberg
- Clinical Director, Englander Institute for Precision Medicine, Professor of Medicine, Weill Cornell Medicine Hematology/Oncology, Sandra and Edward Meyer Cancer Center, New York, NY, USA
| | | | | | | | - Andrea Sboner
- Director of Informatics and Computational Biology, Englander Institute for Precision Medicine; Assistant Professor at the Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, NY, USA
| |
Collapse
|
5
|
Robledo S, Grisales Aguirre AM, Hughes M, Eggers F. “Hasta la vista, baby” – will machine learning terminate human literature reviews in entrepreneurship? JOURNAL OF SMALL BUSINESS MANAGEMENT 2021. [DOI: 10.1080/00472778.2021.1955125] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Sebastian Robledo
- Faculty of Management Science, Universidad Católica Luis Amigó, Program of Management, Colombia
- Centro de Bioinformática y Biología Computacional de Colombia, BIOS, Ecoparque Los Yarumos, Edificio BIOS Manizales, Colombia
| | - Andrés Mauricio Grisales Aguirre
- Basic Sciences, Faculty of Management Science, Program of Management, Universidad Católica Luis Amigó, Colombia
- Mathematical Sciences, Caldas University, Colombia
| | - Mathew Hughes
- School of Business and Economics, Loughborough University, UK
| | | |
Collapse
|
6
|
De Silva K, Mathews N, Teede H, Forbes A, Jönsson D, Demmer RT, Enticott J. Clinical notes as prognostic markers of mortality associated with diabetes mellitus following critical care: A retrospective cohort analysis using machine learning and unstructured big data. Comput Biol Med 2021; 132:104305. [PMID: 33705995 DOI: 10.1016/j.compbiomed.2021.104305] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2020] [Revised: 02/23/2021] [Accepted: 02/27/2021] [Indexed: 12/14/2022]
Abstract
BACKGROUND Clinical notes are ubiquitous resources offering potential value in optimizing critical care via data mining technologies. OBJECTIVE To determine the predictive value of clinical notes as prognostic markers of 1-year all-cause mortality among people with diabetes following critical care. MATERIALS AND METHODS Mortality of diabetes patients were predicted using three cohorts of clinical text in a critical care database, written by physicians (n = 45253), nurses (159027), and both (n = 204280). Natural language processing was used to pre-process text documents and LASSO-regularized logistic regression models were trained and tested. Confusion matrix metrics of each model were calculated and AUROC estimates between models were compared. All predictive words and corresponding coefficients were extracted. Outcome probability associated with each text document was estimated. RESULTS Models built on clinical text of physicians, nurses, and the combined cohort predicted mortality with AUROC of 0.996, 0.893, and 0.922, respectively. Predictive performance of the models significantly differed from one another whereas inter-rater reliability ranged from substantial to almost perfect across them. Number of predictive words with non-zero coefficients were 3994, 8159, and 10579, respectively, in the models of physicians, nurses, and the combined cohort. Physicians' and nursing notes, both individually and when combined, strongly predicted 1-year all-cause mortality among people with diabetes following critical care. CONCLUSION Clinical notes of physicians and nurses are strong and novel prognostic markers of diabetes-associated mortality in critical care, offering potentially generalizable and scalable applications. Clinical text-derived personalized risk estimates of prognostic outcomes such as mortality could be used to optimize patient care.
Collapse
Affiliation(s)
- Kushan De Silva
- Monash Centre for Health Research and Implementation, School of Public Health and Preventive Medicine, Faculty of Medicine, Nursing, and Health Sciences, Monash University, Clayton, 3168, Australia.
| | - Noel Mathews
- Monash Centre for Health Research and Implementation, School of Public Health and Preventive Medicine, Faculty of Medicine, Nursing, and Health Sciences, Monash University, Clayton, 3168, Australia
| | - Helena Teede
- Monash Centre for Health Research and Implementation, School of Public Health and Preventive Medicine, Faculty of Medicine, Nursing, and Health Sciences, Monash University, Clayton, 3168, Australia
| | - Andrew Forbes
- Biostatistics Unit, Division of Research Methodology, School of Public Health and Preventive Medicine, Faculty of Medicine, Nursing, and Health Sciences, Monash University, Melbourne, 3004, Australia
| | - Daniel Jönsson
- Department of Periodontology, Faculty of Odontology, Malmö University, Malmö, 21119, Sweden; Swedish Dental Service of Skane, Lund, 22647, Sweden
| | - Ryan T Demmer
- Division of Epidemiology and Community Health, School of Public Health, University of Minnesota, Minneapolis, MN, USA; Mailman School of Public Health, Columbia University, New York, USA
| | - Joanne Enticott
- Monash Centre for Health Research and Implementation, School of Public Health and Preventive Medicine, Faculty of Medicine, Nursing, and Health Sciences, Monash University, Clayton, 3168, Australia
| |
Collapse
|
7
|
Anderson DM, Cronk R, Fejfar D, Pak E, Cawley M, Bartram J. Safe Healthcare Facilities: A Systematic Review on the Costs of Establishing and Maintaining Environmental Health in Facilities in Low- and Middle-Income Countries. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:817. [PMID: 33477905 PMCID: PMC7833392 DOI: 10.3390/ijerph18020817] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/06/2020] [Revised: 01/14/2021] [Accepted: 01/15/2021] [Indexed: 01/21/2023]
Abstract
A hygienic environment is essential to provide quality patient care and prevent healthcare-acquired infections. Understanding costs is important to budget for service delivery, but costs evidence for environmental health services (EHS) in healthcare facilities (HCFs) is lacking. We present the first systematic review to evaluate the costs of establishing, operating, and maintaining EHS in HCFs in low- and middle-income countries (LMICs). We systematically searched for studies costing water, sanitation, hygiene, cleaning, waste management, personal protective equipment, vector control, laundry, and lighting in LMICs. Our search yielded 36 studies that reported costs for 51 EHS. There were 3 studies that reported costs for water, 3 for sanitation, 4 for hygiene, 13 for waste management, 16 for cleaning, 2 for personal protective equipment, 10 for laundry, and none for lighting or vector control. Quality of evidence was low. Reported costs were rarely representative of the total costs of EHS provision. Unit costs were infrequently reported. This review identifies opportunities to improve costing research through efforts to categorize and disaggregate EHS costs, greater dissemination of existing unpublished data, improvements to indicators to monitor EHS demand and quality necessary to contextualize costs, and development of frameworks to define EHS needs and essential inputs to guide future costing.
Collapse
Affiliation(s)
- Darcy M. Anderson
- The Water Institute, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA; (D.F.); (E.P.); (J.B.)
| | - Ryan Cronk
- ICF International, Durham, NC 27713, USA;
| | - Donald Fejfar
- The Water Institute, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA; (D.F.); (E.P.); (J.B.)
| | - Emily Pak
- The Water Institute, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA; (D.F.); (E.P.); (J.B.)
| | - Michelle Cawley
- Health Sciences Library, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA;
| | - Jamie Bartram
- The Water Institute, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA; (D.F.); (E.P.); (J.B.)
- School of Civil Engineering, University of Leeds, Leeds LS2 9JT, UK
| |
Collapse
|
8
|
Escrivá L, Hessel E, Gustafsson S, van Spronsen R, Svanberg M, Beronius A. A validated search filter for the identification of endocrine disruptors based on the ECHA/EFSA guidance recommendations. ENVIRONMENT INTERNATIONAL 2020; 142:105828. [PMID: 32502797 DOI: 10.1016/j.envint.2020.105828] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/14/2020] [Revised: 05/08/2020] [Accepted: 05/19/2020] [Indexed: 06/11/2023]
Abstract
A guidance document for the identification of endocrine disruptors (EDs) in the regulatory assessment of plant protection products (PPP) and biocidal products (BP) has been published by the European Chemical Agency (ECHA) and the European Food Safety Authority (EFSA). The ECHA/EFSA guidance, mainly addressing EATS (estrogen, androgen, thyroid, steroidogenesis) modalities, is intended to guide applicants and assessors of the competent regulatory authorities on the implementation of the scientific criteria for the determination of ED properties pursuant to the recently implemented PPP (EU 2018/605) and BP (EU 2017/2100) EU Regulations. In this study, a search filter for targeted literature search in context of assessing if a substance can be identified as an ED relevant for human health was developed and validated. Development of the search filter was based on the search strategy presented in the ECHA/EFSA guidance and using the estrogenic chemical Bisphenol AF (BPAF) as a model substance. Information specialists from two independent institutions developed refined search filters based on the suggested original search strategy published (ECHA/EFSA guidance - Appendix F). Articles identified by a systematic literature search for BPAF were screened for relevance with inclusion and exclusion criteria by two independent reviewers obtaining positive (relevant) and negative (irrelevant) controls. The developed search filter was quantitatively evaluated in terms of sensitivity, specificity and precision based on the positive and negative controls. The developed filter was then validated for T modality by its application to the known thyroid-disruptor perchlorate. The result is a sensitive search filter with sufficient specificity, which can be applied for all chemicals where a targeted literature search is needed to assess and identify ED properties of chemicals with relevance for humans. Future application of the filter to a broader range of chemicals may identify further points of improvement.
Collapse
Affiliation(s)
- Laura Escrivá
- Karolinska Institutet, Institute of Environmental Medicine, Stockholm, Sweden; Laboratory of Food Chemistry and Toxicology, Faculty of Pharmacy, University of Valencia, Burjassot, Spain.
| | - Ellen Hessel
- National Institute for Public Health and the Environment (RIVM), Utrecht, the Netherlands.
| | | | - Rob van Spronsen
- National Institute for Public Health and the Environment (RIVM), Utrecht, the Netherlands.
| | | | - Anna Beronius
- Karolinska Institutet, Institute of Environmental Medicine, Stockholm, Sweden.
| |
Collapse
|