1
|
Hair K, Wilson E, Maksym O, Macleod MR, Sena ES. A Systematic Online Living Evidence Summary of experimental Alzheimer's disease research. J Neurosci Methods 2024; 409:110209. [PMID: 38964475 DOI: 10.1016/j.jneumeth.2024.110209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 05/02/2024] [Accepted: 06/28/2024] [Indexed: 07/06/2024]
Abstract
BACKGROUND Despite extensive investment, the development of effective treatments for Alzheimer's disease (AD) has been largely unsuccessful. To improve translation, it is crucial to ensure the quality and reproducibility of foundational evidence generated from laboratory models. Systematic reviews play a key role in providing an unbiased overview of the evidence, assessing rigour and reporting, and identifying factors that influence reproducibility. However, the sheer pace of evidence generation is prohibitive to evidence synthesis and assessment. NEW METHOD To address these challenges, we have developed AD-SOLES, an integrated workflow of automated tools that collect, curate, and visualise the totality of evidence from in vivo experiments. RESULTS AD-SOLES is a publicly accessible interactive dashboard aiming to surface and expose data from in vivo experiments. It summarises the latest evidence, tracks reporting quality and transparency, and allows research users to easily locate evidence relevant to their specific research question. COMPARISON WITH EXISTING METHODS Using automated screening methodologies within AD-SOLES, systematic reviews can begin at an accelerated starting point compared to traditional approaches. Furthermore, through text-mining approaches within the full-text of publications, users can identify research of interest using specific models, outcomes, or interventions without relying on details in the title and/or abstract. CONCLUSIONS By automating the collection, curation, and visualisation of evidence from in vivo experiments, AD-SOLES addresses the challenges posed by the rapid pace of evidence generation. AD-SOLES aims to offer guidance for research improvement, reduce research waste, highlight knowledge gaps, and support informed decision making for researchers, funders, patients, and the public.
Collapse
Affiliation(s)
- Kaitlyn Hair
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Emma Wilson
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Olena Maksym
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Malcolm R Macleod
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Emily S Sena
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK.
| |
Collapse
|
2
|
Hair K, Bahor Z, Macleod M, Liao J, Sena ES. The Automated Systematic Search Deduplicator (ASySD): a rapid, open-source, interoperable tool to remove duplicate citations in biomedical systematic reviews. BMC Biol 2023; 21:189. [PMID: 37674179 PMCID: PMC10483700 DOI: 10.1186/s12915-023-01686-z] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Accepted: 08/21/2023] [Indexed: 09/08/2023] Open
Abstract
BACKGROUND Researchers performing high-quality systematic reviews search across multiple databases to identify relevant evidence. However, the same publication is often retrieved from several databases. Identifying and removing such duplicates ("deduplication") can be extremely time-consuming, but failure to remove these citations can lead to the wrongful inclusion of duplicate data. Many existing tools are not sensitive enough, lack interoperability with other tools, are not freely accessible, or are difficult to use without programming knowledge. Here, we report the performance of our Automated Systematic Search Deduplicator (ASySD), a novel tool to perform automated deduplication of systematic searches for biomedical reviews. METHODS We evaluated ASySD's performance on 5 unseen biomedical systematic search datasets of various sizes (1845-79,880 citations). We compared the performance of ASySD with EndNote's automated deduplication option and with the Systematic Review Assistant Deduplication Module (SRA-DM). RESULTS ASySD identified more duplicates than either SRA-DM or EndNote, with a sensitivity in different datasets of 0.95 to 0.99. The false-positive rate was comparable to human performance, with a specificity of > 0.99. The tool took less than 1 h to identify and remove duplicates within each dataset. CONCLUSIONS For duplicate removal in biomedical systematic reviews, ASySD is a highly sensitive, reliable, and time-saving tool. It is open source and freely available online as both an R package and a user-friendly web application.
Collapse
Affiliation(s)
- Kaitlyn Hair
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK.
| | - Zsanett Bahor
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Jing Liao
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Emily S Sena
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| |
Collapse
|
3
|
Advanced methods and implementations for the meta-analyses of animal models: Current practices and future recommendations. Neurosci Biobehav Rev 2023; 146:105016. [PMID: 36566804 DOI: 10.1016/j.neubiorev.2022.105016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Revised: 12/19/2022] [Accepted: 12/20/2022] [Indexed: 12/24/2022]
Abstract
Meta-analytic techniques have been widely used to synthesize data from animal models of human diseases and conditions, but these analyses often face two statistical challenges due to complex nature of animal data (e.g., multiple effect sizes and multiple species): statistical dependency and confounding heterogeneity. These challenges can lead to unreliable and less informative evidence, which hinders the translation of findings from animal to human studies. We present a literature survey of meta-analysis using animal models (animal meta-analysis), showing that these issues are not adequately addressed in current practice. To address these challenges, we propose a meta-analytic framework based on multilevel (linear mixed-effects) models. Through conceptualization, formulations, and worked examples, we illustrate how this framework can appropriately address these issues while allowing for testing new questions. Additionally, we introduce other advanced techniques such as multivariate models, robust variance estimation, and meta-analysis of emergent effect sizes, which can deliver robust inferences and novel biological insights. We also provide a tutorial with annotated R code to demonstrate the implementation of these techniques.
Collapse
|
4
|
Zurrer WE, Cannon AE, Ewing E, Rosso M, Reich DS, Ineichen BV. Auto-STEED: A data mining tool for automated extraction of experimental parameters and risk of bias items from in vivo publications. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.02.24.529867. [PMID: 37205453 PMCID: PMC10187249 DOI: 10.1101/2023.02.24.529867] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/21/2023]
Abstract
Background Systematic reviews, i.e., research summaries that address focused questions in a structured and reproducible manner, are a cornerstone of evidence-based medicine and research. However, certain systematic review steps such as data extraction are labour-intensive which hampers their applicability, not least with the rapidly expanding body of biomedical literature. Objective To bridge this gap, we aimed at developing a data mining tool in the R programming environment to automate data extraction from neuroscience in vivo publications. The function was trained on a literature corpus (n=45 publications) of animal motor neuron disease studies and tested in two validation corpora (motor neuron diseases, n=31 publications; multiple sclerosis, n=244 publications). Results Our data mining tool Auto-STEED (Automated and STructured Extraction of Experimental Data) was able to extract key experimental parameters such as animal models and species as well as risk of bias items such as randomization or blinding from in vivo studies. Sensitivity and specificity were over 85 and 80%, respectively, for most items in both validation corpora. Accuracy and F-scores were above 90% and 0.9 for most items in the validation corpora. Time savings were above 99%. Conclusions Our developed text mining tool Auto-STEED is able to extract key experimental parameters and risk of bias items from the neuroscience in vivo literature. With this, the tool can be deployed to probe a field in a research improvement context or to replace one human reader during data extraction resulting in substantial time-savings and contribute towards automation of systematic reviews. The function is available on Github.
Collapse
|
5
|
Borissov N, Haas Q, Minder B, Kopp-Heim D, von Gernler M, Janka H, Teodoro D, Amini P. Reducing systematic review burden using Deduklick: a novel, automated, reliable, and explainable deduplication algorithm to foster medical research. Syst Rev 2022; 11:172. [PMID: 35978441 PMCID: PMC9382798 DOI: 10.1186/s13643-022-02045-9] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Accepted: 08/01/2022] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND Identifying and removing reference duplicates when conducting systematic reviews (SRs) remain a major, time-consuming issue for authors who manually check for duplicates using built-in features in citation managers. To address issues related to manual deduplication, we developed an automated, efficient, and rapid artificial intelligence-based algorithm named Deduklick. Deduklick combines natural language processing algorithms with a set of rules created by expert information specialists. METHODS Deduklick's deduplication uses a multistep algorithm of data normalization, calculates a similarity score, and identifies unique and duplicate references based on metadata fields, such as title, authors, journal, DOI, year, issue, volume, and page number range. We measured and compared Deduklick's capacity to accurately detect duplicates with the information specialists' standard, manual duplicate removal process using EndNote on eight existing heterogeneous datasets. Using a sensitivity analysis, we manually cross-compared the efficiency and noise of both methods. DISCUSSION Deduklick achieved average recall of 99.51%, average precision of 100.00%, and average F1 score of 99.75%. In contrast, the manual deduplication process achieved average recall of 88.65%, average precision of 99.95%, and average F1 score of 91.98%. Deduklick achieved equal to higher expert-level performance on duplicate removal. It also preserved high metadata quality and drastically reduced time spent on analysis. Deduklick represents an efficient, transparent, ergonomic, and time-saving solution for identifying and removing duplicates in SRs searches. Deduklick could therefore simplify SRs production and represent important advantages for scientists, including saving time, increasing accuracy, reducing costs, and contributing to quality SRs.
Collapse
Affiliation(s)
- Nikolay Borissov
- Risklick AG, Spin-Off, University of Bern, Bern, Switzerland
- CTU Bern, University of Bern, Bern, Switzerland
| | - Quentin Haas
- Risklick AG, Spin-Off, University of Bern, Bern, Switzerland
- CTU Bern, University of Bern, Bern, Switzerland
| | - Beatrice Minder
- Public Health & Primary Care Library, University Library of Bern, University of Bern, Bern, Switzerland
| | - Doris Kopp-Heim
- Public Health & Primary Care Library, University Library of Bern, University of Bern, Bern, Switzerland
| | - Marc von Gernler
- Medical Library, University Library of Bern, University of Bern, Bern, Switzerland
| | - Heidrun Janka
- Medical Library, University Library of Bern, University of Bern, Bern, Switzerland
| | - Douglas Teodoro
- University of Applied Sciences and Arts Western Switzerland, Geneva, Switzerland
- Department of Radiology and Medical Informatics, University of Geneva, Geneva, Switzerland
| | - Poorya Amini
- Risklick AG, Spin-Off, University of Bern, Bern, Switzerland, .
- CTU Bern, University of Bern, Bern, Switzerland.
| |
Collapse
|
6
|
Galligioni V, Movia D, Ruiz-Pérez D, Sánchez-Morgado JM, Prina-Mello A. The Case for Modernizing Biomedical Research in Ireland through the Creation of an Irish 3Rs Centre. Animals (Basel) 2022; 12:ani12091078. [PMID: 35565505 PMCID: PMC9105506 DOI: 10.3390/ani12091078] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Revised: 04/11/2022] [Accepted: 04/13/2022] [Indexed: 02/04/2023] Open
Abstract
Simple Summary The 3Rs principle indicates the measures to refine, reduce, and replace the use of animals in research. Thirteen out of the twenty-seven European countries already have 3Rs Centres. These support the development, acceptance, and/or implementation of methods that address the 3Rs principle at a research and educational level. To date, Ireland has no 3Rs centre. In this commentary, we present the reasons for Ireland to embrace the creation of a national Irish 3Rs Centre. We believe the centre will enable Ireland to be ready for the paradigm shift that is internationally occurring in biomedical research, towards the modernisation and replacement of tests carried out in animals when it is scientifically possible to do so. Abstract Since its publication, the 3Rs principle has provided a cornerstone for more ethical and humane biomedical and regulatory research. In Europe, the 3Rs principle has been incorporated into the European Directive 63/2010/EU, with the ultimate aim of fully replacing the procedures on live animals for scientific and educational purposes as soon as it is scientifically possible to do so. Thus, a critical shift in the discussion on animal use in biomedical and regulatory research is undergoing in Europe, a discussion where satisfying the “replacement” principle is becoming more and more defined as a scientific rather than ethical need. 3Rs Centres have been established in recent years across Europe. To date, Ireland has no 3Rs Centre, and the uptake of the 3Rs principle, and in particular of the “replacement” aspect, has been slow. In this Commentary, we present the Irish context of the use of animal models in biomedical and regulatory research, and urge for what, in the authors’ opinion, are the most critical actions that Ireland must undertake to align its biomedical (basic, applied and translational) research with the European 3Rs strategy.
Collapse
Affiliation(s)
- Viola Galligioni
- Comparative Medicine Unit, Trinity College Dublin, College Green, D02 PN40 Dublin, Ireland; (V.G.); (D.R.-P.)
| | - Dania Movia
- Laboratory for Biological Characterisation of Advanced Materials (LBCAM), Trinity Translational Medicine Institute/Department of Clinical Medicine, School of Medicine, Trinity College Dublin, James’s Street, D08 W9RT Dublin, Ireland;
| | - Daniel Ruiz-Pérez
- Comparative Medicine Unit, Trinity College Dublin, College Green, D02 PN40 Dublin, Ireland; (V.G.); (D.R.-P.)
| | - José Manuel Sánchez-Morgado
- Comparative Medicine Unit, Trinity College Dublin, College Green, D02 PN40 Dublin, Ireland; (V.G.); (D.R.-P.)
- Correspondence: (J.M.S.-M.); (A.P.-M.); Tel.: +353-1-896-3260 (A.P.-M.)
| | - Adriele Prina-Mello
- Laboratory for Biological Characterisation of Advanced Materials (LBCAM), Trinity Translational Medicine Institute/Department of Clinical Medicine, School of Medicine, Trinity College Dublin, James’s Street, D08 W9RT Dublin, Ireland;
- Correspondence: (J.M.S.-M.); (A.P.-M.); Tel.: +353-1-896-3260 (A.P.-M.)
| |
Collapse
|