1
|
Vacek A, Mair G, White P, Bath PM, Muir KW, Al-Shahi Salman R, Martin C, Dye D, Chappell FM, von Kummer R, Macleod M, Sprigg N, Wardlaw JM. Evaluating artificial intelligence software for delineating hemorrhage extent on CT brain imaging in stroke: AI delineation of ICH on CT. J Stroke Cerebrovasc Dis 2024; 33:107512. [PMID: 38007987 DOI: 10.1016/j.jstrokecerebrovasdis.2023.107512] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Revised: 10/25/2023] [Accepted: 11/21/2023] [Indexed: 11/28/2023] Open
Abstract
BACKGROUND The extent and distribution of intracranial hemorrhage (ICH) directly affects clinical management. Artificial intelligence (AI) software can detect and may delineate ICH extent on brain CT. We evaluated e-ASPECTS software (Brainomix Ltd.) performance for ICH delineation. METHODS We qualitatively assessed software delineation of ICH on CT using patients from six stroke trials. We assessed hemorrhage delineation in five compartments: lobar, deep, posterior fossa, intraventricular, extra-axial. We categorized delineation as excellent, good, moderate, or poor. We assessed quality of software delineation with number of affected compartments in univariate analysis (Kruskall-Wallis test) and ICH location using logistic regression (dependent variable: dichotomous delineation categories 'excellent-good' versus 'moderate-poor'), and report odds ratios (OR) and 95 % confidence intervals (95 %CI). RESULTS From 651 patients with ICH (median age 75 years, 53 % male), we included 628 with assessable CTs. Software delineation of ICH extent was 'excellent' in 189/628 (30 %), 'good' in 255/628 (41 %), 'moderate' in 127/628 (20 %), and 'poor' in 57/628 cases (9 %). The quality of software delineation of ICH was better when fewer compartments were affected (Z = 3.61-6.27; p = 0.0063). Software delineation of ICH extent was more likely to be 'excellent-good' quality when lobar alone (OR = 1.56, 95 %CI = 0.97-2.53) but 'moderate-poor' with any intraventricular (OR = 0.56, 95 %CI = 0.39-0.81, p = 0.002) or any extra-axial (OR = 0.41, 95 %CI = 0.27-0.62, p<0.001) extension. CONCLUSIONS Delineation of ICH extent on stroke CT scans by AI software was excellent or good in 71 % of cases but was more likely to over- or under-estimate extent when ICH was either more extensive, intraventricular, or extra-axial.
Collapse
Affiliation(s)
- Adam Vacek
- Centre for Clinical Brain Sciences & UK Dementia Research Institute Centre, University of Edinburgh, UK
| | - Grant Mair
- Centre for Clinical Brain Sciences & UK Dementia Research Institute Centre, University of Edinburgh, UK.
| | - Philip White
- Translational and Clinical Research Institute, Newcastle University, UK
| | - Philip M Bath
- Stroke Trials Unit, Mental Health & Clinical Neuroscience, University of Nottingham, UK
| | - Keith W Muir
- School of Psychology & Neuroscience, University of Glasgow, UK
| | - Rustam Al-Shahi Salman
- Centre for Clinical Brain Sciences & UK Dementia Research Institute Centre, University of Edinburgh, UK
| | - Chloe Martin
- Centre for Clinical Brain Sciences & UK Dementia Research Institute Centre, University of Edinburgh, UK
| | - David Dye
- Centre for Clinical Brain Sciences & UK Dementia Research Institute Centre, University of Edinburgh, UK
| | - Francesca M Chappell
- Centre for Clinical Brain Sciences & UK Dementia Research Institute Centre, University of Edinburgh, UK
| | - Rüdiger von Kummer
- Department of Neuroradiology, University Hospital, Technische Universität Dresden, Germany
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences & UK Dementia Research Institute Centre, University of Edinburgh, UK
| | - Nikola Sprigg
- Stroke Trials Unit, Mental Health & Clinical Neuroscience, University of Nottingham, UK
| | - Joanna M Wardlaw
- Centre for Clinical Brain Sciences & UK Dementia Research Institute Centre, University of Edinburgh, UK
| |
Collapse
|
2
|
Hair K, Bahor Z, Macleod M, Liao J, Sena ES. The Automated Systematic Search Deduplicator (ASySD): a rapid, open-source, interoperable tool to remove duplicate citations in biomedical systematic reviews. BMC Biol 2023; 21:189. [PMID: 37674179 PMCID: PMC10483700 DOI: 10.1186/s12915-023-01686-z] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Accepted: 08/21/2023] [Indexed: 09/08/2023] Open
Abstract
BACKGROUND Researchers performing high-quality systematic reviews search across multiple databases to identify relevant evidence. However, the same publication is often retrieved from several databases. Identifying and removing such duplicates ("deduplication") can be extremely time-consuming, but failure to remove these citations can lead to the wrongful inclusion of duplicate data. Many existing tools are not sensitive enough, lack interoperability with other tools, are not freely accessible, or are difficult to use without programming knowledge. Here, we report the performance of our Automated Systematic Search Deduplicator (ASySD), a novel tool to perform automated deduplication of systematic searches for biomedical reviews. METHODS We evaluated ASySD's performance on 5 unseen biomedical systematic search datasets of various sizes (1845-79,880 citations). We compared the performance of ASySD with EndNote's automated deduplication option and with the Systematic Review Assistant Deduplication Module (SRA-DM). RESULTS ASySD identified more duplicates than either SRA-DM or EndNote, with a sensitivity in different datasets of 0.95 to 0.99. The false-positive rate was comparable to human performance, with a specificity of > 0.99. The tool took less than 1 h to identify and remove duplicates within each dataset. CONCLUSIONS For duplicate removal in biomedical systematic reviews, ASySD is a highly sensitive, reliable, and time-saving tool. It is open source and freely available online as both an R package and a user-friendly web application.
Collapse
Affiliation(s)
- Kaitlyn Hair
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK.
| | - Zsanett Bahor
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Jing Liao
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Emily S Sena
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| |
Collapse
|
3
|
Butler J, Jacobs N, Macleod M, Munafò M. The UK Reproducibility Network: A progress report. J Neurosci Methods 2023; 397:109949. [PMID: 37586662 DOI: 10.1016/j.jneumeth.2023.109949] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2023] [Revised: 08/07/2023] [Accepted: 08/11/2023] [Indexed: 08/18/2023]
Abstract
There is growing awareness that the ways in which academic research is conducted could be improved. A number of exciting innovations are emerging, alongside a broader agenda that includes a growing emphasis on open research. This short article outlines the rationale, progress and plans of the UK Reproducibility Network, which is one of a growing number of similar initiatives internationally that promote more rigorous and transparent research.
Collapse
Affiliation(s)
- Jess Butler
- Institute of Applied Health Sciences, University of Aberdeen, King's College, Aberdeen AB24 3FX, United Kingdom.
| | - Neil Jacobs
- School of Psychological Science, University of Bristol, 12a, Priory Road, Bristol BS8 1TU, United Kingdom.
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, United Kingdom.
| | - Marcus Munafò
- School of Psychological Science, University of Bristol, 12a, Priory Road, Bristol BS8 1TU, United Kingdom.
| |
Collapse
|
4
|
Mair G, White P, Bath PM, Muir K, Martin C, Dye D, Chappell F, von Kummer R, Macleod M, Sprigg N, Wardlaw JM. Accuracy of artificial intelligence software for CT angiography in stroke. Ann Clin Transl Neurol 2023; 10:1072-1082. [PMID: 37208850 PMCID: PMC10351662 DOI: 10.1002/acn3.51790] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Accepted: 05/01/2023] [Indexed: 05/21/2023] Open
Abstract
OBJECTIVE Software developed using artificial intelligence may automatically identify arterial occlusion and provide collateral vessel scoring on CT angiography (CTA) performed acutely for ischemic stroke. We aimed to assess the diagnostic accuracy of e-CTA by Brainomix™ Ltd by large-scale independent testing using expert reading as the reference standard. METHODS We identified a large clinically representative sample of baseline CTA from 6 studies that recruited patients with acute stroke symptoms involving any arterial territory. We compared e-CTA results with masked expert interpretation of the same scans for the presence and location of laterality-matched arterial occlusion and/or abnormal collateral score combined into a single measure of arterial abnormality. We tested the diagnostic accuracy of e-CTA for identifying any arterial abnormality (and in a sensitivity analysis compliant with the manufacturer's guidance that software only be used to assess the anterior circulation). RESULTS We include CTA from 668 patients (50% female; median: age 71 years, NIHSS 9, 2.3 h from stroke onset). Experts identified arterial occlusion in 365 patients (55%); most (343, 94%) involved the anterior circulation. Software successfully processed 545/668 (82%) CTAs. The sensitivity, specificity and diagnostic accuracy of e-CTA for detecting arterial abnormality were each 72% (95% CI = 66-77%). Diagnostic accuracy was non-significantly improved in a sensitivity analysis excluding occlusions from outside the anterior circulation (76%, 95% CI = 72-80%). INTERPRETATION Compared to experts, the diagnostic accuracy of e-CTA for identifying acute arterial abnormality was 72-76%. Users of e-CTA should be competent in CTA interpretation to ensure all potential thrombectomy candidates are identified.
Collapse
Affiliation(s)
- Grant Mair
- Centre for Clinical Brain SciencesUniversity of EdinburghEdinburghUK
| | - Philip White
- Translational and Clinical Research Institute, Newcastle University, Newcastle Upon Tyne Hospitals NHS TrustNewcastle upon TyneUK
| | - Philip M. Bath
- Stroke Trials Unit, Mental Health & Clinical NeuroscienceUniversity of NottinghamNottinghamUK
| | - Keith Muir
- Institute of Neuroscience & Psychology, University of GlasgowGlasgowUK
| | - Chloe Martin
- Centre for Clinical Brain SciencesUniversity of EdinburghEdinburghUK
| | - David Dye
- Centre for Clinical Brain SciencesUniversity of EdinburghEdinburghUK
| | | | - Rüdiger von Kummer
- Department of NeuroradiologyUniversity Hospital, Technische Universität DresdenDresdenGermany
| | - Malcolm Macleod
- Centre for Clinical Brain SciencesUniversity of EdinburghEdinburghUK
| | - Nikola Sprigg
- Translational and Clinical Research Institute, Newcastle University, Newcastle Upon Tyne Hospitals NHS TrustNewcastle upon TyneUK
| | - Joanna M. Wardlaw
- Centre for Clinical Brain SciencesUniversity of EdinburghEdinburghUK
- UK Dementia Research Institute Centre at the University of EdinburghEdinburghUK
| | | |
Collapse
|
5
|
Cipriani A, Seedat S, Milligan L, Salanti G, Macleod M, Hastings J, Thomas J, Michie S, Furukawa TA, Gilbert D, Soares-Weiser K, Moreno C, Leucht S, Egger M, Mansoori P, Barker JM, Siafis S, Ostinelli EG, McCutcheon R, Wright S, Simpson M, Elugbadebo O, Chiocchia V, Tonia T, Elgarf R, Kurtulmus A, Sena E, Simple O, Boyce N, Chung S, Sharma A, Wolpert M, Potts J, Elliott JH. New living evidence resource of human and non-human studies for early intervention and research prioritisation in anxiety, depression and psychosis. BMJ Ment Health 2023; 26:e300759. [PMID: 37290906 PMCID: PMC10255027 DOI: 10.1136/bmjment-2023-300759] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/02/2023] [Accepted: 05/04/2023] [Indexed: 06/10/2023]
Abstract
In anxiety, depression and psychosis, there has been frustratingly slow progress in developing novel therapies that make a substantial difference in practice, as well as in predicting which treatments will work for whom and in what contexts. To intervene early in the process and deliver optimal care to patients, we need to understand the underlying mechanisms of mental health conditions, develop safe and effective interventions that target these mechanisms, and improve our capabilities in timely diagnosis and reliable prediction of symptom trajectories. Better synthesis of existing evidence is one way to reduce waste and improve efficiency in research towards these ends. Living systematic reviews produce rigorous, up-to-date and informative evidence summaries that are particularly important where research is emerging rapidly, current evidence is uncertain and new findings might change policy or practice. Global Alliance for Living Evidence on aNxiety, depressiOn and pSychosis (GALENOS) aims to tackle the challenges of mental health science research by cataloguing and evaluating the full spectrum of relevant scientific research including both human and preclinical studies. GALENOS will also allow the mental health community-including patients, carers, clinicians, researchers and funders-to better identify the research questions that most urgently need to be answered. By creating open-access datasets and outputs in a state-of-the-art online resource, GALENOS will help identify promising signals early in the research process. This will accelerate translation from discovery science into effective new interventions for anxiety, depression and psychosis, ready to be translated in clinical practice across the world.
Collapse
Affiliation(s)
- Andrea Cipriani
- Department of Psychiatry, University of Oxford, Oxford, UK
- Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, Oxford, UK
- Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
| | - Soraya Seedat
- South African Medical Research Council/Stellenbosch University Extramural Genomics of Brain Disorders Unit, Department of Psychiatry, Stellenbosch University, Cape Town, South Africa
| | | | - Georgia Salanti
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Janna Hastings
- Institute for Implementation Science in Health Care, Faculty of Medicine, University of Zurich, Zurich, Switzerland
- School of Medicine, University of St. Gallen, St. Gallen, Switzerland
| | - James Thomas
- EPPI Centre, Social Research Institute, University College London, London, UK
| | - Susan Michie
- UCL Centre for Behaviour Change, University College London, London, UK
| | - Toshi A Furukawa
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine / School of Public Health, Kyoto, Japan
| | - David Gilbert
- Chair, GALENOS Global Experiential Advisory Board, InHealth Associates, London, UK
| | | | - Carmen Moreno
- Department of Child and Adolescent Psychiatry, Institute of Psychiatry and Mental Health, Hospital General Universitario Gregorio Marañón, IiSGM, CIBERSAM, ISCIII, School of Medicine, Universidad Complutense, Madrid, Spain
| | - Stefan Leucht
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munich, Germany
| | - Matthias Egger
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Centre for Infectious Disease Epidemiology and Research, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | | | | | - Spyridon Siafis
- Department of Psychiatry and Psychotherapy, School of Medicine, Technical University of Munich, Munich, Germany
| | - Edoardo Giuseppe Ostinelli
- Department of Psychiatry, University of Oxford, Oxford, UK
- Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, Oxford, UK
| | - Robert McCutcheon
- Department of Psychiatry, University of Oxford, Oxford, UK
- Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
- Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King's College, London, UK
| | - Simonne Wright
- South African Medical Research Council/Stellenbosch University Extramural Genomics of Brain Disorders Unit, Department of Psychiatry, Stellenbosch University, Cape Town, South Africa
| | | | - Olufisayo Elugbadebo
- South African Medical Research Council/Stellenbosch University Extramural Genomics of Brain Disorders Unit, Department of Psychiatry, Stellenbosch University, Cape Town, South Africa
- Department of Psychiatry, University of Ibadan, Ibadan, Nigeria
| | - Virginia Chiocchia
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | - Thomy Tonia
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | - Rania Elgarf
- Department of Psychiatry, University of Oxford, Oxford, UK
- Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, Oxford, UK
| | - Ayse Kurtulmus
- Department of Psychiatry, Istanbul Medeniyet University, Turkey, Turkey
| | - Emily Sena
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Ouma Simple
- South African Medical Research Council/Stellenbosch University Extramural Genomics of Brain Disorders Unit, Department of Psychiatry, Stellenbosch University, Cape Town, South Africa
- College of Health Sciences, Makerere University, Kampala, Uganda
| | | | | | | | | | - Jennifer Potts
- Department of Psychiatry, University of Oxford, Oxford, UK
- Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, Oxford, UK
| | - Julian H Elliott
- Cochrane Australia, School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia
- Future Evidence Foundation, Melbourne, Victoria, Australia
| |
Collapse
|
6
|
Hair K, Wilson E, Wong C, Tsang A, Macleod M, Bannach-Brown A. Systematic online living evidence summaries: emerging tools to accelerate evidence synthesis. Clin Sci (Lond) 2023; 137:773-784. [PMID: 37219941 DOI: 10.1042/cs20220494] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Revised: 01/30/2023] [Accepted: 03/06/2023] [Indexed: 05/24/2023]
Abstract
Systematic reviews and meta-analysis are the cornerstones of evidence-based decision making and priority setting. However, traditional systematic reviews are time and labour intensive, limiting their feasibility to comprehensively evaluate the latest evidence in research-intensive areas. Recent developments in automation, machine learning and systematic review technologies have enabled efficiency gains. Building upon these advances, we developed Systematic Online Living Evidence Summaries (SOLES) to accelerate evidence synthesis. In this approach, we integrate automated processes to continuously gather, synthesise and summarise all existing evidence from a research domain, and report the resulting current curated content as interrogatable databases via interactive web applications. SOLES can benefit various stakeholders by (i) providing a systematic overview of current evidence to identify knowledge gaps, (ii) providing an accelerated starting point for a more detailed systematic review, and (iii) facilitating collaboration and coordination in evidence synthesis.
Collapse
Affiliation(s)
- Kaitlyn Hair
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
| | - Emma Wilson
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
| | - Charis Wong
- Anne Rowling Regenerative Neurology Clinic, University of Edinburgh, Edinburgh, U.K
- Euan Macdonald Centre for Motor Neuron Disease Research, University of Edinburgh, Edinburgh, U.K
| | - Anthony Tsang
- King's Technology Evaluation Centre, King's College London, U.K
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
| | - Alexandra Bannach-Brown
- Charité Universitaetsmedizin Berlin, Berlin Institute of Health - QUEST Center, Berlin, Germany
| |
Collapse
|
7
|
Wong C, Gregory JM, Liao J, Egan K, Vesterinen HM, Ahmad Khan A, Anwar M, Beagan C, Brown FS, Cafferkey J, Cardinali A, Chiam JY, Chiang C, Collins V, Dormido J, Elliott E, Foley P, Foo YC, Fulton-Humble L, Gane AB, Glasmacher SA, Heffernan Á, Jayaprakash K, Jayasuriya N, Kaddouri A, Kiernan J, Langlands G, Leighton D, Liu J, Lyon J, Mehta AR, Meng A, Nguyen V, Park NH, Quigley S, Rashid Y, Salzinger A, Shiell B, Singh A, Soane T, Thompson A, Tomala O, Waldron FM, Selvaraj BT, Chataway J, Swingler R, Connick P, Pal S, Chandran S, Macleod M. Systematic, comprehensive, evidence-based approach to identify neuroprotective interventions for motor neuron disease: using systematic reviews to inform expert consensus. BMJ Open 2023; 13:e064169. [PMID: 36725099 PMCID: PMC9896226 DOI: 10.1136/bmjopen-2022-064169] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/25/2022] [Accepted: 01/10/2023] [Indexed: 02/03/2023] Open
Abstract
OBJECTIVES Motor neuron disease (MND) is an incurable progressive neurodegenerative disease with limited treatment options. There is a pressing need for innovation in identifying therapies to take to clinical trial. Here, we detail a systematic and structured evidence-based approach to inform consensus decision making to select the first two drugs for evaluation in Motor Neuron Disease-Systematic Multi-arm Adaptive Randomised Trial (MND-SMART: NCT04302870), an adaptive platform trial. We aim to identify and prioritise candidate drugs which have the best available evidence for efficacy, acceptable safety profiles and are feasible for evaluation within the trial protocol. METHODS We conducted a two-stage systematic review to identify potential neuroprotective interventions. First, we reviewed clinical studies in MND, Alzheimer's disease, Huntington's disease, Parkinson's disease and multiple sclerosis, identifying drugs described in at least one MND publication or publications in two or more other diseases. We scored and ranked drugs using a metric evaluating safety, efficacy, study size and study quality. In stage two, we reviewed efficacy of drugs in MND animal models, multicellular eukaryotic models and human induced pluripotent stem cell (iPSC) studies. An expert panel reviewed candidate drugs over two shortlisting rounds and a final selection round, considering the systematic review findings, late breaking evidence, mechanistic plausibility, safety, tolerability and feasibility of evaluation in MND-SMART. RESULTS From the clinical review, we identified 595 interventions. 66 drugs met our drug/disease logic. Of these, 22 drugs with supportive clinical and preclinical evidence were shortlisted at round 1. Seven drugs proceeded to round 2. The panel reached a consensus to evaluate memantine and trazodone as the first two arms of MND-SMART. DISCUSSION For future drug selection, we will incorporate automation tools, text-mining and machine learning techniques to the systematic reviews and consider data generated from other domains, including high-throughput phenotypic screening of human iPSCs.
Collapse
Affiliation(s)
- Charis Wong
- Anne Rowling Regenerative Neurology Clinic, The University of Edinburgh, Edinburgh, UK
- Euan MacDonald Centre for Motor Neuron Disease Research, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
- Medical Research Council Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, University College London, London, UK
| | - Jenna M Gregory
- Euan MacDonald Centre for Motor Neuron Disease Research, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
- Institute of Medical Sciences, University of Aberdeen, Aberdeen, UK
| | - Jing Liao
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Kieren Egan
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
- Computer and Information Science, University of Strathclyde, Glasgow, UK
| | - Hanna M Vesterinen
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Aimal Ahmad Khan
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | - Maarij Anwar
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | - Caitlin Beagan
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | - Fraser S Brown
- Anne Rowling Regenerative Neurology Clinic, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - John Cafferkey
- Royal Infirmary of Edinburgh, NHS Lothian, Edinburgh, UK
| | - Alessandra Cardinali
- Euan MacDonald Centre for Motor Neuron Disease Research, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
- UK Dementia Research Institute, University of Edinburgh, Edinburgh, UK
| | - Jane Yi Chiam
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | - Claire Chiang
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | - Victoria Collins
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | | | - Elizabeth Elliott
- Anne Rowling Regenerative Neurology Clinic, The University of Edinburgh, Edinburgh, UK
- Euan MacDonald Centre for Motor Neuron Disease Research, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Peter Foley
- Anne Rowling Regenerative Neurology Clinic, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Yu Cheng Foo
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | | | - Angus B Gane
- College of Medicine and Veterinary Medicine, The University of Edinburgh, Edinburgh, UK
| | - Stella A Glasmacher
- Anne Rowling Regenerative Neurology Clinic, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Áine Heffernan
- UK Dementia Research Institute, University of Edinburgh, Edinburgh, UK
| | - Kiran Jayaprakash
- Anne Rowling Regenerative Neurology Clinic, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Nimesh Jayasuriya
- Royal Infirmary of Edinburgh, NHS Lothian, Edinburgh, UK
- College of Medicine and Veterinary Medicine, The University of Edinburgh, Edinburgh, UK
| | - Amina Kaddouri
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | - Jamie Kiernan
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | - Gavin Langlands
- Institute of Neurological Sciences, NHS Greater Glasgow and Clyde, Glasgow, UK
| | - D Leighton
- Euan MacDonald Centre for Motor Neuron Disease Research, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
- School of Psychology and Neuroscience, University of Glasgow, Glasgow, UK
| | - Jiaming Liu
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | - James Lyon
- Royal Infirmary of Edinburgh, NHS Lothian, Edinburgh, UK
| | - Arpan R Mehta
- Anne Rowling Regenerative Neurology Clinic, The University of Edinburgh, Edinburgh, UK
- Euan MacDonald Centre for Motor Neuron Disease Research, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
- UK Dementia Research Institute, University of Edinburgh, Edinburgh, UK
| | - Alyssa Meng
- Centre for Discovery Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Vivienne Nguyen
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | - Na Hyun Park
- Royal Infirmary of Edinburgh, NHS Lothian, Edinburgh, UK
| | - Suzanne Quigley
- Anne Rowling Regenerative Neurology Clinic, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Yousuf Rashid
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | - Andrea Salzinger
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
- UK Dementia Research Institute, University of Edinburgh, Edinburgh, UK
| | - Bethany Shiell
- College of Medicine and Veterinary Medicine, The University of Edinburgh, Edinburgh, UK
| | - Ankur Singh
- College of Medicine and Veterinary Medicine, The University of Edinburgh, Edinburgh, UK
| | - Tim Soane
- Neurology Department, NHS Forth Valley, Stirling, UK
| | - Alexandra Thompson
- College of Medicine and Veterinary Medicine, The University of Edinburgh, Edinburgh, UK
| | - Olaf Tomala
- Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | - Fergal M Waldron
- Institute of Medical Sciences, University of Aberdeen, Aberdeen, UK
- Institute of Evolutionary Biology, The University of Edinburgh, Edinburgh, UK
| | - Bhuvaneish T Selvaraj
- Anne Rowling Regenerative Neurology Clinic, The University of Edinburgh, Edinburgh, UK
- Euan MacDonald Centre for Motor Neuron Disease Research, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
- UK Dementia Research Institute, University of Edinburgh, Edinburgh, UK
| | - Jeremy Chataway
- Medical Research Council Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, University College London, London, UK
- Queen Square Multiple Sclerosis Centre, Department of Neuroinflammation, UCL Queen Square Institute of Neurology, London, UK
- University College London Hospitals, Biomedical Research Centre, National Institute for Health Research, London, UK
| | - Robert Swingler
- Euan MacDonald Centre for Motor Neuron Disease Research, The University of Edinburgh, Edinburgh, UK
| | - Peter Connick
- Anne Rowling Regenerative Neurology Clinic, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Suvankar Pal
- Anne Rowling Regenerative Neurology Clinic, The University of Edinburgh, Edinburgh, UK
- Euan MacDonald Centre for Motor Neuron Disease Research, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Siddharthan Chandran
- Anne Rowling Regenerative Neurology Clinic, The University of Edinburgh, Edinburgh, UK
- Euan MacDonald Centre for Motor Neuron Disease Research, The University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
- UK Dementia Research Institute, University of Edinburgh, Edinburgh, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, UK
| |
Collapse
|
8
|
Wilson E, Cruz F, Maclean D, Ghanawi J, McCann S, Brennan P, Liao J, Sena E, Macleod M. Screening for in vitro systematic reviews: a comparison of screening methods and training of a machine learning classifier. Clin Sci (Lond) 2023; 137:181-193. [PMID: 36630537 PMCID: PMC9885807 DOI: 10.1042/cs20220594] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 12/15/2022] [Accepted: 01/11/2023] [Indexed: 01/13/2023]
Abstract
OBJECTIVE Existing strategies to identify relevant studies for systematic review may not perform equally well across research domains. We compare four approaches based on either human or automated screening of either title and abstract or full text, and report the training of a machine learning algorithm to identify in vitro studies from bibliographic records. METHODS We used a systematic review of oxygen-glucose deprivation (OGD) in PC-12 cells to compare approaches. For human screening, two reviewers independently screened studies based on title and abstract or full text, with disagreements reconciled by a third. For automated screening, we applied text mining to either title and abstract or full text. We trained a machine learning algorithm with decisions from 2000 randomly selected PubMed Central records enriched with a dataset of known in vitro studies. RESULTS Full-text approaches performed best, with human (sensitivity: 0.990, specificity: 1.000 and precision: 0.994) outperforming text mining (sensitivity: 0.972, specificity: 0.980 and precision: 0.764). For title and abstract, text mining (sensitivity: 0.890, specificity: 0.995 and precision: 0.922) outperformed human screening (sensitivity: 0.862, specificity: 0.998 and precision: 0.975). At our target sensitivity of 95% the algorithm performed with specificity of 0.850 and precision of 0.700. CONCLUSION In this in vitro systematic review, human screening based on title and abstract erroneously excluded 14% of relevant studies, perhaps because title and abstract provide an incomplete description of methods used. Our algorithm might be used as a first selection phase in in vitro systematic reviews to limit the extent of full text screening required.
Collapse
Affiliation(s)
- Emma Wilson
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
- Correspondence: Emma Wilson ()
| | - Florenz Cruz
- Berlin Institute of Health at Charité-Universitätsmedizin Berlin, QUEST Center, Berlin, Germany
| | - Duncan Maclean
- University of Edinburgh Medical School, University of Edinburgh, Edinburgh, U.K
| | | | - Sarah K. McCann
- Berlin Institute of Health at Charité-Universitätsmedizin Berlin, QUEST Center, Berlin, Germany
| | - Paul M. Brennan
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
| | - Jing Liao
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
| | - Emily S. Sena
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, U.K
| |
Collapse
|
9
|
Mair G, White P, Bath PM, Muir KW, Al‐Shahi Salman R, Martin C, Dye D, Chappell FM, Vacek A, von Kummer R, Macleod M, Sprigg N, Wardlaw JM. External Validation of e-ASPECTS Software for Interpreting Brain CT in Stroke. Ann Neurol 2022; 92:943-957. [PMID: 36053916 PMCID: PMC9826303 DOI: 10.1002/ana.26495] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2022] [Revised: 07/08/2022] [Accepted: 08/29/2022] [Indexed: 01/11/2023]
Abstract
OBJECTIVE The purpose of this study was to test e-ASPECTS software in patients with stroke. Marketed as a decision-support tool, e-ASPECTS may detect features of ischemia or hemorrhage on computed tomography (CT) imaging and quantify ischemic extent using Alberta Stroke Program Early CT Score (ASPECTS). METHODS Using CT from 9 stroke studies, we compared software with masked experts. As per indications for software use, we assessed e-ASPECTS results for patients with/without middle cerebral artery (MCA) ischemia but no other cause of stroke. In an analysis outside the intended use of the software, we enriched our dataset with non-MCA ischemia, hemorrhage, and mimics to simulate a representative "front door" hospital population. With final diagnosis as the reference standard, we tested the diagnostic accuracy of e-ASPECTS for identifying stroke features (ischemia, hyperattenuated arteries, and hemorrhage) in the representative population. RESULTS We included 4,100 patients (51% women, median age = 78 years, National Institutes of Health Stroke Scale [NIHSS] = 10, onset to scan = 2.5 hours). Final diagnosis was ischemia (78%), hemorrhage (14%), or mimic (8%). From 3,035 CTs with expert-rated ASPECTS, most (2084/3035, 69%) e-ASPECTS results were within one point of experts. In the representative population, the diagnostic accuracy of e-ASPECTS was 71% (95% confidence interval [CI] = 70-72%) for detecting ischemic features, 85% (83-86%) for hemorrhage. Software identified more false positive ischemia (12% vs 2%) and hemorrhage (14% vs <1%) than experts. INTERPRETATION On independent testing, e-ASPECTS provided moderate agreement with experts and overcalled stroke features. Therefore, future prospective trials testing impacts of artificial intelligence (AI) software on patient care and outcome are required before widespread implementation of stroke decision-support software. ANN NEUROL 2022;92:943-957.
Collapse
Affiliation(s)
- Grant Mair
- Centre for Clinical Brain SciencesUniversity of EdinburghEdinburghUK
| | - Philip White
- Translational and Clinical Research InstituteNewcastle University and Newcastle upon Tyne Hospitals NHS TrustNewcastle upon TyneUK
| | - Philip M. Bath
- Stroke Trials Unit, Mental Health & Clinical NeuroscienceUniversity of NottinghamNottinghamUK
| | - Keith W. Muir
- School of Psychology & NeuroscienceUniversity of GlasgowGlasgowUK
| | | | - Chloe Martin
- Centre for Clinical Brain SciencesUniversity of EdinburghEdinburghUK
| | - David Dye
- Centre for Clinical Brain SciencesUniversity of EdinburghEdinburghUK
| | | | - Adam Vacek
- Centre for Clinical Brain SciencesUniversity of EdinburghEdinburghUK
| | - Rüdiger von Kummer
- Department of NeuroradiologyUniversity Hospital, Technische Universität DresdenDresdenGermany
| | - Malcolm Macleod
- Centre for Clinical Brain SciencesUniversity of EdinburghEdinburghUK
| | - Nikola Sprigg
- Translational and Clinical Research InstituteNewcastle University and Newcastle upon Tyne Hospitals NHS TrustNewcastle upon TyneUK
| | - Joanna M. Wardlaw
- Centre for Clinical Brain SciencesUniversity of EdinburghEdinburghUK
- UK Dementia Research Institute Centre at the University of EdinburghEdinburghUK
| |
Collapse
|
10
|
Wang Q, Liao J, Lapata M, Macleod M. PICO entity extraction for preclinical animal literature. Syst Rev 2022; 11:209. [PMID: 36180888 PMCID: PMC9524079 DOI: 10.1186/s13643-022-02074-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/22/2021] [Accepted: 09/12/2022] [Indexed: 12/09/2022] Open
Abstract
BACKGROUND Natural language processing could assist multiple tasks in systematic reviews to reduce workflow, including the extraction of PICO elements such as study populations, interventions, comparators and outcomes. The PICO framework provides a basis for the retrieval and selection for inclusion of evidence relevant to a specific systematic review question, and automatic approaches to PICO extraction have been developed particularly for reviews of clinical trial findings. Considering the difference between preclinical animal studies and clinical trials, developing separate approaches is necessary. Facilitating preclinical systematic reviews will inform the translation from preclinical to clinical research. METHODS We randomly selected 400 abstracts from the PubMed Central Open Access database which described in vivo animal research and manually annotated these with PICO phrases for Species, Strain, methods of Induction of disease model, Intervention, Comparator and Outcome. We developed a two-stage workflow for preclinical PICO extraction. Firstly we fine-tuned BERT with different pre-trained modules for PICO sentence classification. Then, after removing the text irrelevant to PICO features, we explored LSTM-, CRF- and BERT-based models for PICO entity recognition. We also explored a self-training approach because of the small training corpus. RESULTS For PICO sentence classification, BERT models using all pre-trained modules achieved an F1 score of over 80%, and models pre-trained on PubMed abstracts achieved the highest F1 of 85%. For PICO entity recognition, fine-tuning BERT pre-trained on PubMed abstracts achieved an overall F1 of 71% and satisfactory F1 for Species (98%), Strain (70%), Intervention (70%) and Outcome (67%). The score of Induction and Comparator is less satisfactory, but F1 of Comparator can be improved to 50% by applying self-training. CONCLUSIONS Our study indicates that of the approaches tested, BERT pre-trained on PubMed abstracts is the best for both PICO sentence classification and PICO entity recognition in the preclinical abstracts. Self-training yields better performance for identifying comparators and strains.
Collapse
Affiliation(s)
- Qianying Wang
- CCBS, Edinburgh Medical School, University of Edinburgh, Edinburgh, UK
| | - Jing Liao
- CCBS, Edinburgh Medical School, University of Edinburgh, Edinburgh, UK
| | - Mirella Lapata
- ILCC, School of Informatics, University of Edinburgh, Edinburgh, UK
| | - Malcolm Macleod
- CCBS, Edinburgh Medical School, University of Edinburgh, Edinburgh, UK.
| |
Collapse
|
11
|
Sander T, Ghanawi J, Wilson E, Muhammad S, Macleod M, Kahlert UD. Meta-analysis on reporting practices as a source of heterogeneity in in vitro cancer research. BMJ Open Science 2022; 6:e100272. [PMID: 35721833 PMCID: PMC9171230 DOI: 10.1136/bmjos-2021-100272] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2021] [Accepted: 05/09/2022] [Indexed: 11/13/2022] Open
Abstract
Objectives Heterogeneity of results of exact same research experiments oppose a significant socioeconomic burden. Insufficient methodological reporting is likely to be one of the contributors to results heterogeneity; however, little knowledge on reporting habits of in vitro cancer research and their effects on results reproducibility is available. Exemplified by a commonly performed in vitro assay, we aim to fill this knowledge gap and to derive recommendations necessary for reproducible, robust and translational preclinical science. Methods Here, we use systematic review to describe reporting practices in in vitro glioblastoma research using the Uppsala-87 Malignant Glioma (U-87 MG) cell line and perform multilevel random-effects meta-analysis followed by meta-regression to explore sources of heterogeneity within that literature, and any associations between reporting characteristics and reported findings. Literature that includes experiments measuring the effect of temozolomide on the viability of U-87 MG cells is searched on three databases (Embase, PubMed and Web of Science). Results In 137 identified articles, the methodological reporting is incomplete, for example, medium glucose level and cell density are reported in only 21.2% and 16.8% of the articles. After adjustments for different drug concentrations and treatment durations, the results heterogeneity across the studies (I2=68.5%) is concerningly large. Differences in culture medium glucose level are a driver of this heterogeneity. However, infrequent reporting of most experimental parameters limits the analysis of reproducibility moderating parameters. Conclusions Our results further support the ongoing efforts of establishing consensus reporting practices to elevate durability of results. By doing so, this work can raise awareness of how stricter reporting may help to improve the frequency of successful translation of preclinical results into human application. The authors received no specific funding for this work. A preregistered protocol is available at the Open Science Framework (https://osf.io/9k3dq).
Collapse
Affiliation(s)
- Timo Sander
- Department of Neurosurgery, Medical Faculty, Heinrich Heine University, Düsseldorf, Germany
| | | | - Emma Wilson
- Centre for Clinical Brain Sciences, The University of Edinburgh Medical School, Edinburgh, UK
| | - Sajjad Muhammad
- Department of Neurosurgery, Medical Faculty, Heinrich Heine University, Düsseldorf, Germany
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, The University of Edinburgh Medical School, Edinburgh, UK
| | - Ulf Dietrich Kahlert
- Department of Molecular and Experimental Surgery, Clinic for General, Visceral, Vascular and Transplant Surgery, Otto von Guericke Universität Magdeburg, Magdeburg, Germany
| |
Collapse
|
12
|
Wong C, Dakin R, Chataway J, Swingler R, Weir C, Stallard N, Parmar M, Macleod M, Pal S, Chandran S. 090 Clinical trials in amyotrophic lateral sclerosis: a systematic review. J Neurol Neurosurg Psychiatry 2022. [DOI: 10.1136/jnnp-2022-abn.127] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Abstract
BackgroundAmyotrophic lateral sclerosis (ALS) is a rapidly fatal neurodegenerative disease. Despite decades of clinical trials, there remains a pressing unmet need for effective treatments. We reviewed past and present ALS clinical trials to understand the methodological challenges in trial design and delivery.MethodsTrial registry databases including clinicaltrials. gov, International Clinical Trials Registry Platform, European Union Clinical Trials Register, and PubMed were systematically searched to identify Phase II, Phase II/III and Phase III Clinical Trials of Investigational Medicinal Products (CTIMPs) assessing potential disease modifying treatments in ALS. Trials registered, completed or published during 2008–2019 were included.Results125 CTIMPs, evaluating 76 drugs, involving 15647 people with ALS (pwALS) were reviewed. Ten drugs were tested in three or more trials. Trials employed predominantly traditional two-arm designs; only 12 used novel designs. Median number of participants was 86. 40% of trials had an attrition rate ≥ 20%. There was a wide variation of primary outcome measures and primary endpoints used.ConclusionHistorically, limited participation of pwALS in trials, resources and outcome measures hindered definitive and timely evaluation of drugs in two-arm trials. We propose that future trials will need to be more flexible, scalable and acceptable to all stakeholders.charis.wong@ed.ac.uk
Collapse
|
13
|
Macleod M. Improving the reproducibility and integrity of research: what can different stakeholders contribute? BMC Res Notes 2022; 15:146. [PMID: 35468858 PMCID: PMC9036698 DOI: 10.1186/s13104-022-06030-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2021] [Accepted: 04/08/2022] [Indexed: 11/10/2022] Open
Abstract
Increasing awareness of problems with the reproducibility and integrity of research led the UK Parliament Science and Technology Committee to launch, in July 2021, an inquiry into reproducibility and research integrity. We recognise at least four potential reasons why attempts to replicate a research finding may be unsuccessful: false positive statistical analyses, low generalisability of findings, suboptimal study designs (research integrity), and deliberate malfeasance (researcher integrity). It is important to make a distinction between the contributions of research integrity and of researcher integrity to the reproducibility crisis. While the impact of an individual instance of compromised researcher integrity is substantial, the aggregate impact of more prevalent problems with research integrity is likely much greater. The research community will be most efficient when failed replication efforts are never due to issues of research integrity or of researcher integrity, as this would allow focus on the scientific reasons for why two apparently similar experiments should reach different conclusions. We discuss the role of funders, institutions and government in addressing the “reproducibility crisis” before considering which interventions might have a positive impact on academia’s approach to reproducible research, and a possible role for a committee on research integrity.
Collapse
|
14
|
Russell AAM, Sutherland BA, Landowski LM, Macleod M, Howells DW. What has preclinical systematic review ever done for us? BMJ Open Science 2022; 6:e100219. [PMID: 35360370 PMCID: PMC8921935 DOI: 10.1136/bmjos-2021-100219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/09/2022] Open
Abstract
Systematic review and meta-analysis are a gift to the modern researcher, delivering a crystallised understanding of the existing research data in any given space. This can include whether candidate drugs are likely to work or not and which are better than others, whether our models of disease have predictive value and how this might be improved and also how these all interact with disease pathophysiology.Grappling with the literature needed for such analyses is becoming increasingly difficult as the number of publications grows. However, narrowing the focus of a review to reduce workload runs the risk of diminishing the generalisability of conclusions drawn from such increasingly specific analyses.Moreover, at the same time as we gain greater insight into our topic, we also discover more about the flaws that undermine much scientific research. Systematic review and meta-analysis have also shown that the quality of much preclinical research is inadequate. Systematic review has helped reveal the extent of selection bias, performance bias, detection bias, attrition bias and low statistical power, raising questions about the validity of many preclinical research studies. This is perhaps the greatest virtue of systematic review and meta-analysis, the knowledge generated ultimately helps shed light on the limitations of existing research practice, and in doing so, helps bring reform and rigour to research across the sciences.In this commentary, we explore the lessons that we have identified through the lens of preclinical systematic review and meta-analysis.
Collapse
Affiliation(s)
- Ash Allanna Mark Russell
- Tasmanian School of Medicine, College of Health and Medicine, University of Tasmania, Hobart, Tasmania, Australia
| | - Brad A Sutherland
- Tasmanian School of Medicine, College of Health and Medicine, University of Tasmania, Hobart, Tasmania, Australia
| | - Lila M Landowski
- Tasmanian School of Medicine, College of Health and Medicine, University of Tasmania, Hobart, Tasmania, Australia
- School of Health Sciences, College of Health and Medicine, University of Tasmania, Hobart, Tasmania, Australia
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, Edinburgh Medical School, The University of Edinburgh, Edinburgh, UK
| | - David W Howells
- Tasmanian School of Medicine, College of Health and Medicine, University of Tasmania, Hobart, Tasmania, Australia
| |
Collapse
|
15
|
Munafò MR, Chambers C, Collins A, Fortunato L, Macleod M. The reproducibility debate is an opportunity, not a crisis. BMC Res Notes 2022; 15:43. [PMID: 35144667 PMCID: PMC8832688 DOI: 10.1186/s13104-022-05942-3] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2021] [Accepted: 01/31/2022] [Indexed: 11/25/2022] Open
Abstract
There are many factors that contribute to the reproducibility and replicability of scientific research. There is a need to understand the research ecosystem, and improvements will require combined efforts across all parts of this ecosystem. National structures can play an important role in coordinating these efforts, working collaboratively with researchers, institutions, funders, publishers, learned societies and other sectoral organisations, and providing a monitoring and reporting function. Whilst many new ways of working and emerging innovations hold a great deal of promise, it will be important to invest in meta-research activity to ensure that these approaches are evidence based, work as intended, and do not have unintended consequences. Addressing reproducibility will require working collaboratively across the research ecosystem to share best practice and to make the most effective use of resources. The UK Reproducibility Network (UKRN) brings together Local Networks of researchers, Institutions, and External Stakeholders (funders, publishers, learned societies and other sectoral organisations), to coordinate action on reproducibility and work to ensure the UK retains its place as a centre for world-leading research. This activity is coordinated by the UKRN Steering Group. We consider this structure as valuable, bringing together a range of voices at a range of levels to support the combined efforts required to enact change.
Collapse
Affiliation(s)
- Marcus R Munafò
- School of Psychological Science, University of Bristol, 12a Priory Road, Bristol, BS8 1TU, UK. .,MRC Integrative Epidemiology Unit at the University of Bristol, Bristol, UK.
| | - Chris Chambers
- Cardiff University Brain Research Imaging Centre (CUBRIC), School of Psychology, Cardiff University, Cardiff, UK
| | - Alexandra Collins
- Centre for Environmental Policy, Imperial College London, London, UK
| | - Laura Fortunato
- Institute of Human Sciences, University of Oxford, Oxford, UK.,Santa Fe Institute, Santa Fe, NM, USA
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| |
Collapse
|
16
|
Stewart AJ, Farran EK, Grange JA, Macleod M, Munafò M, Newton P, Shanks DR. Improving research quality: the view from the UK Reproducibility Network institutional leads for research improvement. BMC Res Notes 2021; 14:458. [PMID: 34930427 PMCID: PMC8686561 DOI: 10.1186/s13104-021-05883-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2021] [Accepted: 12/09/2021] [Indexed: 11/10/2022] Open
Abstract
The adoption and incentivisation of open and transparent research practices is critical in addressing issues around research reproducibility and research integrity. These practices will require training and funding. Individuals need to be incentivised to adopt open and transparent research practices (e.g., added as desirable criteria in hiring, probation, and promotion decisions, recognition that funded research should be conducted openly and transparently, the importance of publishers mandating the publication of research workflows and appropriately curated data associated with each research output). Similarly, institutions need to be incentivised to encourage the adoption of open and transparent practices by researchers. Research quality should be prioritised over research quantity. As research transparency will look different for different disciplines, there can be no one-size-fits-all approach. An outward looking and joined up UK research strategy is needed that places openness and transparency at the heart of research activity. This should involve key stakeholders (institutions, research organisations, funders, publishers, and Government) and crucially should be focused on action. Failure to do this will have negative consequences not just for UK research, but also for our ability to innovate and subsequently commercialise UK-led discovery.
Collapse
|
17
|
Wong C, Stavrou M, Elliott E, Gregory JM, Leigh N, Pinto AA, Williams TL, Chataway J, Swingler R, Parmar MKB, Stallard N, Weir CJ, Parker RA, Chaouch A, Hamdalla H, Ealing J, Gorrie G, Morrison I, Duncan C, Connelly P, Carod-Artal FJ, Davenport R, Reitboeck PG, Radunovic A, Srinivasan V, Preston J, Mehta AR, Leighton D, Glasmacher S, Beswick E, Williamson J, Stenson A, Weaver C, Newton J, Lyle D, Dakin R, Macleod M, Pal S, Chandran S. Clinical trials in amyotrophic lateral sclerosis: a systematic review and perspective. Brain Commun 2021; 3:fcab242. [PMID: 34901853 PMCID: PMC8659356 DOI: 10.1093/braincomms/fcab242] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2021] [Revised: 09/07/2021] [Accepted: 09/13/2021] [Indexed: 12/15/2022] Open
Abstract
Amyotrophic lateral sclerosis is a progressive and devastating neurodegenerative disease. Despite decades of clinical trials, effective disease-modifying drugs remain scarce. To understand the challenges of trial design and delivery, we performed a systematic review of Phase II, Phase II/III and Phase III amyotrophic lateral sclerosis clinical drug trials on trial registries and PubMed between 2008 and 2019. We identified 125 trials, investigating 76 drugs and recruiting more than 15 000 people with amyotrophic lateral sclerosis. About 90% of trials used traditional fixed designs. The limitations in understanding of disease biology, outcome measures, resources and barriers to trial participation in a rapidly progressive, disabling and heterogenous disease hindered timely and definitive evaluation of drugs in two-arm trials. Innovative trial designs, especially adaptive platform trials may offer significant efficiency gains to this end. We propose a flexible and scalable multi-arm, multi-stage trial platform where opportunities to participate in a clinical trial can become the default for people with amyotrophic lateral sclerosis.
Collapse
Affiliation(s)
- Charis Wong
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Maria Stavrou
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK.,UK Dementia Research Institute, Chancellor's Building, The University of Edinburgh, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Elizabeth Elliott
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK.,UK Dementia Research Institute, Chancellor's Building, The University of Edinburgh, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Jenna M Gregory
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK.,UK Dementia Research Institute, Chancellor's Building, The University of Edinburgh, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Nigel Leigh
- Department of Neuroscience, Brighton and Sussex Medical School, University of Sussex, Brighton, BN1 9PX, UK
| | - Ashwin A Pinto
- Neurology Department, Wessex Neurosciences Centre, Southampton General Hospital, Southampton, SO16 6YD, UK
| | - Timothy L Williams
- Department of Neurology, Royal Victoria Infirmary, Newcastle upon Tyne NE1 4LP, UK
| | - Jeremy Chataway
- Queen Square Multiple Sclerosis Centre, Department of Neuroinflammation, UCL Queen Square Institute of Neurology, Faculty of Brain Sciences, University College London, London WC1B 5EH, UK.,National Institute for Health Research, University College London Hospitals, Biomedical Research Centre, London, W1T 7DN, UK.,MRC CTU at UCL, Institute of Clinical Trials and Methodology, University College London, London, WC1V 6LJ, UK
| | - Robert Swingler
- Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Mahesh K B Parmar
- MRC CTU at UCL, Institute of Clinical Trials and Methodology, University College London, London, WC1V 6LJ, UK
| | - Nigel Stallard
- Statistics and Epidemiology, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, CV4 7AL, UK
| | - Christopher J Weir
- Edinburgh Clinical Trials Unit, Usher Institute, Level 2, NINE Edinburgh BioQuarter, 9 Little France Road, Edinburgh EH16 4UX, UK
| | - Richard A Parker
- Edinburgh Clinical Trials Unit, Usher Institute, Level 2, NINE Edinburgh BioQuarter, 9 Little France Road, Edinburgh EH16 4UX, UK
| | - Amina Chaouch
- Motor Neurone Disease Care Centre, Manchester Centre for Clinical Neurosciences, Salford, M6 8HD, UK
| | - Hisham Hamdalla
- Motor Neurone Disease Care Centre, Manchester Centre for Clinical Neurosciences, Salford, M6 8HD, UK
| | - John Ealing
- Motor Neurone Disease Care Centre, Manchester Centre for Clinical Neurosciences, Salford, M6 8HD, UK
| | - George Gorrie
- Department of Neurology, Institute of Neurological Sciences, Queen Elizabeth University Hospital, NHS Greater Glasgow and Clyde, Glasgow, G51 4TF, UK
| | - Ian Morrison
- Department of Neurology, NHS Tayside, Dundee, DD2 1UB, UK
| | - Callum Duncan
- Department of Neurology, Aberdeen Royal Infirmary, Aberdeen, AB25 2ZN, UK
| | - Peter Connelly
- NHS Research Scotland Neuroprogressive Disorders and Dementia Network, Ninewells Hospital, Dundee, DD1 9SY, UK
| | | | - Richard Davenport
- Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Department of Clinical Neurosciences, NHS Lothian, Edinburgh, EH16 4SA, UK
| | - Pablo Garcia Reitboeck
- Atkinson Morley Regional Neurosciences Centre, St. George's University Hospitals NHS Foundation Trust, London SW17 0QT, UK
| | | | | | - Jenny Preston
- Department of Neurology, NHS Ayrshire & Arran, KA12 8SS, UK
| | - Arpan R Mehta
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK.,UK Dementia Research Institute, Chancellor's Building, The University of Edinburgh, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Danielle Leighton
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Stella Glasmacher
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Emily Beswick
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Jill Williamson
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Amy Stenson
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Christine Weaver
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Judith Newton
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Dawn Lyle
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Rachel Dakin
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK
| | - Suvankar Pal
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| | - Siddharthan Chandran
- Centre for Clinical Brain Sciences, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Anne Rowling Regenerative Neurology Clinic, Chancellor's Building, 49 Little France Crescent, The University of Edinburgh, Edinburgh, EH16 4SB, UK.,Euan MacDonald Centre for MND Research, University of Edinburgh, FU303F, Chancellor's Building, 49 Little France Crescent, Edinburgh EH16 4SB, UK.,UK Dementia Research Institute, Chancellor's Building, The University of Edinburgh, 49 Little France Crescent, Edinburgh EH16 4SB, UK
| |
Collapse
|
18
|
Wang Q, Liao J, Lapata M, Macleod M. Risk of bias assessment in preclinical literature using natural language processing. Res Synth Methods 2021; 13:368-380. [PMID: 34709718 PMCID: PMC9298308 DOI: 10.1002/jrsm.1533] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2021] [Revised: 10/01/2021] [Accepted: 10/09/2021] [Indexed: 12/09/2022]
Abstract
We sought to apply natural language processing to the task of automatic risk of bias assessment in preclinical literature, which could speed the process of systematic review, provide information to guide research improvement activity, and support translation from preclinical to clinical research. We use 7840 full‐text publications describing animal experiments with yes/no annotations for five risk of bias items. We implement a series of models including baselines (support vector machine, logistic regression, random forest), neural models (convolutional neural network, recurrent neural network with attention, hierarchical neural network) and models using BERT with two strategies (document chunk pooling and sentence extraction). We tune hyperparameters to obtain the highest F1 scores for each risk of bias item on the validation set and compare evaluation results on the test set to our previous regular expression approach. The F1 scores of best models on test set are 82.0% for random allocation, 81.6% for blinded assessment of outcome, 82.6% for conflict of interests, 91.4% for compliance with animal welfare regulations and 46.6% for reporting animals excluded from analysis. Our models significantly outperform regular expressions for four risk of bias items. For random allocation, blinded assessment of outcome, conflict of interests and animal exclusions, neural models achieve good performance; for animal welfare regulations, BERT model with a sentence extraction strategy works better. Convolutional neural networks are the overall best models. The tool is publicly available which may contribute to the future monitoring of risk of bias reporting for research improvement activities.
Collapse
Affiliation(s)
- Qianying Wang
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Jing Liao
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Mirella Lapata
- School of Informatics, University of Edinburgh, Edinburgh, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| |
Collapse
|
19
|
Al-Shahi Salman R, Keerie C, Stephen J, Lewis S, Norrie J, Dennis MS, Newby DE, Wardlaw JM, Lip GY, Parry-Jones A, White PM, Baigent C, Lasserson D, Oliver C, O'Mahony F, Amoils S, Bamford J, Armitage J, Emberson J, Rinkel GJ, Lowe G, Innes K, Adamczuk K, Dinsmore L, Drever J, Milne G, Walker A, Hutchison A, Williams C, Fraser R, Anderson R, Covil K, Stewart K, Rees J, Hall P, Bullen A, Stoddart A, Moullaali TJ, Palmer J, Sakka E, Perthen J, Lyttle N, Samarasekera N, MacRaild A, Burgess S, Teasdale J, Coakley M, Taylor P, Blair G, Whiteley W, Shenkin S, Clancy U, Macleod M, Sutherland R, Moullaali T, Barugh A, Lerpiniere C, Moreton F, Fethers N, Anjum T, Krishnan M, Slade P, Storton S, Williams M, Davies C, Connor L, Gainard G, Murphy C, Barber M, Esson D, Choulerton J, Shaw L, Lucas S, Hierons S, Avis J, Stone A, Gbadamoshi L, Costa T, Pearce L, Harkness K, Richards E, Howe J, Kamara C, Lindert R, Ali A, Rehan J, Chapman S, Edwards M, Bathula R, Cohen D, Devine J, Mpelembue M, Yesupatham P, Chhabra S, Adewetan G, Ballantine R, Brooks D, Smith G, Rogers G, Marsden S, Clark S, Wilkinson A, Brown E, Stephenson L, Nyo K, Abraham A, Pai Y, Shim G, Baliga V, Nair A, Robinson M, Hawksworth C, Greig J, Alam I, Nortcliffe T, Ramiz R, Shaw R, Parry-Jones A, Lee S, Marsden T, Perez J, Birleson E, Yadava R, Sangombe M, Stafford S, Hughes T, Knibbs L, Morse B, Schwarz S, Jelley B, White S, Richard B, Lawson H, Moseley S, Tayler M, Edwards M, Triscott C, Wallace R, Hall A, Dell A, Rashed K, Board S, Buckley C, Tanate A, Pitt-Kerby T, Beesley K, Perry J, Hellyer C, Guyler P, Menon N, Tysoe S, Prabakaran R, Cooper M, Rajapakse A, Wynter I, Smith S, Weir N, Boxall C, Yates H, Smith S, Crawford P, Marigold J, Smith F, Harvey J, Evans S, Baldwin L, Hammond S, Mudd P, Bowring A, Keenan S, Thorpe K, Haque M, Taaffe J, Temple N, Peachey T, Wells K, Haines F, Butterworth-Cowin N, Horne Z, Licenik R, Boughton H, England T, Hedstrom A, Menezes B, Davies R, Johnson V, Whittingham-Jones S, Werring D, Obarey S, Watchurst C, Ashton A, Feerick S, Francia N, Banaras A, Epstein D, Marinescu M, Williams A, Robinson A, Humphries F, Anwar I, Annamalai A, Crawford S, Collins V, Shepherd L, Siddle E, Penge J, Epstein D, Qureshi S, Krishnamurthy V, Papavasileiou V, Waugh D, Veraque E, Douglas N, Khan N, Ramachandran S, Sommerville P, Rudd A, Kullane S, Bhalla A, Birns J, Ahmed R, Gibbons M, Klamerus E, Cendreda B, Muir K, Day N, Welch A, Smith W, Elliot J, Eltawil S, Mahmood A, Hatherley K, Mitchell S, Bains H, Quinn L, Teal R, Gbinigie I, Harston G, Mathieson P, Ford G, Schulz U, Kennedy J, Nagaratnam K, Bangalore K, Bhupathiraju N, Wharton C, Fotherby K, Nasar A, Stevens A, Willberry A, Evans R, Rai B, Blake C, Thavanesan K, Hann G, Changuion T, Nix S, Whiting A, Dharmasiri M, Mallon L, Keltos M, Smyth N, Eglinton C, Duffy J, Tone E, Sykes L, Porter E, Fitton C, Kirkineziadis N, Cluckie G, Kennedy K, Trippier S, Williams R, Hayter E, Rackie J, Patel B, Rita G, Blight A, Jones V, Zhang L, Choy L, Pereira A, Clarke B, Al-Hussayni S, Dixon L, Young A, Bergin A, Broughton D, Raghunathan S, Jackson B, Appleton J, Wilkes G, Buck A, Richardson C, Clarke J, Fleming L, Squires G, Law Z, Hutchinson C, Cvoro V, Couser M, McGregor A, McAuley S, Pound S, Cochrane P, Holmes C, Murphy P, Devitt N, Osborn M, Steele A, Guthrie LB, Smith E, Hewitt J, Chaston N, Myint M, Smith A, Fairlie L, Davis M, Atkinson B, Woodward S, Hogg V, Fawcett M, Finlay L, Dixit A, Cameron E, Keegan B, Kelly J, Concannon D, Dutta D, Ward D, Glass J, O'Connell S, Ngeh J, O'Kelly A, Williams E, Ragab S, Jenkinson D, Dube J, Gleave L, Leggett J, Kissoon N, Southern L, Naghotra U, Bokhari M, McClelland B, Adie K, Mate A, Harrington F, James A, Swanson E, Chant T, Naccache M, Coutts A, Courtauld G, Whurr S, Webber S, Shead E, Luder R, Bhargava M, Murali E, Cuenoud L, Pasco K, Speirs O, Chapman L, Inskip L, Kavanagh L, Srinivasan M, Motherwell N, Mukherjee I, Tonks L, Donaldson D, Button H, Wilcox R, Hurford F, Logan R, Taylor A, Arden T, Carpenter M, Datta P, Zahoor T, Jackson L, Needle A, Stanners A, Ghouri I, Exley D, Akhtar S, Brooke H, Beadle S, O'Brien E, Francis J, McGee J, Amis E, Mitchell J, Finlay S, Sinha D, Manoczki C, King S, Tarka J, Choudhary S, Premaruban J, Sutton D, Kumar P, Culmsee C, Winckley C, Davies H, Thatcher H, Vasileiadis E, Aweid B, Holden M, Mason C, Hlaing T, Madzamba G, Ingram T, Linforth M, Cullen C, Thomas N, France J, Saulat A, Bhaskaran B, Fitzell P, Horan K, Manyoni C, Garfield-Smith J, Griffin H, Atkins S, Redome J, Muddegowda G, Maguire H, Barry A, Abano N, Varquez R, Hiden J, Lyjko S, Remegoso A, Finney K, Butler A, Strecker M, MaCleod MJ, Irvine J, Nelson S, Guzmangutierrez G, Furnace J, Taylor V, Ramadan H, Storton K, Hassan S, Abdus Sami E, Bellfield R, Stewart K, Quinn O, Patterson C, Emsley H, Gregary B, Ahmed S, Patel S, Raj S, Sultan S, Wright F, Langhorne P, Graham R, Quinn T, McArthur K. Effects of oral anticoagulation for atrial fibrillation after spontaneous intracranial haemorrhage in the UK: a randomised, open-label, assessor-masked, pilot-phase, non-inferiority trial. Lancet Neurol 2021; 20:842-853. [PMID: 34487722 DOI: 10.1016/s1474-4422(21)00264-7] [Citation(s) in RCA: 34] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2021] [Revised: 08/03/2021] [Accepted: 08/04/2021] [Indexed: 01/19/2023]
Abstract
BACKGROUND Oral anticoagulation reduces the rate of systemic embolism for patients with atrial fibrillation by two-thirds, but its benefits for patients with previous intracranial haemorrhage are uncertain. In the Start or STop Anticoagulants Randomised Trial (SoSTART), we aimed to establish whether starting is non-inferior to avoiding oral anticoagulation for survivors of intracranial haemorrhage who have atrial fibrillation. METHODS SoSTART was a prospective, randomised, open-label, assessor-masked, parallel-group, pilot phase trial done at 67 hospitals in the UK. We recruited adults (aged ≥18 years) who had survived at least 24 h after symptomatic spontaneous intracranial haemorrhage, had atrial fibrillation, and had a CHA2DS2-VASc score of at least 2. Web-based computerised randomisation incorporating a minimisation algorithm allocated participants (1:1) to start or avoid long-term (≥1 year) full treatment dose open-label oral anticoagulation. The participants assigned to start oral anticoagulation received either a direct oral anticoagulant or vitamin K antagonist, and the group assigned to avoid oral anticoagulation received standard clinical practice (antiplatelet agent or no antithrombotic agent). The primary outcome was recurrent symptomatic spontaneous intracranial haemorrhage, and was adjudicated by an individual masked to treatment allocation. All outcomes were ascertained for at least 1 year after randomisation and assessed in the intention-to-treat population of all randomly assigned participants, using Cox proportional hazards regression adjusted for minimisation covariates. We planned a sample size of 190 participants (one-sided p=0·025, power 90%, allowing for non-adherence) based on a non-inferiority margin of 12% (or adjusted hazard ratio [HR] of 3·2). This trial is registered with ClinicalTrials.gov (NCT03153150) and is complete. FINDINGS Between March 29, 2018, and Feb 27, 2020, consent was obtained at 61 sites for 218 participants, of whom 203 were randomly assigned at a median of 115 days (IQR 49-265) after intracranial haemorrhage onset. 101 were assigned to start and 102 to avoid oral anticoagulation. Participants were followed up for median of 1·2 years (IQR 0·97-1·95; completeness 97·2%). Starting oral anticoagulation was not non-inferior to avoiding oral anticoagulation: eight (8%) of 101 in the start group versus four (4%) of 102 in the avoid group had intracranial haemorrhage recurrences (adjusted HR 2·42 [95% CI 0·72-8·09]; p=0·152). Serious adverse events occurred in 17 (17%) participants in the start group and 15 (15%) in the avoid group. 22 (22%) patients in the start group and 11 (11%) patients in the avoid group died during the study. INTERPRETATION Whether starting oral anticoagulation was non-inferior to avoiding it for people with atrial fibrillation after intracranial haemorrhage was inconclusive, although rates of recurrent intracranial haemorrhage were lower than expected. In view of weak evidence from analyses of three composite secondary outcomes, the possibility that oral anticoagulation might be superior for preventing symptomatic major vascular events should be investigated in adequately powered randomised trials. FUNDING British Heart Foundation, Medical Research Council, Chest Heart & Stroke Scotland.
Collapse
|
20
|
Affiliation(s)
- Alexandra Bannach-Brown
- Berlin Institute of Health, QUEST Center, Charité Universitätsmedizin Berlin, Berlin, Germany
- Institute for Evidence-Based Practice, Bond University, Robina, Queensland, Australia
| | - Kaitlyn Hair
- Centre for Clinical Brain Sciences, The University of Edinburgh Edinburgh Medical School, Edinburgh, Scotland, UK
| | - Zsanett Bahor
- Centre for Clinical Brain Sciences, The University of Edinburgh Edinburgh Medical School, Edinburgh, Scotland, UK
| | - Nadia Soliman
- Pain Research; Faculty of Medicine, Department of Surgery and Cancer, Imperial College London, London, Greater London, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, The University of Edinburgh Edinburgh Medical School, Edinburgh, Scotland, UK
| | - Jing Liao
- Centre for Clinical Brain Sciences, The University of Edinburgh Edinburgh Medical School, Edinburgh, Scotland, UK
| |
Collapse
|
21
|
Soliman N, Haroutounian S, Hohmann AG, Krane E, Liao J, Macleod M, Segelcke D, Sena C, Thomas J, Vollert J, Wever K, Alaverdyan H, Barakat A, Barthlow T, Bozer ALH, Davidson A, Diaz-delCastillo M, Dolgorukova A, Ferdousi MI, Healy C, Hong S, Hopkins M, James A, Leake HB, Malewicz NM, Mansfield M, Mardon AK, Mattimoe D, McLoone DP, Noes-Holt G, Pogatzki-Zahn EM, Power E, Pradier B, Romanos-Sirakis E, Segelcke A, Vinagre R, Yanes JA, Zhang J, Zhang XY, Finn DP, Rice AS. Systematic review and meta-analysis of cannabinoids, cannabis-based medicines, and endocannabinoid system modulators tested for antinociceptive effects in animal models of injury-related or pathological persistent pain. Pain 2021; 162:S26-S44. [PMID: 33729209 PMCID: PMC8216112 DOI: 10.1097/j.pain.0000000000002269] [Citation(s) in RCA: 37] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Revised: 03/12/2021] [Accepted: 03/12/2021] [Indexed: 12/11/2022]
Abstract
ABSTRACT We report a systematic review and meta-analysis of studies that assessed the antinociceptive efficacy of cannabinoids, cannabis-based medicines, and endocannabinoid system modulators on pain-associated behavioural outcomes in animal models of pathological or injury-related persistent pain. In April 2019, we systematically searched 3 online databases and used crowd science and machine learning to identify studies for inclusion. We calculated a standardised mean difference effect size for each comparison and performed a random-effects meta-analysis. We assessed the impact of study design characteristics and reporting of mitigations to reduce the risk of bias. We meta-analysed 374 studies in which 171 interventions were assessed for antinociceptive efficacy in rodent models of pathological or injury-related pain. Most experiments were conducted in male animals (86%). Antinociceptive efficacy was most frequently measured by attenuation of hypersensitivity to evoked limb withdrawal. Selective cannabinoid type 1, cannabinoid type 2, nonselective cannabinoid receptor agonists (including delta-9-tetrahydrocannabinol) and peroxisome proliferator-activated receptor-alpha agonists (predominantly palmitoylethanolamide) significantly attenuated pain-associated behaviours in a broad range of inflammatory and neuropathic pain models. Fatty acid amide hydrolase inhibitors, monoacylglycerol lipase inhibitors, and cannabidiol significantly attenuated pain-associated behaviours in neuropathic pain models but yielded mixed results in inflammatory pain models. The reporting of criteria to reduce the risk of bias was low; therefore, the studies have an unclear risk of bias. The value of future studies could be enhanced by improving the reporting of methodological criteria, the clinical relevance of the models, and behavioural assessments. Notwithstanding, the evidence supports the hypothesis of cannabinoid-induced analgesia.
Collapse
Affiliation(s)
- Nadia Soliman
- Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | - Simon Haroutounian
- Department of Anesthesiology, Washington University Pain Center, Washington University School of Medicine, St. Louis, Missouri, United States
| | - Andrea G. Hohmann
- Department of Psychological and Brain Sciences, Program in Neuroscience and Gill Center for Biomolecular Science, Bloomington, IN, United States
| | - Elliot Krane
- Departments of Anesthesiology, Perioperative, and Pain Medicine, & Pediatrics, Stanford University School of Medicine, Stanford, CA, United States
| | - Jing Liao
- CAMARADES, Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Malcolm Macleod
- CAMARADES, Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Daniel Segelcke
- Department of Anesthesiology, Intensive Care and Pain Medicine University Hospital Muenster, Muenster, Germany
| | - Christopher Sena
- CAMARADES, Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - James Thomas
- EPPI-Centre, University College London, London, United Kingdom
| | - Jan Vollert
- Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | - Kimberley Wever
- SYRCLE at Central Animal Laboratory, Radbound University Medical Center, Nijmegen, the Netherlands
| | - Harutyun Alaverdyan
- Department of Anesthesiology, Washington University Pain Center, Washington University School of Medicine, St. Louis, Missouri, United States
| | - Ahmed Barakat
- Department of Drug Design and Pharmacology, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
- Department of Pharmacology and Toxicology, Faculty of Pharmacy, Assiut University, Asyut, Egypt
| | - Tyler Barthlow
- Department of Drug Design and Pharmacology, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Amber L. Harris Bozer
- Department of Psychological Sciences, Tarleton State University, Stephenville, TX, United States
| | | | - Marta Diaz-delCastillo
- Department of Drug Design and Pharmacology, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Antonina Dolgorukova
- Valdman Institute of Pharmacology, Pavlov First Saint Petersburg State Medical University, Saint Petersburg, Russia
| | - Mehnaz I. Ferdousi
- Pharmacology and Therapeutics, School of Medicine, Galway Neuroscience Centre and Centre for Pain Research, Human Biology Building, National University of Ireland Galway, Galway, Ireland
| | - Catherine Healy
- Pharmacology and Therapeutics, School of Medicine, Galway Neuroscience Centre and Centre for Pain Research, Human Biology Building, National University of Ireland Galway, Galway, Ireland
| | - Simon Hong
- Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, United States
| | - Mary Hopkins
- Pharmacology and Therapeutics, School of Medicine, Galway Neuroscience Centre and Centre for Pain Research, Human Biology Building, National University of Ireland Galway, Galway, Ireland
| | - Arul James
- Leicester General Hospital, University Hospitals of Leicester NHS Trust, Leicester, United Kingdom
| | - Hayley B. Leake
- IIMPACT in Health, University of South Australia, Adelaide, South Australia, Australia
- Centre for Pain IMPACT, Neuroscience Research Australia, Sydney, New South Wales, Australia
| | - Nathalie M. Malewicz
- Department of Anaesthesiology, Intensive Care Medicine and Pain Management, Medical Faculty of Ruhr-University Bochum, BG University Hospital Bergmannsheil gGmbH, Bochum, Germany
| | - Michael Mansfield
- Department of Allied Health Sciences, Institute of Health and Social Care, Pain Research Cluster, Ageing, Acute and Long Term Conditions Research Group, London South Bank University, London, United Kingdom
| | - Amelia K. Mardon
- IIMPACT in Health, University of South Australia, Adelaide, South Australia, Australia
| | - Darragh Mattimoe
- Pharmacology and Therapeutics, School of Medicine, Galway Neuroscience Centre and Centre for Pain Research, Human Biology Building, National University of Ireland Galway, Galway, Ireland
| | - Daniel P. McLoone
- Pharmacology and Therapeutics, School of Medicine, Galway Neuroscience Centre and Centre for Pain Research, Human Biology Building, National University of Ireland Galway, Galway, Ireland
| | - Gith Noes-Holt
- Molecular Neuropharmacology and Genetics Laboratory, Department of Neuroscience, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Esther M. Pogatzki-Zahn
- Department of Anesthesiology, Intensive Care and Pain Medicine University Hospital Muenster, Muenster, Germany
| | - Emer Power
- Pharmacology and Therapeutics, School of Medicine, Galway Neuroscience Centre and Centre for Pain Research, Human Biology Building, National University of Ireland Galway, Galway, Ireland
| | - Bruno Pradier
- Department of Anesthesiology, Intensive Care and Pain Medicine University Hospital Muenster, Muenster, Germany
| | - Eleny Romanos-Sirakis
- Staten Island University Hospital Northwell Health, Staten Island, NY, United States
- Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, United States
| | | | - Rafael Vinagre
- Visiting Scholar, Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, CA, United States
| | - Julio A. Yanes
- Department of Psychological Sciences, Auburn University, Auburn, AL, United States
| | - Jingwen Zhang
- King's College London GKT School of Medical Education, King's College London, London, United Kingdom
| | - Xue Ying Zhang
- Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | - David P. Finn
- Pharmacology and Therapeutics, School of Medicine, Galway Neuroscience Centre and Centre for Pain Research, Human Biology Building, National University of Ireland Galway, Galway, Ireland
| | - Andrew S.C. Rice
- Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| |
Collapse
|
22
|
Hunniford VT, Montroy J, Fergusson DA, Avey MT, Wever KE, McCann SK, Foster M, Fox G, Lafreniere M, Ghaly M, Mannell S, Godwinska K, Gentles A, Selim S, MacNeil J, Sikora L, Sena ES, Page MJ, Macleod M, Moher D, Lalu MM. Epidemiology and reporting characteristics of preclinical systematic reviews. PLoS Biol 2021; 19:e3001177. [PMID: 33951050 PMCID: PMC8128274 DOI: 10.1371/journal.pbio.3001177] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Revised: 05/17/2021] [Accepted: 03/05/2021] [Indexed: 01/10/2023] Open
Abstract
In an effort to better utilize published evidence obtained from animal experiments, systematic reviews of preclinical studies are increasingly more common-along with the methods and tools to appraise them (e.g., SYstematic Review Center for Laboratory animal Experimentation [SYRCLE's] risk of bias tool). We performed a cross-sectional study of a sample of recent preclinical systematic reviews (2015-2018) and examined a range of epidemiological characteristics and used a 46-item checklist to assess reporting details. We identified 442 reviews published across 43 countries in 23 different disease domains that used 26 animal species. Reporting of key details to ensure transparency and reproducibility was inconsistent across reviews and within article sections. Items were most completely reported in the title, introduction, and results sections of the reviews, while least reported in the methods and discussion sections. Less than half of reviews reported that a risk of bias assessment for internal and external validity was undertaken, and none reported methods for evaluating construct validity. Our results demonstrate that a considerable number of preclinical systematic reviews investigating diverse topics have been conducted; however, their quality of reporting is inconsistent. Our study provides the justification and evidence to inform the development of guidelines for conducting and reporting preclinical systematic reviews.
Collapse
Affiliation(s)
- Victoria T. Hunniford
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Joshua Montroy
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Dean A. Fergusson
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | | | - Kimberley E. Wever
- SYstematic Review Center for Laboratory animal Experimentation (SYRCLE), Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Sarah K. McCann
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health (BIH) and Charité—Universitätsmedizin Berlin, Berlin, Germany
| | - Madison Foster
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Grace Fox
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Mackenzie Lafreniere
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Mira Ghaly
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Sydney Mannell
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Karolina Godwinska
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Avonae Gentles
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Shehab Selim
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Jenna MacNeil
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Lindsey Sikora
- Health Sciences Library, University of Ottawa, Ottawa, Canada
| | - Emily S. Sena
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Matthew J. Page
- School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Manoj M. Lalu
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Anesthesiology and Pain Medicine, The Ottawa Hospital, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
23
|
Macleod M, Collings AM, Graf C, Kiermer V, Mellor D, Swaminathan S, Sweet D, Vinson V. The MDAR (Materials Design Analysis Reporting) Framework for transparent reporting in the life sciences. Proc Natl Acad Sci U S A 2021; 118:e2103238118. [PMID: 33893240 PMCID: PMC8092464 DOI: 10.1073/pnas.2103238118] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022] Open
Affiliation(s)
- Malcolm Macleod
- Edinburgh CAMARADES group, Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh EH8 9YL, United Kingdom;
| | | | - Chris Graf
- John Wiley & Sons, Oxford OX4 2DQ, United Kingdom
| | | | - David Mellor
- Center for Open Science, Charlottesville, VA 22903;
| | | | | | - Valda Vinson
- Science, American Association for the Advancement of Science, Washington, DC 20005
| |
Collapse
|
24
|
Bahor Z, Liao J, Currie G, Ayder C, Macleod M, McCann SK, Bannach-Brown A, Wever K, Soliman N, Wang Q, Doran-Constant L, Young L, Sena ES, Sena C. Development and uptake of an online systematic review platform: the early years of the CAMARADES Systematic Review Facility (SyRF). BMJ Open Sci 2021; 5:e100103. [PMID: 35047698 PMCID: PMC8647599 DOI: 10.1136/bmjos-2020-100103] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2020] [Revised: 02/10/2021] [Accepted: 02/11/2021] [Indexed: 01/20/2023] Open
Abstract
Preclinical research is a vital step in the drug discovery pipeline and more generally in helping to better understand human disease aetiology and its management. Systematic reviews (SRs) can be powerful in summarising and appraising this evidence concerning a specific research question, to highlight areas of improvements, areas for further research and areas where evidence may be sufficient to take forward to other research domains, for instance clinical trial. Guidance and tools for preclinical research synthesis remain limited despite their clear utility. We aimed to create an online end-to-end platform primarily for conducting SRs of preclinical studies, that was flexible enough to support a wide variety of experimental designs, was adaptable to different research questions, would allow users to adopt emerging automated tools and support them during their review process using best practice. In this article, we introduce the Systematic Review Facility (https://syrf.org.uk), which was launched in 2016 and designed to support primarily preclinical SRs from small independent projects to large, crowdsourced projects. We discuss the architecture of the app and its features, including the opportunity to collaborate easily, to efficiently manage projects, to screen and annotate studies for important features (metadata), to extract outcome data into a secure database, and tailor these steps to each project. We introduce how we are working to leverage the use of automation tools and allow the integration of these services to accelerate and automate steps in the systematic review workflow.
Collapse
Affiliation(s)
- Zsanett Bahor
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, Scotland, UK
| | - Jing Liao
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, Scotland, UK
| | - Gillian Currie
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, Scotland, UK
| | - Can Ayder
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, Scotland, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, Scotland, UK
| | - Sarah K McCann
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany
- QUEST - Center for Transforming Biomedical Research, Berlin Institute of Health (BIH), Berlin, Germany
| | - Alexandra Bannach-Brown
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, Scotland, UK
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany
- QUEST - Center for Transforming Biomedical Research, Berlin Institute of Health (BIH), Berlin, Germany
- Institute for Evidence-Based Practice, Bond University, Robina, Queensland, Australia
| | - Kimberley Wever
- Systematic Review Centre for Laboratory animal Experimentation (SYRCLE), Department for Health Evidence, Nijmegen Institute for Health Sciences, Radboud University Medical Center, Nijmegen, The Netherlands
| | | | - Qianying Wang
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, Scotland, UK
| | | | | | - Emily S Sena
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, Scotland, UK
| | - Chris Sena
- Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, Scotland, UK
| |
Collapse
|
25
|
Harman NL, Sanz-Moreno A, Papoutsopoulou S, Lloyd KA, Ameen-Ali KE, Macleod M, Williamson PR. Can harmonisation of outcomes bridge the translation gap for pre-clinical research? A systematic review of outcomes measured in mouse models of type 2 diabetes. J Transl Med 2020; 18:468. [PMID: 33298112 PMCID: PMC7727210 DOI: 10.1186/s12967-020-02649-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2020] [Accepted: 11/29/2020] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND In pre-clinical research, systematic reviews have the potential to mitigate translational challenges by facilitating understanding of how pre-clinical studies can inform future clinical research. Yet their conduct is encumbered by heterogeneity in the outcomes measured and reported, and those outcomes may not always relate to the most clinically important outcomes. We aimed to systematically review outcomes measured and reported in pre-clinical in vivo studies of pharmacological interventions to treat high blood glucose in mouse models of type 2 diabetes. METHODS A systematic review of pre-clinical in vivo studies of pharmacological interventions aimed at addressing elevated blood glucose in mouse models of type 2 diabetes was completed. Studies were screened for eligibility and outcomes extracted from the included studies. The outcomes were recorded verbatim and classified into outcome domains using an existing outcome taxonomy. Outcomes were also compared to those identified in a systematic review of registered phase 3/4 clinical trials for glucose lowering interventions in people with type 2 diabetes. RESULTS Review of 280 included studies identified 532 unique outcomes across 19 domains. No single outcome, or domain, was measured in all studies and only 132 (21%) had also been measured in registered phase 3/4 clinical trials. A core outcome set, representing the minimum that should be measured and reported, developed for type 2 diabetes effectiveness clinical trials includes 18 core outcomes, of these 12 (71%) outcomes were measured and reported in one or more of the included pre-clinical studies. CONCLUSIONS There is heterogeneity of outcomes reported in pre-clinical research. Harmonisation of outcomes across the research pathway using a core outcome set may facilitate interpretation, evidence synthesis and translational success, and may contribute to the refinement of the use of animals in research. Systematic review registration: The study was prospectively registered on the PROSPERO Database, registration number CRD42018106831.
Collapse
Affiliation(s)
- Nicola L Harman
- Department of Health Data Science, University of Liverpool, Liverpool, L69 3GL, UK.
| | - Adrián Sanz-Moreno
- German Mouse Clinic, Institute of Experimental Genetics, HMGU, Neuherberg, 85764, Germany
| | - Stamatia Papoutsopoulou
- Cellular and Molecular Physiology, Institute of Translational Medicine, University of Liverpool, Liverpool, L69 3GL, UK
| | - Katie A Lloyd
- Clinical Translational Research Innovation Centre (CTRIC), Altnagelvin Hospital, University of Ulster, Londonderry, BT47 6SB, UK
| | - Kamar E Ameen-Ali
- Translational and Clinical Research Institute, Faculty of Medical Sciences, Newcastle University, Newcastle, NE4 5PL, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Paula R Williamson
- Department of Health Data Science, University of Liverpool, Liverpool, L69 3GL, UK
| |
Collapse
|
26
|
Janiaud P, Axfors C, van't Hooft J, Saccilotto R, Agarwal A, Appenzeller-Herzog C, Contopoulos-Ioannidis DG, Danchev V, Dirnagl U, Ewald H, Gartlehner G, Goodman SN, Haber NA, Ioannidis AD, Ioannidis JPA, Lythgoe MP, Ma W, Macleod M, Malički M, Meerpohl JJ, Min Y, Moher D, Nagavci B, Naudet F, Pauli-Magnus C, O'Sullivan JW, Riedel N, Roth JA, Sauermann M, Schandelmaier S, Schmitt AM, Speich B, Williamson PR, Hemkens LG. The worldwide clinical trial research response to the COVID-19 pandemic - the first 100 days. F1000Res 2020; 9:1193. [PMID: 33082937 PMCID: PMC7539080 DOI: 10.12688/f1000research.26707.2] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 10/20/2020] [Indexed: 12/13/2022] Open
Abstract
Background: Never before have clinical trials drawn as much public attention as those testing interventions for COVID-19. We aimed to describe the worldwide COVID-19 clinical research response and its evolution over the first 100 days of the pandemic. Methods: Descriptive analysis of planned, ongoing or completed trials by April 9, 2020 testing any intervention to treat or prevent COVID-19, systematically identified in trial registries, preprint servers, and literature databases. A survey was conducted of all trials to assess their recruitment status up to July 6, 2020. Results: Most of the 689 trials (overall target sample size 396,366) were small (median sample size 120; interquartile range [IQR] 60-300) but randomized (75.8%; n=522) and were often conducted in China (51.1%; n=352) or the USA (11%; n=76). 525 trials (76.2%) planned to include 155,571 hospitalized patients, and 25 (3.6%) planned to include 96,821 health-care workers. Treatments were evaluated in 607 trials (88.1%), frequently antivirals (n=144) or antimalarials (n=112); 78 trials (11.3%) focused on prevention, including 14 vaccine trials. No trial investigated social distancing. Interventions tested in 11 trials with >5,000 participants were also tested in 169 smaller trials (median sample size 273; IQR 90-700). Hydroxychloroquine alone was investigated in 110 trials. While 414 trials (60.0%) expected completion in 2020, only 35 trials (4.1%; 3,071 participants) were completed by July 6. Of 112 trials with detailed recruitment information, 55 had recruited <20% of the targeted sample; 27 between 20-50%; and 30 over 50% (median 14.8% [IQR 2.0-62.0%]). Conclusions: The size and speed of the COVID-19 clinical trials agenda is unprecedented. However, most trials were small investigating a small fraction of treatment options. The feasibility of this research agenda is questionable, and many trials may end in futility, wasting research resources. Much better coordination is needed to respond to global health threats.
Collapse
Affiliation(s)
- Perrine Janiaud
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Department of Clinical Research, University of Basel, Basel, Switzerland
| | - Cathrine Axfors
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Department for Women’s and Children’s Health, Uppsala University, Uppsala, Sweden
| | - Janneke van't Hooft
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Amsterdam University Medical Center, Amsterdam University, Amsterdam, The Netherlands
| | - Ramon Saccilotto
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
| | - Arnav Agarwal
- Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| | | | | | - Valentin Danchev
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Stanford Prevention Research Center, Department of Medicine,, Stanford University School of Medicine, Stanford, California, USA
| | - Ulrich Dirnagl
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health, Berlin, Germany
| | - Hannah Ewald
- University Medical Library, University of Basel, Basel, Switzerland
| | - Gerald Gartlehner
- Department for Evidence-based Medicine and Evaluation, Danube University Krems, Krems, Austria
- RTI International, Research Triangle Park Laboratories, Raleigh, North Carolina, USA
| | - Steven N. Goodman
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Stanford University School of Medicine, Stanford University School of Medicine, Stanford, California, USA
- Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, California, USA
| | - Noah A. Haber
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
| | - Angeliki Diotima Ioannidis
- Molecular Toxicology Interdepartmental Program, University of California, Los Angeles, Los Angeles, California, USA
| | - John P. A. Ioannidis
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Stanford Prevention Research Center, Department of Medicine,, Stanford University School of Medicine, Stanford, California, USA
- Stanford University School of Medicine, Stanford University School of Medicine, Stanford, California, USA
- Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, California, USA
- Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany
| | - Mark P. Lythgoe
- Department of Surgery & Cancer, Imperial College London, London, UK
| | - Wenyan Ma
- Department of Clinical Research, University of Basel, Basel, Switzerland
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Mario Malički
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
| | - Joerg J. Meerpohl
- Institute for Evidence in Medicine, Medical Center and Faculty of Medicine, University of Freiburg, Freiburg, Germany
- Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany
| | - Yan Min
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Stanford University School of Medicine, Stanford University School of Medicine, Stanford, California, USA
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Health Research Institute, Ottawa, Canada
| | - Blin Nagavci
- Institute for Evidence in Medicine, Medical Center and Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Florian Naudet
- CHU Rennes, Inserm, CIC 1414 [(Centre d’Investigation Clinique de Rennes)],, University of Rennes 1, Rennes, France
| | | | - Jack W. O'Sullivan
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Division of Cardiology, Department of Medicine, Stanford University School of Medicine, Stanford, California, USA
| | - Nico Riedel
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health, Berlin, Germany
| | - Jan A. Roth
- Department of Clinical Research, University of Basel, Basel, Switzerland
- Division of Infectious Diseases and Hospital Epidemiology, University Hospital Basel, Basel, Switzerland
| | - Mandy Sauermann
- Division of Infectious Diseases and Hospital Epidemiology, University Hospital Basel, Basel, Switzerland
| | - Stefan Schandelmaier
- Department of Clinical Research, University of Basel, Basel, Switzerland
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Andreas M. Schmitt
- Department of Clinical Research, University of Basel, Basel, Switzerland
- Deparment of Medical Oncology, University Hospital Basel, Basel, Switzerland
| | - Benjamin Speich
- Department of Clinical Research, University of Basel, Basel, Switzerland
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Paula R. Williamson
- MRC/NIHR Trials Methodology Research Partnership, University of Liverpool, Liverpool, UK
| | - Lars G. Hemkens
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Department of Clinical Research, University of Basel, Basel, Switzerland
- Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany
| |
Collapse
|
27
|
Janiaud P, Axfors C, van't Hooft J, Saccilotto R, Agarwal A, Appenzeller-Herzog C, Contopoulos-Ioannidis DG, Danchev V, Dirnagl U, Ewald H, Gartlehner G, Goodman SN, Haber NA, Ioannidis AD, Ioannidis JPA, Lythgoe MP, Ma W, Macleod M, Malički M, Meerpohl JJ, Min Y, Moher D, Nagavci B, Naudet F, Pauli-Magnus C, O'Sullivan JW, Riedel N, Roth JA, Sauermann M, Schandelmaier S, Schmitt AM, Speich B, Williamson PR, Hemkens LG. The worldwide clinical trial research response to the COVID-19 pandemic - the first 100 days. F1000Res 2020; 9:1193. [PMID: 33082937 PMCID: PMC7539080 DOI: 10.12688/f1000research.26707.1] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 09/28/2020] [Indexed: 12/13/2022] Open
Abstract
Background: Never before have clinical trials drawn as much public attention as those testing interventions for COVID-19. We aimed to describe the worldwide COVID-19 clinical research response and its evolution over the first 100 days of the pandemic. Methods: Descriptive analysis of planned, ongoing or completed trials by April 9, 2020 testing any intervention to treat or prevent COVID-19, systematically identified in trial registries, preprint servers, and literature databases. A survey was conducted of all trials to assess their recruitment status up to July 6, 2020. Results: Most of the 689 trials (overall target sample size 396,366) were small (median sample size 120; interquartile range [IQR] 60-300) but randomized (75.8%; n=522) and were often conducted in China (51.1%; n=352) or the USA (11%; n=76). 525 trials (76.2%) planned to include 155,571 hospitalized patients, and 25 (3.6%) planned to include 96,821 health-care workers. Treatments were evaluated in 607 trials (88.1%), frequently antivirals (n=144) or antimalarials (n=112); 78 trials (11.3%) focused on prevention, including 14 vaccine trials. No trial investigated social distancing. Interventions tested in 11 trials with >5,000 participants were also tested in 169 smaller trials (median sample size 273; IQR 90-700). Hydroxychloroquine alone was investigated in 110 trials. While 414 trials (60.0%) expected completion in 2020, only 35 trials (4.1%; 3,071 participants) were completed by July 6. Of 112 trials with detailed recruitment information, 55 had recruited <20% of the targeted sample; 27 between 20-50%; and 30 over 50% (median 14.8% [IQR 2.0-62.0%]). Conclusions: The size and speed of the COVID-19 clinical trials agenda is unprecedented. However, most trials were small investigating a small fraction of treatment options. The feasibility of this research agenda is questionable, and many trials may end in futility, wasting research resources. Much better coordination is needed to respond to global health threats.
Collapse
Affiliation(s)
- Perrine Janiaud
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Department of Clinical Research, University of Basel, Basel, Switzerland
| | - Cathrine Axfors
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Department for Women’s and Children’s Health, Uppsala University, Uppsala, Sweden
| | - Janneke van't Hooft
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Amsterdam University Medical Center, Amsterdam University, Amsterdam, The Netherlands
| | - Ramon Saccilotto
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
| | - Arnav Agarwal
- Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| | | | | | - Valentin Danchev
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Stanford Prevention Research Center, Department of Medicine,, Stanford University School of Medicine, Stanford, California, USA
| | - Ulrich Dirnagl
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health, Berlin, Germany
| | - Hannah Ewald
- University Medical Library, University of Basel, Basel, Switzerland
| | - Gerald Gartlehner
- Department for Evidence-based Medicine and Evaluation, Danube University Krems, Krems, Austria
- RTI International, Research Triangle Park Laboratories, Raleigh, North Carolina, USA
| | - Steven N. Goodman
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Stanford University School of Medicine, Stanford University School of Medicine, Stanford, California, USA
- Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, California, USA
| | - Noah A. Haber
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
| | - Angeliki Diotima Ioannidis
- Molecular Toxicology Interdepartmental Program, University of California, Los Angeles, Los Angeles, California, USA
| | - John P. A. Ioannidis
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Stanford Prevention Research Center, Department of Medicine,, Stanford University School of Medicine, Stanford, California, USA
- Stanford University School of Medicine, Stanford University School of Medicine, Stanford, California, USA
- Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, California, USA
- Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany
| | - Mark P. Lythgoe
- Department of Surgery & Cancer, Imperial College London, London, UK
| | - Wenyan Ma
- Department of Clinical Research, University of Basel, Basel, Switzerland
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Mario Malički
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
| | - Joerg J. Meerpohl
- Institute for Evidence in Medicine, Medical Center and Faculty of Medicine, University of Freiburg, Freiburg, Germany
- Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany
| | - Yan Min
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Stanford University School of Medicine, Stanford University School of Medicine, Stanford, California, USA
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Health Research Institute, Ottawa, Canada
| | - Blin Nagavci
- Institute for Evidence in Medicine, Medical Center and Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Florian Naudet
- CHU Rennes, Inserm, CIC 1414 [(Centre d’Investigation Clinique de Rennes)],, University of Rennes 1, Rennes, France
| | | | - Jack W. O'Sullivan
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Division of Cardiology, Department of Medicine, Stanford University School of Medicine, Stanford, California, USA
| | - Nico Riedel
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health, Berlin, Germany
| | - Jan A. Roth
- Department of Clinical Research, University of Basel, Basel, Switzerland
- Division of Infectious Diseases and Hospital Epidemiology, University Hospital Basel, Basel, Switzerland
| | - Mandy Sauermann
- Division of Infectious Diseases and Hospital Epidemiology, University Hospital Basel, Basel, Switzerland
| | - Stefan Schandelmaier
- Department of Clinical Research, University of Basel, Basel, Switzerland
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Andreas M. Schmitt
- Department of Clinical Research, University of Basel, Basel, Switzerland
- Deparment of Medical Oncology, University Hospital Basel, Basel, Switzerland
| | - Benjamin Speich
- Department of Clinical Research, University of Basel, Basel, Switzerland
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Paula R. Williamson
- MRC/NIHR Trials Methodology Research Partnership, University of Liverpool, Liverpool, UK
| | - Lars G. Hemkens
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University,, Stanford, California, USA
- Department of Clinical Research, University of Basel, Basel, Switzerland
- Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany
| |
Collapse
|
28
|
Percie du Sert N, Hurst V, Ahluwalia A, Alam S, Avey MT, Baker M, Browne WJ, Clark A, Cuthill IC, Dirnagl U, Emerson M, Garner P, Holgate ST, Howells DW, Karp NA, Lazic SE, Lidster K, MacCallum CJ, Macleod M, Pearl EJ, Petersen OH, Rawle F, Reynolds P, Rooney K, Sena ES, Silberberg SD, Steckler T, Würbel H. The ARRIVE guidelines 2.0: updated guidelines for reporting animal research. J Physiol 2020; 598:3793-3801. [PMID: 32666574 PMCID: PMC7610696 DOI: 10.1113/jp280389] [Citation(s) in RCA: 159] [Impact Index Per Article: 39.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2020] [Accepted: 06/23/2020] [Indexed: 12/14/2022] Open
Abstract
Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the 'ARRIVE Essential 10,' which constitutes the minimum requirement, and the 'Recommended Set,' which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.
Collapse
Affiliation(s)
| | - Viki Hurst
- Science Manager – Experimental Design and Reporting, NC3Rs, London, United Kingdom
| | - Amrita Ahluwalia
- Professor of Vascular Pharmacology, Co-Director, The William Harvey Research Institute, London, United Kingdom
- Director of the Barts Cardiovascular CTU, Queen Mary University of London, London, United Kingdom
| | - Sabina Alam
- Director of Publishing Ethics and Integrity, Taylor & Francis Group, London, United Kingdom
| | - Marc T. Avey
- Lead Health Scientist, Health Science Practice, ICF, Durham, North Carolina, United States of America
| | - Monya Baker
- Senior Editor, Opinion, Nature, San Francisco, California, United States of America
| | - William J. Browne
- Professor of Statistics, School of Education, University of Bristol, Bristol, United Kingdom
| | - Alejandra Clark
- Senior Editor, Team Manager – Life Sciences, PLOS ONE, Cambridge, United Kingdom
| | - Innes C. Cuthill
- Professor of Behavioural Ecology, School of Biological Sciences, University of Bristol, Bristol, United Kingdom
| | - Ulrich Dirnagl
- Director, QUEST Center for Transforming Biomedical Research, Berlin Institute of Health & Department of Experimental Neurology, Charite Universitätsmedizin Berlin, Berlin, Germany
| | - Michael Emerson
- Reader in Platelet Pharmacology, National Heart and Lung Institute, Imperial College London, London, United Kingdom
| | - Paul Garner
- Professor, and Director of the Centre for Evidence Synthesis in Global Health, Clinical Sciences Department, Liverpool School of Tropical Medicin Liverpool, United Kingdom
| | - Stephen T. Holgate
- MRC Clinical Professor, Clinical and Experimental Sciences, University of Southampton, Southampton, United Kingdom
| | - David W. Howells
- Professor of Neuroscience and Brain Plasticity, Tasmanian School of Medicine, University of Tasmania, Hobart, Australia
| | - Natasha A. Karp
- Principal Scientist– Statistician & UK Team Lead, Data Sciences & Quantitative Biology, Discovery Sciences, R&D, AstraZeneca, Cambridge, Unite Kingdom
| | | | - Katie Lidster
- Programme Manager – Animal Welfare, NC3Rs, London, United Kingdom
| | | | - Malcolm Macleod
- Professor of Neurology and Translational Neuroscience, Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
- Academic Lead for Research Improvement and Research Integrity, University of Edinburgh, Edinburgh, United Kingdom
| | - Esther J. Pearl
- Programme Manager – Experimental Design, NC3Rs, London, United Kingdom
| | - Ole H. Petersen
- Director of the Academia Europaea Knowledge Hub, Cardiff University, Cardiff, United Kingdom
| | - Frances Rawle
- Director of Policy, Ethics and Governance, Medical Research Council, London, United Kingdom
| | - Penny Reynolds
- Biostatistician, Statistics in Anesthesiology Research (STAR) Core & Research Assistant Professor, Department of Anesthesiology College of Medicin University of Florida, Gainesville, Florida, United States of America
| | - Kieron Rooney
- Associate Professor, Discipline of Exercise and Sport Science, Faculty of Medicine and Health, University of Sydney, Sydney, Australia
| | - Emily S. Sena
- Stroke Association Kirby Laing Foundation Senior Non-Clinical Lecturer, Centre for Clinical Brain Sciences, University of Edinburgh, Edinburg United Kingdom
| | - Shai D. Silberberg
- Director of Research Quality, National Institute of Neurological Disorders and Stroke, Bethesda, Maryland, United States of America
| | - Thomas Steckler
- Associate Director, BRQC Animal Welfare Strategy Lead, Janssen Pharmaceutica NV, Beerse, Belgium
| | - Hanno Würbel
- Professor for Animal Welfare, Veterinary Public Health Institute, Vetsuisse Faculty, University of Bern, Bern, Switzerland
| |
Collapse
|
29
|
Percie du Sert N, Hurst V, Ahluwalia A, Alam S, Avey MT, Baker M, Browne WJ, Clark A, Cuthill IC, Dirnagl U, Emerson M, Garner P, Holgate ST, Howells DW, Karp NA, Lazic SE, Lidster K, MacCallum CJ, Macleod M, Pearl EJ, Petersen OH, Rawle F, Reynolds P, Rooney K, Sena ES, Silberberg SD, Steckler T, Würbel H. The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research. J Cereb Blood Flow Metab 2020; 40:1769-1777. [PMID: 32663096 PMCID: PMC7430098 DOI: 10.1177/0271678x20943823] [Citation(s) in RCA: 499] [Impact Index Per Article: 124.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/29/2020] [Accepted: 06/30/2020] [Indexed: 01/04/2023]
Abstract
Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the "ARRIVE Essential 10," which constitutes the minimum requirement, and the "Recommended Set," which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.
Collapse
Affiliation(s)
| | - Viki Hurst
- Science Manager – Experimental Design and Reporting, NC3Rs, London, United Kingdom
| | - Amrita Ahluwalia
- Professor of Vascular Pharmacology, Co-Director, The William Harvey Research Institute, London, United Kingdom
- Director of the Barts Cardiovascular CTU, Queen Mary University of London, London, United Kingdom
| | - Sabina Alam
- Director of Publishing Ethics and Integrity, Taylor & Francis Group, London, United Kingdom
| | - Marc T. Avey
- Lead Health Scientist, Health Science Practice, ICF, Durham, North Carolina, United States of America
| | - Monya Baker
- Senior Editor, Opinion, Nature, San Francisco, California, United States of America
| | - William J. Browne
- Professor of Statistics, School of Education, University of Bristol, Bristol, United Kingdom
| | - Alejandra Clark
- Senior Editor, Team Manager – Life Sciences, PLOS ONE, Cambridge, United Kingdom
| | - Innes C. Cuthill
- Professor of Behavioural Ecology, School of Biological Sciences, University of Bristol, Bristol, United Kingdom
| | - Ulrich Dirnagl
- Director, QUEST Center for Transforming Biomedical Research, Berlin Institute of Health & Department of Experimental Neurology, Charite Universitätsmedizin Berlin, Berlin, Germany
| | - Michael Emerson
- Reader in Platelet Pharmacology, National Heart and Lung Institute, Imperial College London, London, United Kingdom
| | - Paul Garner
- Professor, and Director of the Centre for Evidence Synthesis in Global Health, Clinical Sciences Department, Liverpool School of Tropical Medicine, Liverpool, United Kingdom
| | - Stephen T. Holgate
- MRC Clinical Professor, Clinical and Experimental Sciences, University of Southampton, Southampton, United Kingdom
| | - David W. Howells
- Professor of Neuroscience and Brain Plasticity, Tasmanian School of Medicine, University of Tasmania, Hobart, Australia
| | - Natasha A. Karp
- Principal Scientist – Statistician & UK Team Lead, Data Sciences & Quantitative Biology, Discovery Sciences, R&D, AstraZeneca, Cambridge, United Kingdom
| | | | - Katie Lidster
- Programme Manager – Animal Welfare, NC3Rs, London, United Kingdom
| | | | - Malcolm Macleod
- Professor of Neurology and Translational Neuroscience, Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
- Academic Lead for Research Improvement and Research Integrity, University of Edinburgh, Edinburgh, United Kingdom
| | - Esther J. Pearl
- Programme Manager – Experimental Design, NC3Rs, London, United Kingdom
| | - Ole H. Petersen
- Director of the Academia Europaea Knowledge Hub, Cardiff University, Cardiff, United Kingdom
| | - Frances Rawle
- Director of Policy, Ethics and Governance, Medical Research Council, London, United Kingdom
| | - Penny Reynolds
- Biostatistician, Statistics in Anesthesiology Research (STAR) Core & Research Assistant Professor, Department of Anesthesiology College of Medicine, University of Florida, Gainesville, Florida, United States of America
| | - Kieron Rooney
- Associate Professor, Discipline of Exercise and Sport Science, Faculty of Medicine and Health, University of Sydney, Sydney, Australia
| | - Emily S. Sena
- Stroke Association Kirby Laing Foundation Senior Non-Clinical Lecturer, Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Shai D. Silberberg
- Director of Research Quality, National Institute of Neurological Disorders and Stroke, Bethesda, Maryland, United States of America
| | - Thomas Steckler
- Associate Director, BRQC Animal Welfare Strategy Lead, Janssen Pharmaceutica NV, Beerse, Belgium
| | - Hanno Würbel
- Professor for Animal Welfare, Veterinary Public Health Institute, Vetsuisse Faculty, University of Bern, Bern, Switzerland
| |
Collapse
|
30
|
Percie du Sert N, Hurst V, Ahluwalia A, Alam S, Avey MT, Baker M, Browne WJ, Clark A, Cuthill IC, Dirnagl U, Emerson M, Garner P, Holgate ST, Howells DW, Karp NA, Lazic SE, Lidster K, MacCallum CJ, Macleod M, Pearl EJ, Petersen OH, Rawle F, Reynolds P, Rooney K, Sena ES, Silberberg SD, Steckler T, Würbel H. The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research. Exp Physiol 2020; 105:1459-1466. [PMID: 32666546 PMCID: PMC7610926 DOI: 10.1113/ep088870] [Citation(s) in RCA: 81] [Impact Index Per Article: 20.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the "ARRIVE Essential 10," which constitutes the minimum requirement, and the "Recommended Set," which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.
Collapse
Affiliation(s)
| | - Viki Hurst
- Science Manager – Experimental Design and Reporting, NC3Rs, London, United Kingdom
| | - Amrita Ahluwalia
- Professor of Vascular Pharmacology, Co-Director, The William Harvey Research Institute, London, United Kingdom
- Director of the Barts Cardiovascular CTU, Queen Mary University of London, London, United Kingdom
| | - Sabina Alam
- Director of Publishing Ethics and Integrity, Taylor & Francis Group, London, United Kingdom
| | - Marc T. Avey
- Lead Health Scientist, Health Science Practice, ICF, Durham, North Carolina, United States of America
| | - Monya Baker
- Senior Editor, Opinion, Nature, San Francisco, California, United States of America
| | - William J. Browne
- Professor of Statistics, School of Education, University of Bristol, Bristol, United Kingdom
| | - Alejandra Clark
- Senior Editor, Team Manager – Life Sciences, PLOS ONE, Cambridge, United Kingdom
| | - Innes C. Cuthill
- Professor of Behavioural Ecology, School of Biological Sciences, University of Bristol, Bristol, United Kingdom
| | - Ulrich Dirnagl
- Director, QUEST Center for Transforming Biomedical Research, Berlin Institute of Health & Department of Experimental Neurology, Charite Universitätsmedizin Berlin, Berlin, Germany
| | - Michael Emerson
- Reader in Platelet Pharmacology, National Heart and Lung Institute, Imperial College London, London, United Kingdom
| | - Paul Garner
- Professor, and Director of the Centre for Evidence Synthesis in Global Health, Clinical Sciences Department, Liverpool School of Tropical Medicine, Liverpool, United Kingdom
| | - Stephen T. Holgate
- MRC Clinical Professor, Clinical and Experimental Sciences, University of Southampton, Southampton, United Kingdom
| | - David W. Howells
- Professor of Neuroscience and Brain Plasticity, Tasmanian School of Medicine, University of Tasmania, Hobart, Australia
| | - Natasha A. Karp
- Principal Scientist – Statistician & UK Team Lead, Data Sciences & Quantitative Biology, Discovery Sciences, R&D, AstraZeneca, Cambridge, United Kingdom
| | | | - Katie Lidster
- Programme Manager – Animal Welfare, NC3Rs, London, United Kingdom
| | | | - Malcolm Macleod
- Professor of Neurology and Translational Neuroscience, Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
- Academic Lead for Research Improvement and Research Integrity, University of Edinburgh, Edinburgh, United Kingdom
| | - Esther J. Pearl
- Programme Manager – Experimental Design, NC3Rs, London, United Kingdom
| | - Ole H. Petersen
- Director of the Academia Europaea Knowledge Hub, Cardiff University, Cardiff, United Kingdom
| | - Frances Rawle
- Director of Policy, Ethics and Governance, Medical Research Council, London, United Kingdom
| | - Penny Reynolds
- Biostatistician, Statistics in Anesthesiology Research (STAR) Core & Research Assistant Professor, Department of Anesthesiology College of Medicine University of Florida, Gainesville, Florida, United States of America
| | - Kieron Rooney
- Associate Professor, Discipline of Exercise and Sport Science, Faculty of Medicine and Health, University of Sydney, Sydney, Australia
| | - Emily S. Sena
- Stroke Association Kirby Laing Foundation Senior Non-Clinical Lecturer, Centre for Clinical Brain Sciences, University of Edinburgh, Edinburg United Kingdom
| | - Shai D. Silberberg
- Director of Research Quality, National Institute of Neurological Disorders and Stroke, Bethesda, Maryland, United States of America
| | - Thomas Steckler
- Associate Director, BRQC Animal Welfare Strategy Lead, Janssen Pharmaceutica NV, Beerse, Belgium
| | - Hanno Würbel
- Professor for Animal Welfare, Veterinary Public Health Institute, Vetsuisse Faculty, University of Bern, Bern, Switzerland
| |
Collapse
|
31
|
Percie du Sert N, Hurst V, Ahluwalia A, Alam S, Avey MT, Baker M, Browne WJ, Clark A, Cuthill IC, Dirnagl U, Emerson M, Garner P, Holgate ST, Howells DW, Karp NA, Lazic SE, Lidster K, MacCallum CJ, Macleod M, Pearl EJ, Petersen OH, Rawle F, Reynolds P, Rooney K, Sena ES, Silberberg SD, Steckler T, Würbel H. The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research. Br J Pharmacol 2020; 177:3617-3624. [PMID: 32662519 PMCID: PMC7393194 DOI: 10.1111/bph.15193] [Citation(s) in RCA: 292] [Impact Index Per Article: 73.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022] Open
Abstract
Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the "ARRIVE Essential 10," which constitutes the minimum requirement, and the "Recommended Set," which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration (E&E) document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.
Collapse
Affiliation(s)
| | | | - Amrita Ahluwalia
- The William Harvey Research Institute, London, UK
- Barts Cardiovascular CTU, Queen Mary University of London, London, UK
| | | | - Marc T Avey
- Health Science Practice, ICF, Durham, North Carolina, USA
| | | | | | | | - Innes C Cuthill
- School of Biological Sciences, University of Bristol, Bristol, UK
| | - Ulrich Dirnagl
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health & Department of Experimental Neurology, Charite Universitätsmedizin Berlin, Berlin, Germany
| | - Michael Emerson
- National Heart and Lung Institute, Imperial College London, London, UK
| | - Paul Garner
- Centre for Evidence Synthesis in Global Health, Clinical Sciences Department, Liverpool School of Tropical Medicine, Liverpool, UK
| | - Stephen T Holgate
- Clinical and Experimental Sciences, University of Southampton, Southampton, UK
| | - David W Howells
- Tasmanian School of Medicine, University of Tasmania, Hobart, Australia
| | - Natasha A Karp
- Data Sciences & Quantitative Biology, Discovery Sciences, R&D, AstraZeneca, Cambridge, UK
| | | | | | | | | | | | - Ole H Petersen
- Academia Europaea Knowledge Hub, Cardiff University, Cardiff, UK
| | | | - Penny Reynolds
- Statistics in Anesthesiology Research (STAR) Core & Research Assistant Professor, Department of Anesthesiology, College of Medicine, University of Florida, Gainesville, Florida, USA
| | - Kieron Rooney
- Discipline of Exercise and Sport Science, Faculty of Medicine and Health, University of Sydney, Sydney, Australia
| | | | - Shai D Silberberg
- National Institute of Neurological Disorders and Stroke, Bethesda, Maryland, USA
| | | | - Hanno Würbel
- Veterinary Public Health Institute, Vetsuisse Faculty, University of Bern, Bern, Switzerland
| |
Collapse
|
32
|
Percie du Sert N, Hurst V, Ahluwalia A, Alam S, Avey MT, Baker M, Browne WJ, Clark A, Cuthill IC, Dirnagl U, Emerson M, Garner P, Holgate ST, Howells DW, Karp NA, Lazic SE, Lidster K, MacCallum CJ, Macleod M, Pearl EJ, Petersen OH, Rawle F, Reynolds P, Rooney K, Sena ES, Silberberg SD, Steckler T, Wuerbel H. The ARRIVE guidelines 2.0: updated guidelines for reporting animal research. BMJ Open Sci 2020; 4:e100115. [PMID: 34095516 PMCID: PMC7610906 DOI: 10.1136/bmjos-2020-100115] [Citation(s) in RCA: 100] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into two sets, the 'ARRIVE Essential 10', which constitutes the minimum requirement, and the 'Recommended Set', which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.
Collapse
Affiliation(s)
| | - Viki Hurst
- Experimental Design and Reporting, NC3Rs, London, UK
| | - Amrita Ahluwalia
- William Harvey Research Institute, London, UK
- Queen Mary University of London, London, UK
| | | | | | - Monya Baker
- Opinion, Nature, San Francisco, California, USA
| | | | | | - Innes C Cuthill
- School of Biological Sciences, University of Bristol, Bristol, UK
| | - Ulrich Dirnagl
- Quest Center for Transforming Biomedical Research, Berlin Institute of Health, Berlin, Germany
- Department of Experimental Neurology, Charite Universitatsmedizin Berlin, Berlin, Germany
| | - Michael Emerson
- National Heart and Lung Institute, Imperial College London, London, UK
| | - Paul Garner
- Centre for Evidence Synthesis in Global Health, Clinical Sciences Department, Liverpool School of Tropical Medicine, Liverpool, UK
| | - Stephen T Holgate
- Clinical and Experimental Sciences, University of Southampton, Southampton, Hampshire, UK
| | - David W Howells
- Tasmanian School of Medicine, University of Tasmania, Hobart, Tasmania, Australia
| | - Natasha A Karp
- Data Sciences & Quantitative Biology, Discovery Sciences, R&D, AstraZeneca PLC, Cambridge, Cambridgeshire, UK
| | | | | | | | - Malcolm Macleod
- Academic Lead for Research Improvement and Research Integrity, University of Edinburgh, Edinburgh, UK
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | | | - Ole H Petersen
- Academia Europaea Knowledge Hub, Cardiff University, Cardiff, South Glamorgan, UK
| | - Frances Rawle
- Policy, Ethics and Governance, Medical Research Council, London, UK
| | - Penny Reynolds
- Department of Anesthesiology, University of Florida College of Medicine, Gainesville, Florida, USA
| | - Kieron Rooney
- Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia
| | - Emily S Sena
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Shai D Silberberg
- Research Quality, National Institute of Neurological Disorders and Stroke, Bethesda, Maryland, USA
| | | | - Hanno Wuerbel
- Veterinary Public Health Institute, Vetsuisse Faculty, University of Bern, Bern, Switzerland
| |
Collapse
|
33
|
Percie du Sert N, Hurst V, Ahluwalia A, Alam S, Avey MT, Baker M, Browne WJ, Clark A, Cuthill IC, Dirnagl U, Emerson M, Garner P, Holgate ST, Howells DW, Karp NA, Lazic SE, Lidster K, MacCallum CJ, Macleod M, Pearl EJ, Petersen OH, Rawle F, Reynolds P, Rooney K, Sena ES, Silberberg SD, Steckler T, Würbel H. The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research. BMC Vet Res 2020; 16:242. [PMID: 32660541 PMCID: PMC7359286 DOI: 10.1186/s12917-020-02451-y] [Citation(s) in RCA: 115] [Impact Index Per Article: 28.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022] Open
Abstract
Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the "ARRIVE Essential 10," which constitutes the minimum requirement, and the "Recommended Set," which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.
Collapse
Affiliation(s)
| | - Viki Hurst
- Experimental Design and Reporting, NC3Rs, London, UK
| | - Amrita Ahluwalia
- The William Harvey Research Institute, London, UK
- Barts Cardiovascular CTU, Queen Mary University of London, London, UK
| | - Sabina Alam
- Publishing Ethics and Integrity, Taylor & Francis Group, London, UK
| | - Marc T Avey
- Health Science Practice, ICF, Durham, North Carolina, USA
| | - Monya Baker
- Opinion, Nature, San Francisco, California, USA
| | | | | | - Innes C Cuthill
- School of Biological Sciences, University of Bristol, Bristol, UK
| | - Ulrich Dirnagl
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health & Department of Experimental Neurology, Charite Universitätsmedizin Berlin, Berlin, Germany
| | - Michael Emerson
- National Heart and Lung Institute, Imperial College London, London, UK
| | - Paul Garner
- Centre for Evidence Synthesis in Global Health, Clinical Sciences Department, Liverpool School of Tropical Medicine, Liverpool, UK
| | - Stephen T Holgate
- Clinical and Experimental Sciences, University of Southampton, Southampton, UK
| | - David W Howells
- Tasmanian School of Medicine, University of Tasmania, Hobart, Australia
| | - Natasha A Karp
- Data Sciences & Quantitative Biology, Discovery Sciences, R&D, AstraZeneca, Cambridge, UK
| | | | | | | | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | | | - Ole H Petersen
- Academia Europaea Knowledge Hub, Cardiff University, Cardiff, UK
| | - Frances Rawle
- Policy, Ethics and Governance, Medical Research Council, London, UK
| | - Penny Reynolds
- Statistics in Anesthesiology Research (STAR), Department of Anesthesiology College of Medicine, University of Florida, Gainesville, Florida, USA
| | - Kieron Rooney
- Discipline of Exercise and Sport Science, Faculty of Medicine and Health, University of Sydney, Sydney, Australia
| | - Emily S Sena
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Shai D Silberberg
- Research Quality, National Institute of Neurological Disorders and Stroke, Bethesda, MD, USA
| | | | - Hanno Würbel
- Veterinary Public Health Institute, Vetsuisse Faculty, University of Bern, Bern, Switzerland
| |
Collapse
|
34
|
Percie du Sert N, Ahluwalia A, Alam S, Avey MT, Baker M, Browne WJ, Clark A, Cuthill IC, Dirnagl U, Emerson M, Garner P, Holgate ST, Howells DW, Hurst V, Karp NA, Lazic SE, Lidster K, MacCallum CJ, Macleod M, Pearl EJ, Petersen OH, Rawle F, Reynolds P, Rooney K, Sena ES, Silberberg SD, Steckler T, Würbel H. Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0. PLoS Biol 2020; 18:e3000411. [PMID: 32663221 PMCID: PMC7360025 DOI: 10.1371/journal.pbio.3000411] [Citation(s) in RCA: 903] [Impact Index Per Article: 225.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022] Open
Abstract
Improving the reproducibility of biomedical research is a major challenge. Transparent and accurate reporting is vital to this process; it allows readers to assess the reliability of the findings and repeat or build upon the work of other researchers. The ARRIVE guidelines (Animal Research: Reporting In Vivo Experiments) were developed in 2010 to help authors and journals identify the minimum information necessary to report in publications describing in vivo experiments. Despite widespread endorsement by the scientific community, the impact of ARRIVE on the transparency of reporting in animal research publications has been limited. We have revised the ARRIVE guidelines to update them and facilitate their use in practice. The revised guidelines are published alongside this paper. This explanation and elaboration document was developed as part of the revision. It provides further information about each of the 21 items in ARRIVE 2.0, including the rationale and supporting evidence for their inclusion in the guidelines, elaboration of details to report, and examples of good reporting from the published literature. This document also covers advice and best practice in the design and conduct of animal studies to support researchers in improving standards from the start of the experimental design process through to publication.
Collapse
Affiliation(s)
| | - Amrita Ahluwalia
- The William Harvey Research Institute, London, United Kingdom
- Barts Cardiovascular CTU, Queen Mary University of London, London, United Kingdom
| | - Sabina Alam
- Taylor & Francis Group, London, United Kingdom
| | - Marc T. Avey
- Health Science Practice, ICF, Durham, North Carolina, United States of America
| | - Monya Baker
- Nature, San Francisco, California, United States of America
| | | | | | - Innes C. Cuthill
- School of Biological Sciences, University of Bristol, Bristol, United Kingdom
| | - Ulrich Dirnagl
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health & Department of Experimental Neurology, Charite Universitätsmedizin Berlin, Berlin, Germany
| | - Michael Emerson
- National Heart and Lung Institute, Imperial College London, London, United Kingdom
| | - Paul Garner
- Centre for Evidence Synthesis in Global Health, Clinical Sciences Department, Liverpool School of Tropical Medicine, Liverpool, United Kingdom
| | - Stephen T. Holgate
- Clinical and Experimental Sciences, University of Southampton, Southampton, United Kingdom
| | - David W. Howells
- Tasmanian School of Medicine, University of Tasmania, Hobart, Australia
| | | | - Natasha A. Karp
- Data Sciences & Quantitative Biology, Discovery Sciences, R&D, AstraZeneca, Cambridge, United Kingdom
| | | | | | | | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | | | - Ole H. Petersen
- Academia Europaea Knowledge Hub, Cardiff University, Cardiff, United Kingdom
| | | | - Penny Reynolds
- Statistics in Anesthesiology Research (STAR) Core, Department of Anesthesiology, College of Medicine, University of Florida, Gainesville, Florida, United States of America
| | - Kieron Rooney
- Discipline of Exercise and Sport Science, Faculty of Medicine and Health, University of Sydney, Sydney, Australia
| | - Emily S. Sena
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Shai D. Silberberg
- National Institute of Neurological Disorders and Stroke, Bethesda, Maryland, United States of America
| | | | - Hanno Würbel
- Veterinary Public Health Institute, Vetsuisse Faculty, University of Bern, Bern, Switzerland
| |
Collapse
|
35
|
McFall A, Hietamies TM, Bernard A, Aimable M, Allan SM, Bath PM, Brezzo G, Carare RO, Carswell HV, Clarkson AN, Currie G, Farr TD, Fowler JH, Good M, Hainsworth AH, Hall C, Horsburgh K, Kalaria R, Kehoe P, Lawrence C, Macleod M, McColl BW, McNeilly A, Miller AA, Miners S, Mok V, O’Sullivan M, Platt B, Sena ES, Sharp M, Strangward P, Szymkowiak S, Touyz RM, Trueman RC, White C, McCabe C, Work LM, Quinn TJ. UK consensus on pre-clinical vascular cognitive impairment functional outcomes assessment: Questionnaire and workshop proceedings. J Cereb Blood Flow Metab 2020; 40:1402-1414. [PMID: 32151228 PMCID: PMC7307003 DOI: 10.1177/0271678x20910552] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/03/2019] [Revised: 11/21/2019] [Accepted: 12/06/2019] [Indexed: 11/15/2022]
Abstract
Assessment of outcome in preclinical studies of vascular cognitive impairment (VCI) is heterogenous. Through an ARUK Scottish Network supported questionnaire and workshop (mostly UK-based researchers), we aimed to determine underlying variability and what could be implemented to overcome identified challenges. Twelve UK VCI research centres were identified and invited to complete a questionnaire and attend a one-day workshop. Questionnaire responses demonstrated agreement that outcome assessments in VCI preclinical research vary by group and even those common across groups, may be performed differently. From the workshop, six themes were discussed: issues with preclinical models, reasons for choosing functional assessments, issues in interpretation of functional assessments, describing and reporting functional outcome assessments, sharing resources and expertise, and standardization of outcomes. Eight consensus points emerged demonstrating broadly that the chosen assessment should reflect the deficit being measured, and therefore that one assessment does not suit all models; guidance/standardisation on recording VCI outcome reporting is needed and that uniformity would be aided by a platform to share expertise, material, protocols and procedures thus reducing heterogeneity and so increasing potential for collaboration, comparison and replication. As a result of the workshop, UK wide consensus statements were agreed and future priorities for preclinical research identified.
Collapse
Affiliation(s)
- Aisling McFall
- Institute of Cardiovascular & Medical Sciences, College of
Medical, Veterinary & Life Sciences, University of Glasgow, Glasgow,
UK
| | - Tuuli M Hietamies
- Institute of Cardiovascular & Medical Sciences, College of
Medical, Veterinary & Life Sciences, University of Glasgow, Glasgow,
UK
| | - Ashton Bernard
- Institute of Cardiovascular & Medical Sciences, College of
Medical, Veterinary & Life Sciences, University of Glasgow, Glasgow,
UK
| | - Margaux Aimable
- Centre for Discovery Brain Sciences, University of Edinburgh,
Edinburgh, UK
| | - Stuart M Allan
- Lydia Becker Institute of Immunology and Inflammation, Division
of Neuroscience and Experimental Psychology, School of Biological Sciences,
Faculty of Biology, Medicine and Health, The University of Manchester,
Manchester Academic Health Science Centre, Manchester, UK
| | - Philip M Bath
- Stroke Trials Unit, Division of Clinical Neuroscience,
University of Nottingham, Nottingham, UK
| | - Gaia Brezzo
- Centre for Discovery Brain Sciences, University of Edinburgh,
Edinburgh, UK
| | - Roxana O Carare
- Faculty of Medicine, University of Southampton, Southampton,
UK
| | - Hilary V Carswell
- University of Strathclyde, Strathclyde Institute of Pharmacy and
Biomedical Science, Glasgow, UK
| | - Andrew N Clarkson
- The Department of Anatomy, Brain Health Research Centre and
Brain Research New Zealand, University of Otago, Dunedin, New Zealand
| | - Gillian Currie
- Centre for Discovery Brain Sciences, University of Edinburgh,
Edinburgh, UK
| | - Tracy D Farr
- School of Life Sciences, University of Nottingham, Nottingham ,
UK
| | - Jill H Fowler
- Centre for Discovery Brain Sciences, University of Edinburgh,
Edinburgh, UK
| | - Mark Good
- School of Psychology, Cardiff University, Cardiff, UK
| | - Atticus H Hainsworth
- Molecular & Clinical Sciences Research Institute, St
George’s University of London, London, UK
| | - Catherine Hall
- School of Psychology, University of Sussex, Brighton, UK
| | - Karen Horsburgh
- Centre for Discovery Brain Sciences, University of Edinburgh,
Edinburgh, UK
| | - Rajesh Kalaria
- Institute of Neuroscience, Newcastle University, Newcastle Upon
Tyne, UK
| | - Patrick Kehoe
- Institute of Clinical Neurosciences, University of Bristol,
Bristol, UK
| | - Catherine Lawrence
- Lydia Becker Institute of Immunology and Inflammation, Division
of Neuroscience and Experimental Psychology, School of Biological Sciences,
Faculty of Biology, Medicine and Health, The University of Manchester,
Manchester Academic Health Science Centre, Manchester, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh,
Edinburgh, UK
| | - Barry W McColl
- Centre for Discovery Brain Sciences, University of Edinburgh,
Edinburgh, UK
- UK Dementia Research Institute, Edinburgh Medical School,
University of Edinburgh, Edinburgh, UK
| | - Alison McNeilly
- School of Medicine, University of Dundee, Ninewells Hospital,
Dundee, Scotland
| | - Alyson A Miller
- Institute of Cardiovascular & Medical Sciences, College of
Medical, Veterinary & Life Sciences, University of Glasgow, Glasgow,
UK
| | - Scott Miners
- Institute of Clinical Neurosciences, University of Bristol,
Bristol, UK
| | - Vincent Mok
- Gerald Choa Neuroscience Centre, Therese Pei Fong Chow Research
Centre for Prevention of Dementia, Division of Neurology, Department of Medicine
and Therapeutics, The Chinese University of Hong Kong, Hong Kong
| | - Michael O’Sullivan
- Faculty of Medicine, The University of Queensland, Queensland,
Australia
| | - Bettina Platt
- Institute of Medical Sciences, University of Aberdeen,
Aberdeen, Scotland
| | - Emily S Sena
- Centre for Clinical Brain Sciences, University of Edinburgh,
Edinburgh, UK
| | - Matthew Sharp
- Faculty of Medicine, University of Southampton, Southampton,
UK
| | - Patrick Strangward
- Lydia Becker Institute of Immunology and Inflammation, Division
of Neuroscience and Experimental Psychology, School of Biological Sciences,
Faculty of Biology, Medicine and Health, The University of Manchester,
Manchester Academic Health Science Centre, Manchester, UK
| | - Stefan Szymkowiak
- Centre for Discovery Brain Sciences, University of Edinburgh,
Edinburgh, UK
- UK Dementia Research Institute, Edinburgh Medical School,
University of Edinburgh, Edinburgh, UK
| | - Rhian M Touyz
- Institute of Cardiovascular & Medical Sciences, College of
Medical, Veterinary & Life Sciences, University of Glasgow, Glasgow,
UK
| | | | - Claire White
- Lydia Becker Institute of Immunology and Inflammation, Division
of Neuroscience and Experimental Psychology, School of Biological Sciences,
Faculty of Biology, Medicine and Health, The University of Manchester,
Manchester Academic Health Science Centre, Manchester, UK
| | - Chris McCabe
- Institute of Neuroscience & Psychology, College of Medical,
Veterinary & Life Sciences, University of Glasgow, Glasgow, UK
| | - Lorraine M Work
- Institute of Cardiovascular & Medical Sciences, College of
Medical, Veterinary & Life Sciences, University of Glasgow, Glasgow,
UK
| | - Terence J Quinn
- Institute of Cardiovascular & Medical Sciences, College of
Medical, Veterinary & Life Sciences, University of Glasgow, Glasgow,
UK
| |
Collapse
|
36
|
Abstract
We explored relationships between male mortality and the sex ratio. (We tested relationships across 142 societies and in longitudinal data from Scotland. A male-biased sex ratio was associated with reduced mortality by intentional self-harm across 142 societies. This was replicated in longitudinal Scottish data, and men were less likely to die by suicide and assault when there were more men in the population only when levels of unemployment were low. We argue that this is consistent with a theoretical model in which men increase investment in relationships and offspring as "competition" under a male-biased sex ratio, and that the conflicting results of previous work may stem from divergent effects of the sex ratio on mortality depending upon relative deprivation.
Collapse
|
37
|
Percie du Sert N, Hurst V, Ahluwalia A, Alam S, Avey MT, Baker M, Browne WJ, Clark A, Cuthill IC, Dirnagl U, Emerson M, Garner P, Holgate ST, Howells DW, Karp NA, Lazic SE, Lidster K, MacCallum CJ, Macleod M, Pearl EJ, Petersen OH, Rawle F, Reynolds P, Rooney K, Sena ES, Silberberg SD, Steckler T, Würbel H. The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research. PLoS Biol 2020; 18:e3000410. [PMID: 32663219 PMCID: PMC7360023 DOI: 10.1371/journal.pbio.3000410] [Citation(s) in RCA: 1941] [Impact Index Per Article: 485.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022] Open
Abstract
Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the "ARRIVE Essential 10," which constitutes the minimum requirement, and the "Recommended Set," which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration (E&E) document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.
Collapse
Affiliation(s)
| | | | - Amrita Ahluwalia
- The William Harvey Research Institute, London, United Kingdom
- Barts Cardiovascular CTU, Queen Mary University of London, London, United Kingdom
| | - Sabina Alam
- Taylor & Francis Group, London, United Kingdom
| | - Marc T. Avey
- Health Science Practice, ICF, Durham, North Carolina, United States of America
| | - Monya Baker
- Nature, San Francisco, California, United States of America
| | | | | | - Innes C. Cuthill
- School of Biological Sciences, University of Bristol, Bristol, United Kingdom
| | - Ulrich Dirnagl
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health & Department of Experimental Neurology, Charite Universitätsmedizin Berlin, Berlin, Germany
| | - Michael Emerson
- National Heart and Lung Institute, Imperial College London, London, United Kingdom
| | - Paul Garner
- Centre for Evidence Synthesis in Global Health, Clinical Sciences Department, Liverpool School of Tropical Medicine, Liverpool, United Kingdom
| | - Stephen T. Holgate
- Clinical and Experimental Sciences, University of Southampton, Southampton, United Kingdom
| | - David W. Howells
- Tasmanian School of Medicine, University of Tasmania, Hobart, Australia
| | - Natasha A. Karp
- Data Sciences & Quantitative Biology, Discovery Sciences, R&D, AstraZeneca, Cambridge, United Kingdom
| | | | | | | | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | | | - Ole H. Petersen
- Academia Europaea Knowledge Hub, Cardiff University, Cardiff, United Kingdom
| | | | - Penny Reynolds
- Statistics in Anesthesiology Research (STAR) Core, Department of Anesthesiology, College of Medicine, University of Florida, Gainesville, Florida, United States of America
| | - Kieron Rooney
- Discipline of Exercise and Sport Science, Faculty of Medicine and Health, University of Sydney, Sydney, Australia
| | - Emily S. Sena
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Shai D. Silberberg
- National Institute of Neurological Disorders and Stroke, Bethesda, Maryland, United States of America
| | | | - Hanno Würbel
- Veterinary Public Health Institute, Vetsuisse Faculty, University of Bern, Bern, Switzerland
| |
Collapse
|
38
|
Lalu MM, Montroy J, Begley CG, Bubela T, Hunniford V, Ripsman D, Wesch N, Kimmelman J, Macleod M, Moher D, Tieu A, Sikora L, Fergusson DA. Identifying and understanding factors that affect the translation of therapies from the laboratory to patients: a study protocol. F1000Res 2020; 9:485. [PMID: 33123348 PMCID: PMC7570319 DOI: 10.12688/f1000research.23663.2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 10/09/2020] [Indexed: 11/20/2022] Open
Abstract
Background: The process of translating preclinical findings into a clinical setting takes decades. Previous studies have suggested that only 5-10% of the most promising preclinical studies are successfully translated into viable clinical applications. The underlying determinants of this low success rate (e.g. poor experimental design, suboptimal animal models, poor reporting) have not been examined in an empirical manner. Our study aims to determine the contemporary success rate of preclinical-to-clinical translation, and subsequently determine if an association between preclinical study design and translational success/failure exists. Methods: Established systematic review methodology will be used with regards to the literature search, article screening and study selection process. Preclinical, basic science studies published in high impact basic science journals between 1995 and 2015 will be included. Included studies will focus on publicly available interventions with potential clinical promise. The primary outcome will be successful clinical translation of promising therapies - defined as the conduct of at least one Phase II trial (or greater) with a positive finding. A case-control study will then be performed to evaluate the association between elements of preclinical study design and reporting and the likelihood of successful translation. Discussion: This study will provide a comprehensive analysis of the therapeutic translation from the laboratory bench to the bedside. Importantly, any association between factors of study design and the success of translation will be identified. These findings may inform future research teams attempting preclinical-to-clinical translation. Results will be disseminated to identified knowledge users that fund/support preclinical research.
Collapse
Affiliation(s)
- Manoj M. Lalu
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Anesthesiology and Pain Medicine, Ottawa Hospital, Ottawa, Ontario, Canada
- Department of Cellular and Molecular Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Joshua Montroy
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
| | | | - Tania Bubela
- Faculty of Health Science, Simon Fraser University, Burnaby, British Columbia, Canada
| | - Victoria Hunniford
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
| | - David Ripsman
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Neil Wesch
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
| | | | - Malcolm Macleod
- Center for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - David Moher
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
| | - Alvin Tieu
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Lindsey Sikora
- Health Sciences Library, University of Ottawa, Ottawa, Ontario, Canada
| | - Dean A. Fergusson
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Cellular and Molecular Medicine, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
39
|
Lalu MM, Montroy J, Begley CG, Bubela T, Hunniford V, Ripsman D, Wesch N, Kimmelman J, Macleod M, Moher D, Tieu A, Sikora L, Fergusson DA. Identifying and understanding factors that affect the translation of therapies from the laboratory to patients: a study protocol. F1000Res 2020; 9:485. [PMID: 33123348 PMCID: PMC7570319 DOI: 10.12688/f1000research.23663.1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 05/06/2020] [Indexed: 03/31/2024] Open
Abstract
Background: The process of translating preclinical findings into a clinical setting takes decades. Previous studies have suggested that only 5-10% of the most promising preclinical studies are successfully translated into viable clinical applications. The underlying determinants of this low success rate (e.g. poor experimental design, suboptimal animal models, poor reporting) have not been examined in an empirical manner. Our study aims to determine the contemporary success rate of preclinical-to-clinical translation, and subsequently determine if an association between preclinical study design and translational success/failure exists. Methods: Established systematic review methodology will be used with regards to the literature search, article screening and study selection process. Preclinical, basic science studies published in high impact basic science journals between 1995 and 2015 will be included. Included studies will focus on publicly available interventions with potential clinical promise. The primary outcome will be successful clinical translation of promising therapies - defined as the conduct of at least one Phase II trial (or greater) with a positive finding. A case-control study will then be performed to evaluate the association between elements of preclinical study design and reporting and the likelihood of successful translation. Discussion: This study will provide a comprehensive analysis of the therapeutic translation from the laboratory bench to the bedside. Importantly, any association between factors of study design and the success of translation will be identified. These findings may inform future research teams attempting preclinical-to-clinical translation. Results will be disseminated to identified knowledge users that fund/support preclinical research.
Collapse
Affiliation(s)
- Manoj M. Lalu
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Anesthesiology and Pain Medicine, Ottawa Hospital, Ottawa, Ontario, Canada
- Department of Cellular and Molecular Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Joshua Montroy
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
| | | | - Tania Bubela
- Faculty of Health Science, Simon Fraser University, Burnaby, British Columbia, Canada
| | - Victoria Hunniford
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
| | - David Ripsman
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Neil Wesch
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
| | | | - Malcolm Macleod
- Center for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - David Moher
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
| | - Alvin Tieu
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Lindsey Sikora
- Health Sciences Library, University of Ottawa, Ottawa, Ontario, Canada
| | - Dean A. Fergusson
- Clinical Epidemiology Program, Ottawa General Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Cellular and Molecular Medicine, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
40
|
Mair G, Chappell F, Martin C, Dye D, Bath PM, Muir KW, von Kummer R, Al-Shahi Salman R, Sandercock PAG, Macleod M, Sprigg N, White P, Wardlaw JM. Real-world Independent Testing of e-ASPECTS Software (RITeS): statistical analysis plan. AMRC Open Res 2020; 2:20. [PMID: 35800260 PMCID: PMC7612993 DOI: 10.12688/amrcopenres.12904.1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
Background: Artificial intelligence-based software may automatically detect ischaemic stroke lesions and provide an Alberta Stroke Program Early CT score (ASPECTS) on CT, and identify arterial occlusion and provide a collateral score on CTA. Large-scale independent testing will inform clinical use, but is lacking. We aim to test e-ASPECTS and e-CTA (Brainomix, Oxford UK) using CT scans obtained from a range of clinical studies. Methods: Using prospectively collected baseline CT and CTA scans from 10 national/international clinical stroke trials or registries (total >6600 patients), we will select a large clinically representative sample for testing e-ASPECTS and e-CTA compared to previously acquired independent expert human interpretation (reference standard). Our primary aims are to test agreement between software-derived and masked human expert ASPECTS, and the diagnostic accuracy of e-ASPECTS for identifying all causes of stroke symptoms using follow-up imaging and final clinical opinion as diagnostic ground truth. Our secondary aims are to test when and why e-ASPECTS is more or less accurate, or succeeds/fails to produce results, agreement between e-CTA and human expert CTA interpretation, and repeatability of e-ASPECTS/e-CTA results. All testing will be conducted on an intention-to-analyse basis. We will assess agreement between software and expert-human ratings and test the diagnostic accuracy of software. Conclusions: RITeS will provide comprehensive, robust and representative testing of e-ASPECTS and e-CTA against the current gold-standard, expert-human interpretation.
Collapse
Affiliation(s)
- Grant Mair
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, EH16 4SB, UK
| | - Francesca Chappell
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, EH16 4SB, UK
| | - Chloe Martin
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, EH16 4SB, UK
| | - David Dye
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, EH16 4SB, UK
| | - Philip M. Bath
- Stroke Trials Unit, University of Nottingham, Nottingham, NG5 1PB, UK
| | - Keith W. Muir
- Institute of Neuroscience & Psychology, University of Glasgow, Glasgow, G51 4TF, UK
| | - Rüdiger von Kummer
- Department of Neuroradiology, University Hospital Dresden, Dresden, 01309, Germany
| | | | | | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, EH16 4SB, UK
| | - Nikola Sprigg
- Stroke Trials Unit, University of Nottingham, Nottingham, NG5 1PB, UK
| | - Philip White
- Institute of Neuroscience, Newcastle University, Newcastle upon Tyne, NE2 4AX, UK
| | - Joanna M. Wardlaw
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, EH16 4SB, UK
| |
Collapse
|
41
|
Vollert J, Schenker E, Macleod M, Bespalov A, Wuerbel H, Michel M, Dirnagl U, Potschka H, Waldron AM, Wever K, Steckler T, van de Casteele T, Altevogt B, Sil A, Rice ASC. Systematic review of guidelines for internal validity in the design, conduct and analysis of preclinical biomedical experiments involving laboratory animals. BMJ Open Sci 2020; 4:e100046. [PMID: 35047688 PMCID: PMC8647591 DOI: 10.1136/bmjos-2019-100046] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 12/10/2019] [Accepted: 01/15/2020] [Indexed: 02/01/2023] Open
Abstract
Over the last two decades, awareness of the negative repercussions of flaws in the planning, conduct and reporting of preclinical research involving experimental animals has been growing. Several initiatives have set out to increase transparency and internal validity of preclinical studies, mostly publishing expert consensus and experience. While many of the points raised in these various guidelines are identical or similar, they differ in detail and rigour. Most of them focus on reporting, only few of them cover the planning and conduct of studies. The aim of this systematic review is to identify existing experimental design, conduct, analysis and reporting guidelines relating to preclinical animal research. A systematic search in PubMed, Embase and Web of Science retrieved 13 863 unique results. After screening these on title and abstract, 613 papers entered the full-text assessment stage, from which 60 papers were retained. From these, we extracted unique 58 recommendations on the planning, conduct and reporting of preclinical animal studies. Sample size calculations, adequate statistical methods, concealed and randomised allocation of animals to treatment, blinded outcome assessment and recording of animal flow through the experiment were recommended in more than half of the publications. While we consider these recommendations to be valuable, there is a striking lack of experimental evidence on their importance and relative effect on experiments and effect sizes.
Collapse
Affiliation(s)
- Jan Vollert
- Pain Medicine, Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK
| | - Esther Schenker
- Institut de Recherches Internationales Servier, Suresnes, Île-de-France, France
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, Edinburgh Medical School, The University of Edinburgh, Edinburgh, Scotland, UK
| | - Anton Bespalov
- Partnership for Assessment and Accreditation of Scientific Practice, Heidelberg, Germany
- Valdman Institute of Pharmacology, Pavlov First State Medical University of Saint Petersburg, Sankt Petersburg, Russian Federation
| | - Hanno Wuerbel
- Division of Animal Welfare, Vetsuisse Faculty, VPH Institute, University of Bern, Bern, Switzerland
| | - Martin Michel
- Universitätsmedizin Mainz, Johannes Gutenberg Universität Mainz, Mainz, Rheinland-Pfalz, Germany
| | - Ulrich Dirnagl
- Department of Experimental Neurology, Charité–Universitätsmedizin Berlin, Berlin, Germany
| | - Heidrun Potschka
- Institute of Pharmacology, Toxicology, and Pharmacy, Ludwig-Maximilians-Universitat Munchen, Munchen, Bayern, Germany
| | - Ann-Marie Waldron
- Institute of Pharmacology, Toxicology, and Pharmacy, Ludwig-Maximilians-Universitat Munchen, Munchen, Bayern, Germany
| | - Kimberley Wever
- Systematic Review Centre for Laboratory Animal Experimentation, Department for Health Evidence, Nijmegen Institute for Health Sciences, Radboud Universiteit, Nijmegen, Gelderland, Netherlands
| | | | | | | | - Annesha Sil
- Institute of Medical Sciences, University of Aberdeen, Aberdeen, UK
| | - Andrew S C Rice
- Pain Medicine, Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK
| | | |
Collapse
|
42
|
Abstract
Research synthesis is the process of bringing together findings and attributes from different publications, for example, to give a more complete description of phenomena than is usually possible in a single work. We bring the Research Synthesis Series to BMC Biology to promote meta-analyses, other research syntheses including meta-research studies, and research synthesis methodologies in biology, facilitating their dissemination to broader communities.
Collapse
Affiliation(s)
- Shinichi Nakagawa
- Evolution & Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales, Sydney, NSW, 2052, Australia.
| | - Julia Koricheva
- Department of Biological Sciences, Royal Holloway University of London, Egham, Surrey, TW20 0EX, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, EH16 4SB, UK
| | - Wolfgang Viechtbauer
- Department of Psychiatry and Neuropsychology, School for Mental Health and Neuroscience, Faculty of Health, Medicine, and Life Sciences, Maastricht University, 6200 MD, Maastricht, The Netherlands
| |
Collapse
|
43
|
Olai H, Thornéus G, Watson H, Macleod M, Rhodes J, Friberg H, Nielsen N, Cronberg T, Deierborg T. Meta-analysis of targeted temperature management in animal models of cardiac arrest. Intensive Care Med Exp 2020; 8:3. [PMID: 31953652 PMCID: PMC6969098 DOI: 10.1186/s40635-019-0291-9] [Citation(s) in RCA: 39] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Accepted: 12/29/2019] [Indexed: 01/13/2023] Open
Abstract
BACKGROUND Targeted temperature management (TTM) of 32 to 34 °C has been the standard treatment for out-of-hospital cardiac arrest since clinical trials in 2002 indicated benefit on survival and neurological outcome. In 2013, a clinical trial showed no difference in outcome between TTM of 33 °C and TTM of 36 °C. In this meta-analysis, we investigate the evidence for TTM in animal models of cardiac arrest. METHODS We searched PubMed and EMBASE for adult animal studies using TTM as a treatment in different models of cardiac arrest or global brain ischemia which reported neurobehavioural outcome, brain histology or mortality. We used a random effects model to calculate estimates of efficacy and assessed risk of bias using an adapted eight-item version of the Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies (CAMARADES) quality checklist. We also used a scoring system based on the recommendations of the Stroke Treatment Academic Industry Roundtable (STAIR), to assess the scope of testing in the field. Included studies which investigated a post-ischemic induction of TTM had their treatment regimens characterized with regard to depth, duration and time to treatment and scored against the modified STAIR criteria. RESULTS The initial and updated search generated 17809 studies after duplicate removal. One hundred eighty-one studies met the inclusion criteria, including data from 1,787, 6,495 and 2,945 animals for neurobehavioural, histological and mortality outcomes, respectively. TTM was favoured compared to control for all outcomes. TTM was beneficial using short and prolonged cooling, deep and moderate temperature reduction, and early and delayed time to treatment. Median [IQR] study quality was 4 [3 to 6]. Eighteen studies checked seven or more of the eight CAMARADES quality items. There was no clear correlation between study quality and efficacy for any outcome. STAIR analysis identified 102 studies investigating post-ischemic induction of TTM, comprising 147 different treatment regimens of TTM. Only 2 and 8 out of 147 regimens investigated comorbid and gyrencephalic animals, respectively. CONCLUSIONS TTM is beneficial under most experimental conditions in animal models of cardiac arrest or global brain ischemia. However, research on gyrencephalic species and especially comorbid animals is uncommon and a possible translational gap. Also, low study quality suggests risk of bias within studies. Future animal research should focus on mimicking the clinical scenario and employ similar rigour in trial design to that of modern clinical trials.
Collapse
Affiliation(s)
- Hilmer Olai
- Department of Experimental Medical Science, Experimental Neuroinflammation Laboratory, Lund University, Lund, Sweden
| | - Gustav Thornéus
- Department of Experimental Medical Science, Experimental Neuroinflammation Laboratory, Lund University, Lund, Sweden
| | - Hannah Watson
- Department of Anaesthesia, Western General Hospital, NHS Lothian, Edinburgh, UK
- Department of Critical Care, Western General Hospital, NHS Lothian, Edinburgh, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - Jonathan Rhodes
- Department of Anaesthesia, Critical care and Pain Medicine/NHS Lothian, University of Edinburgh, Edinburgh, UK
| | - Hans Friberg
- Department of Clinical Sciences, Anesthesia & Intensive care, Skåne University Hospital, Lund University, Lund, Sweden
| | - Niklas Nielsen
- Department of Clinical Sciences Lund, Anesthesia & Intensive care, Helsingborg Hospital, Lund University, Lund, Sweden
| | - Tobias Cronberg
- Department of Clinical Sciences Lund, Neurology, Skåne University Hospital, Lund University, Lund, Sweden
| | - Tomas Deierborg
- Department of Experimental Medical Science, Experimental Neuroinflammation Laboratory, Lund University, Lund, Sweden.
| |
Collapse
|
44
|
Abstract
Increasing focus on issues of research reproducibility affords us the opportunity to review some of the key issues related in vivo research. First, we set out some key definitions, to guide the reader through the rest of the paper. Next we consider issues of epistemology, of how animal experiments lead to changes in our understanding of biomedicine and, potentially, to the development of new therapeutics. Here we consider the meaning of statistical significance; the importance of understanding whether findings have general truth; and the advances in knowledge which can result from 'failed' replication. Then, we consider weaknesses in the design, conduct and reporting of experiments, and review evidence for this from systematic reviews and from experimental studies addressing these issues. We consider the impact that these weaknesses have on the development of new treatments for human disease, and reflect on the response to these issues from the biomedical research community. Finally, we consider strategies for improvement including increased use of brief, pre-registered study protocols; pre-registration, open publication and open data; and the central importance of education in improving research performance.
Collapse
Affiliation(s)
- Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Swapna Mohan
- National Institutes of Health, 8600 Rockville Pike, Bethesda, Maryland (previously in the Division of Policy and Education at the NIH Office of Laboratory Animal Welfare, NIH)
| |
Collapse
|
45
|
Watzlawick R, Antonic A, Sena ES, Kopp MA, Rind J, Dirnagl U, Macleod M, Howells DW, Schwab JM. Outcome heterogeneity and bias in acute experimental spinal cord injury: A meta-analysis. Neurology 2019; 93:e40-e51. [PMID: 31175207 DOI: 10.1212/wnl.0000000000007718] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2018] [Accepted: 02/11/2019] [Indexed: 01/18/2023] Open
Abstract
OBJECTIVE To determine whether and to what degree bias and underestimated variability undermine the predictive value of preclinical research for clinical translation. METHODS We investigated experimental spinal cord injury (SCI) studies for outcome heterogeneity and the impact of bias. Data from 549 preclinical SCI studies including 9,535 animals were analyzed with meta-regression to assess the effect of various study characteristics and the quality of neurologic recovery. RESULTS Overall, the included interventions reported a neurobehavioral outcome improvement of 26.3% (95% confidence interval 24.3-28.4). Response to treatment was dependent on experimental modeling paradigms (neurobehavioral score, site of injury, and animal species). Applying multiple outcome measures was consistently associated with smaller effect sizes compared with studies applying only 1 outcome measure. More than half of the studies (51.2%) did not report blinded assessment, constituting a likely source of evaluation bias, with an overstated effect size of 7.2%. Assessment of publication bias, which extrapolates to identify likely missing data, suggested that between 2% and 41% of experiments remain unpublished. Inclusion of these theoretical missing studies suggested an overestimation of efficacy, reducing the effect sizes by between 0.9% and 14.3%. CONCLUSIONS We provide empirical evidence of prevalent bias in the design and reporting of experimental SCI studies, resulting in overestimation of the effectiveness. Bias compromises the internal validity and jeopardizes the successful translation of SCI therapies from the bench to bedside.
Collapse
Affiliation(s)
- Ralf Watzlawick
- From Charité-Universitätsmedizin Berlin (R.W., M.A.K., J.R., U.D., J.M.S.), corporate member of the Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health; Department of Neurology and Experimental Neurology (R.W., M.A.K., J.R., J.M.S.), Charité Campus Mitte, Clinical and Experimental Spinal Cord Injury Research Laboratory (Neuroparaplegiology), Charité-Universitätsmedizin Berlin; Department of Neurosurgery (R.W.), Freiburg University Medical Center, Germany; Department of Neuroscience (A.A.), Central Clinical School, Monash University, Melbourne; Stroke Division (E.S.S., M.M., D.W.H.), Melbourne, Victoria, Australia; Departments of Neurology and Clinical Neurosciences (E.S.S., M.M.), University of Edinburgh, UK; Center for Stroke Research Berlin (U.D.) and Excellence Cluster Neurocure (U.D.), Charité-Universitätsmedizin, Berlin, Germany; German Center for Neurodegenerative Diseases (U.D.), Bonn; Berlin Institute of Health (M.A.K., U.D.), Germany; University of Tasmania (D.W.H.), School of Medicine, Faculty of Health, Medical Sciences Precinct, Hobart, Australia; Department of Neurology (J.M.S.), Spinal Cord Injury Medicine (Paraplegiology), and Belford Center for Spinal Cord Injury (J.M.S.), Departments of Neuroscience and Physical Medicine and Rehabilitation, The Neurological Institute, The Ohio State University, Wexner Medical Center, Columbus
| | - Ana Antonic
- From Charité-Universitätsmedizin Berlin (R.W., M.A.K., J.R., U.D., J.M.S.), corporate member of the Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health; Department of Neurology and Experimental Neurology (R.W., M.A.K., J.R., J.M.S.), Charité Campus Mitte, Clinical and Experimental Spinal Cord Injury Research Laboratory (Neuroparaplegiology), Charité-Universitätsmedizin Berlin; Department of Neurosurgery (R.W.), Freiburg University Medical Center, Germany; Department of Neuroscience (A.A.), Central Clinical School, Monash University, Melbourne; Stroke Division (E.S.S., M.M., D.W.H.), Melbourne, Victoria, Australia; Departments of Neurology and Clinical Neurosciences (E.S.S., M.M.), University of Edinburgh, UK; Center for Stroke Research Berlin (U.D.) and Excellence Cluster Neurocure (U.D.), Charité-Universitätsmedizin, Berlin, Germany; German Center for Neurodegenerative Diseases (U.D.), Bonn; Berlin Institute of Health (M.A.K., U.D.), Germany; University of Tasmania (D.W.H.), School of Medicine, Faculty of Health, Medical Sciences Precinct, Hobart, Australia; Department of Neurology (J.M.S.), Spinal Cord Injury Medicine (Paraplegiology), and Belford Center for Spinal Cord Injury (J.M.S.), Departments of Neuroscience and Physical Medicine and Rehabilitation, The Neurological Institute, The Ohio State University, Wexner Medical Center, Columbus
| | - Emily S Sena
- From Charité-Universitätsmedizin Berlin (R.W., M.A.K., J.R., U.D., J.M.S.), corporate member of the Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health; Department of Neurology and Experimental Neurology (R.W., M.A.K., J.R., J.M.S.), Charité Campus Mitte, Clinical and Experimental Spinal Cord Injury Research Laboratory (Neuroparaplegiology), Charité-Universitätsmedizin Berlin; Department of Neurosurgery (R.W.), Freiburg University Medical Center, Germany; Department of Neuroscience (A.A.), Central Clinical School, Monash University, Melbourne; Stroke Division (E.S.S., M.M., D.W.H.), Melbourne, Victoria, Australia; Departments of Neurology and Clinical Neurosciences (E.S.S., M.M.), University of Edinburgh, UK; Center for Stroke Research Berlin (U.D.) and Excellence Cluster Neurocure (U.D.), Charité-Universitätsmedizin, Berlin, Germany; German Center for Neurodegenerative Diseases (U.D.), Bonn; Berlin Institute of Health (M.A.K., U.D.), Germany; University of Tasmania (D.W.H.), School of Medicine, Faculty of Health, Medical Sciences Precinct, Hobart, Australia; Department of Neurology (J.M.S.), Spinal Cord Injury Medicine (Paraplegiology), and Belford Center for Spinal Cord Injury (J.M.S.), Departments of Neuroscience and Physical Medicine and Rehabilitation, The Neurological Institute, The Ohio State University, Wexner Medical Center, Columbus
| | - Marcel A Kopp
- From Charité-Universitätsmedizin Berlin (R.W., M.A.K., J.R., U.D., J.M.S.), corporate member of the Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health; Department of Neurology and Experimental Neurology (R.W., M.A.K., J.R., J.M.S.), Charité Campus Mitte, Clinical and Experimental Spinal Cord Injury Research Laboratory (Neuroparaplegiology), Charité-Universitätsmedizin Berlin; Department of Neurosurgery (R.W.), Freiburg University Medical Center, Germany; Department of Neuroscience (A.A.), Central Clinical School, Monash University, Melbourne; Stroke Division (E.S.S., M.M., D.W.H.), Melbourne, Victoria, Australia; Departments of Neurology and Clinical Neurosciences (E.S.S., M.M.), University of Edinburgh, UK; Center for Stroke Research Berlin (U.D.) and Excellence Cluster Neurocure (U.D.), Charité-Universitätsmedizin, Berlin, Germany; German Center for Neurodegenerative Diseases (U.D.), Bonn; Berlin Institute of Health (M.A.K., U.D.), Germany; University of Tasmania (D.W.H.), School of Medicine, Faculty of Health, Medical Sciences Precinct, Hobart, Australia; Department of Neurology (J.M.S.), Spinal Cord Injury Medicine (Paraplegiology), and Belford Center for Spinal Cord Injury (J.M.S.), Departments of Neuroscience and Physical Medicine and Rehabilitation, The Neurological Institute, The Ohio State University, Wexner Medical Center, Columbus
| | - Julian Rind
- From Charité-Universitätsmedizin Berlin (R.W., M.A.K., J.R., U.D., J.M.S.), corporate member of the Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health; Department of Neurology and Experimental Neurology (R.W., M.A.K., J.R., J.M.S.), Charité Campus Mitte, Clinical and Experimental Spinal Cord Injury Research Laboratory (Neuroparaplegiology), Charité-Universitätsmedizin Berlin; Department of Neurosurgery (R.W.), Freiburg University Medical Center, Germany; Department of Neuroscience (A.A.), Central Clinical School, Monash University, Melbourne; Stroke Division (E.S.S., M.M., D.W.H.), Melbourne, Victoria, Australia; Departments of Neurology and Clinical Neurosciences (E.S.S., M.M.), University of Edinburgh, UK; Center for Stroke Research Berlin (U.D.) and Excellence Cluster Neurocure (U.D.), Charité-Universitätsmedizin, Berlin, Germany; German Center for Neurodegenerative Diseases (U.D.), Bonn; Berlin Institute of Health (M.A.K., U.D.), Germany; University of Tasmania (D.W.H.), School of Medicine, Faculty of Health, Medical Sciences Precinct, Hobart, Australia; Department of Neurology (J.M.S.), Spinal Cord Injury Medicine (Paraplegiology), and Belford Center for Spinal Cord Injury (J.M.S.), Departments of Neuroscience and Physical Medicine and Rehabilitation, The Neurological Institute, The Ohio State University, Wexner Medical Center, Columbus
| | - Ulrich Dirnagl
- From Charité-Universitätsmedizin Berlin (R.W., M.A.K., J.R., U.D., J.M.S.), corporate member of the Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health; Department of Neurology and Experimental Neurology (R.W., M.A.K., J.R., J.M.S.), Charité Campus Mitte, Clinical and Experimental Spinal Cord Injury Research Laboratory (Neuroparaplegiology), Charité-Universitätsmedizin Berlin; Department of Neurosurgery (R.W.), Freiburg University Medical Center, Germany; Department of Neuroscience (A.A.), Central Clinical School, Monash University, Melbourne; Stroke Division (E.S.S., M.M., D.W.H.), Melbourne, Victoria, Australia; Departments of Neurology and Clinical Neurosciences (E.S.S., M.M.), University of Edinburgh, UK; Center for Stroke Research Berlin (U.D.) and Excellence Cluster Neurocure (U.D.), Charité-Universitätsmedizin, Berlin, Germany; German Center for Neurodegenerative Diseases (U.D.), Bonn; Berlin Institute of Health (M.A.K., U.D.), Germany; University of Tasmania (D.W.H.), School of Medicine, Faculty of Health, Medical Sciences Precinct, Hobart, Australia; Department of Neurology (J.M.S.), Spinal Cord Injury Medicine (Paraplegiology), and Belford Center for Spinal Cord Injury (J.M.S.), Departments of Neuroscience and Physical Medicine and Rehabilitation, The Neurological Institute, The Ohio State University, Wexner Medical Center, Columbus
| | - Malcolm Macleod
- From Charité-Universitätsmedizin Berlin (R.W., M.A.K., J.R., U.D., J.M.S.), corporate member of the Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health; Department of Neurology and Experimental Neurology (R.W., M.A.K., J.R., J.M.S.), Charité Campus Mitte, Clinical and Experimental Spinal Cord Injury Research Laboratory (Neuroparaplegiology), Charité-Universitätsmedizin Berlin; Department of Neurosurgery (R.W.), Freiburg University Medical Center, Germany; Department of Neuroscience (A.A.), Central Clinical School, Monash University, Melbourne; Stroke Division (E.S.S., M.M., D.W.H.), Melbourne, Victoria, Australia; Departments of Neurology and Clinical Neurosciences (E.S.S., M.M.), University of Edinburgh, UK; Center for Stroke Research Berlin (U.D.) and Excellence Cluster Neurocure (U.D.), Charité-Universitätsmedizin, Berlin, Germany; German Center for Neurodegenerative Diseases (U.D.), Bonn; Berlin Institute of Health (M.A.K., U.D.), Germany; University of Tasmania (D.W.H.), School of Medicine, Faculty of Health, Medical Sciences Precinct, Hobart, Australia; Department of Neurology (J.M.S.), Spinal Cord Injury Medicine (Paraplegiology), and Belford Center for Spinal Cord Injury (J.M.S.), Departments of Neuroscience and Physical Medicine and Rehabilitation, The Neurological Institute, The Ohio State University, Wexner Medical Center, Columbus
| | - David W Howells
- From Charité-Universitätsmedizin Berlin (R.W., M.A.K., J.R., U.D., J.M.S.), corporate member of the Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health; Department of Neurology and Experimental Neurology (R.W., M.A.K., J.R., J.M.S.), Charité Campus Mitte, Clinical and Experimental Spinal Cord Injury Research Laboratory (Neuroparaplegiology), Charité-Universitätsmedizin Berlin; Department of Neurosurgery (R.W.), Freiburg University Medical Center, Germany; Department of Neuroscience (A.A.), Central Clinical School, Monash University, Melbourne; Stroke Division (E.S.S., M.M., D.W.H.), Melbourne, Victoria, Australia; Departments of Neurology and Clinical Neurosciences (E.S.S., M.M.), University of Edinburgh, UK; Center for Stroke Research Berlin (U.D.) and Excellence Cluster Neurocure (U.D.), Charité-Universitätsmedizin, Berlin, Germany; German Center for Neurodegenerative Diseases (U.D.), Bonn; Berlin Institute of Health (M.A.K., U.D.), Germany; University of Tasmania (D.W.H.), School of Medicine, Faculty of Health, Medical Sciences Precinct, Hobart, Australia; Department of Neurology (J.M.S.), Spinal Cord Injury Medicine (Paraplegiology), and Belford Center for Spinal Cord Injury (J.M.S.), Departments of Neuroscience and Physical Medicine and Rehabilitation, The Neurological Institute, The Ohio State University, Wexner Medical Center, Columbus
| | - Jan M Schwab
- From Charité-Universitätsmedizin Berlin (R.W., M.A.K., J.R., U.D., J.M.S.), corporate member of the Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health; Department of Neurology and Experimental Neurology (R.W., M.A.K., J.R., J.M.S.), Charité Campus Mitte, Clinical and Experimental Spinal Cord Injury Research Laboratory (Neuroparaplegiology), Charité-Universitätsmedizin Berlin; Department of Neurosurgery (R.W.), Freiburg University Medical Center, Germany; Department of Neuroscience (A.A.), Central Clinical School, Monash University, Melbourne; Stroke Division (E.S.S., M.M., D.W.H.), Melbourne, Victoria, Australia; Departments of Neurology and Clinical Neurosciences (E.S.S., M.M.), University of Edinburgh, UK; Center for Stroke Research Berlin (U.D.) and Excellence Cluster Neurocure (U.D.), Charité-Universitätsmedizin, Berlin, Germany; German Center for Neurodegenerative Diseases (U.D.), Bonn; Berlin Institute of Health (M.A.K., U.D.), Germany; University of Tasmania (D.W.H.), School of Medicine, Faculty of Health, Medical Sciences Precinct, Hobart, Australia; Department of Neurology (J.M.S.), Spinal Cord Injury Medicine (Paraplegiology), and Belford Center for Spinal Cord Injury (J.M.S.), Departments of Neuroscience and Physical Medicine and Rehabilitation, The Neurological Institute, The Ohio State University, Wexner Medical Center, Columbus.
| |
Collapse
|
46
|
Currie GL, Angel-Scott HN, Colvin L, Cramond F, Hair K, Khandoker L, Liao J, Macleod M, McCann SK, Morland R, Sherratt N, Stewart R, Tanriver-Ayder E, Thomas J, Wang Q, Wodarski R, Xiong R, Rice ASC, Sena ES. Animal models of chemotherapy-induced peripheral neuropathy: A machine-assisted systematic review and meta-analysis. PLoS Biol 2019; 17:e3000243. [PMID: 31107871 PMCID: PMC6544332 DOI: 10.1371/journal.pbio.3000243] [Citation(s) in RCA: 38] [Impact Index Per Article: 7.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2018] [Revised: 05/31/2019] [Accepted: 04/08/2019] [Indexed: 12/25/2022] Open
Abstract
We report a systematic review and meta-analysis of research using animal models of chemotherapy-induced peripheral neuropathy (CIPN). We systematically searched 5 online databases in September 2012 and updated the search in November 2015 using machine learning and text mining to reduce the screening for inclusion workload and improve accuracy. For each comparison, we calculated a standardised mean difference (SMD) effect size, and then combined effects in a random-effects meta-analysis. We assessed the impact of study design factors and reporting of measures to reduce risks of bias. We present power analyses for the most frequently reported behavioural tests; 337 publications were included. Most studies (84%) used male animals only. The most frequently reported outcome measure was evoked limb withdrawal in response to mechanical monofilaments. There was modest reporting of measures to reduce risks of bias. The number of animals required to obtain 80% power with a significance level of 0.05 varied substantially across behavioural tests. In this comprehensive summary of the use of animal models of CIPN, we have identified areas in which the value of preclinical CIPN studies might be increased. Using both sexes of animals in the modelling of CIPN, ensuring that outcome measures align with those most relevant in the clinic, and the animal's pain contextualised ethology will likely improve external validity. Measures to reduce risk of bias should be employed to increase the internal validity of studies. Different outcome measures have different statistical power, and this can refine our approaches in the modelling of CIPN.
Collapse
Affiliation(s)
- Gillian L. Currie
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Helena N. Angel-Scott
- Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | - Lesley Colvin
- Department of Anaesthesia, Critical Care & Pain, University of Edinburgh, Edinburgh, United Kingdom
- Division of Population Health and Genomics, University of Dundee, Dundee, United Kingdom
| | - Fala Cramond
- Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | - Kaitlyn Hair
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Laila Khandoker
- Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | - Jing Liao
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Sarah K. McCann
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Rosie Morland
- Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | - Nicki Sherratt
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Robert Stewart
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Ezgi Tanriver-Ayder
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - James Thomas
- EPPI-Centre, University College London, London, United Kingdom
| | - Qianying Wang
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Rachel Wodarski
- Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | - Ran Xiong
- Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | - Andrew S. C. Rice
- Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom
| | - Emily S. Sena
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| |
Collapse
|
47
|
Cramond F, O'Mara-Eves A, Doran-Constant L, Rice ASC, Macleod M, Thomas J. The development and evaluation of an online application to assist in the extraction of data from graphs for use in systematic reviews. Wellcome Open Res 2019; 3:157. [PMID: 30809592 PMCID: PMC6372928 DOI: 10.12688/wellcomeopenres.14738.2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/15/2019] [Indexed: 02/02/2023] Open
Abstract
Background: The extraction of data from the reports of primary studies, on which the results of systematic reviews depend, needs to be carried out accurately. To aid reliability, it is recommended that two researchers carry out data extraction independently. The extraction of statistical data from graphs in PDF files is particularly challenging, as the process is usually completely manual, and reviewers need sometimes to revert to holding a ruler against the page to read off values: an inherently time-consuming and error-prone process. Methods: To mitigate some of the above problems we integrated and customised two existing JavaScript libraries to create a new web-based graphical data extraction tool to assist reviewers in extracting data from graphs. This tool aims to facilitate more accurate and timely data extraction through a user interface which can be used to extract data through mouse clicks. We carried out a non-inferiority evaluation to examine its performance in comparison to standard practice. Results: We found that the customised graphical data extraction tool is not inferior to users' prior preferred current approaches. Our study was not designed to show superiority, but suggests that there may be a saving in time of around 6 minutes per graph, accompanied by a substantial increase in accuracy. Conclusions: Our study suggests that the incorporation of this type of tool in online systematic review software would be beneficial in facilitating the production of accurate and timely evidence synthesis to improve decision-making.
Collapse
Affiliation(s)
- Fala Cramond
- Pain Research, Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK
| | - Alison O'Mara-Eves
- EPPI-Centre, Department of Social Science, UCL Institute of Education, University College London, London, UK
| | | | - Andrew SC Rice
- Pain Research, Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - James Thomas
- EPPI-Centre, Department of Social Science, UCL Institute of Education, University College London, London, UK,
| |
Collapse
|
48
|
Cramond F, O'Mara-Eves A, Doran-Constant L, Rice ASC, Macleod M, Thomas J. The development and evaluation of an online application to assist in the extraction of data from graphs for use in systematic reviews. Wellcome Open Res 2019; 3:157. [PMID: 30809592 PMCID: PMC6372928 DOI: 10.12688/wellcomeopenres.14738.3] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/05/2019] [Indexed: 02/02/2023] Open
Abstract
Background: The extraction of data from the reports of primary studies, on which the results of systematic reviews depend, needs to be carried out accurately. To aid reliability, it is recommended that two researchers carry out data extraction independently. The extraction of statistical data from graphs in PDF files is particularly challenging, as the process is usually completely manual, and reviewers need sometimes to revert to holding a ruler against the page to read off values: an inherently time-consuming and error-prone process. Methods: To mitigate some of the above problems we integrated and customised two existing JavaScript libraries to create a new web-based graphical data extraction tool to assist reviewers in extracting data from graphs. This tool aims to facilitate more accurate and timely data extraction through a user interface which can be used to extract data through mouse clicks. We carried out a non-inferiority evaluation to examine its performance in comparison with participants' standard practice for extracting data from graphs in PDF documents. Results: We found that the customised graphical data extraction tool is not inferior to users' (N=10) prior standard practice. Our study was not designed to show superiority, but suggests that, on average, participants saved around 6 minutes per graph using the new tool, accompanied by a substantial increase in accuracy. Conclusions: Our study suggests that the incorporation of this type of tool in online systematic review software would be beneficial in facilitating the production of accurate and timely evidence synthesis to improve decision-making.
Collapse
Affiliation(s)
- Fala Cramond
- Pain Research, Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK
| | - Alison O'Mara-Eves
- EPPI-Centre, Department of Social Science, UCL Institute of Education, University College London, London, UK
| | | | - Andrew SC Rice
- Pain Research, Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - James Thomas
- EPPI-Centre, Department of Social Science, UCL Institute of Education, University College London, London, UK,
| |
Collapse
|
49
|
Cramond F, O'Mara-Eves A, Doran-Constant L, Rice ASC, Macleod M, Thomas J. The development and evaluation of an online application to assist in the extraction of data from graphs for use in systematic reviews. Wellcome Open Res 2019; 3:157. [PMID: 30809592 PMCID: PMC6372928 DOI: 10.12688/wellcomeopenres.14738.1] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/13/2018] [Indexed: 02/02/2023] Open
Abstract
Background: The extraction of data from the reports of primary studies, on which the results of systematic reviews depend, needs to be carried out accurately. To aid reliability, it is recommended that two researchers carry out data extraction independently. The extraction of statistical data from graphs in PDF files is particularly challenging, as the process is usually completely manual, and reviewers need sometimes to revert to holding a ruler against the page to read off values: an inherently time-consuming and error-prone process. Methods: To mitigate some of the above problems we developed a new web-based graphical data extraction tool to assist reviewers in extracting data from graphs. This tool aims to facilitate more accurate and timely data extraction through a user interface which can be used to extract data through mouse clicks. We carried out a non-inferiority evaluation to examine its performance in comparison to standard practice. Results: We found that our new graphical data extraction tool is not inferior to users' prior preferred current approaches. Our study was not designed to show superiority, but suggests that there may be a saving in time of around 6 minutes per graph, accompanied by a substantial increase in accuracy. Conclusions: Our study suggests that the incorporation of this type of tool in online systematic review software would be beneficial in facilitating the production of accurate and timely evidence synthesis to improve decision-making.
Collapse
Affiliation(s)
- Fala Cramond
- Pain Research, Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK
| | - Alison O'Mara-Eves
- EPPI-Centre, Department of Social Science, UCL Institute of Education, University College London, London, UK
| | | | - Andrew SC Rice
- Pain Research, Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, UK
| | - James Thomas
- EPPI-Centre, Department of Social Science, UCL Institute of Education, University College London, London, UK,
| |
Collapse
|
50
|
Sampaio C, Ware JJ, Macleod M, Wagenmakers EJ, Munafò M. Reader response: Evaluating depression and suicidality in tetrabenazine users with Huntington disease. Neurology 2019; 92:447-448. [PMID: 30804061 DOI: 10.1212/wnl.0000000000006999] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
|