Selective outcome reporting in randomised controlled trials including participants with stroke or transient ischaemic attack: A systematic review

INTRODUCTION

The prevalence of outcome reporting bias (ORB, i.e. selective reporting according to the results observed) across primary outcomes in randomised controlled trials (RCTs) including participants with stroke or transient ischaemic attack (TIA) is unknown.

MATERIALS AND METHODS

We searched the Cochrane Database of Systematic Reviews on 3 February 2021 for reviews published 2008-2020 with at least one RCT of a therapeutic intervention, for participants with stroke or TIA, and a safety or efficacy outcome. We took a random sample of these RCTs and included those with a trial registry record or protocol published before reporting results. Two reviewers assessed discrepancies in outcome reporting across the trial registry record, protocol, statistical analysis plan, and publication for each RCT, using the classification system designed by the Outcome Reporting Bias in Trials group.

RESULTS

Of 600 RCTs, we identified a trial registry record in 120 (20%), a protocol in 28 (5%), and a statistical analysis plan in 5 (1%) with 123 (21%) distinct RCTs being eligible for assessment: 110 (89%, 95% CI 83-94) were at no risk, 7 (6%, 95% CI 3-11) RCTs were at low risk, and 6 (5%, 95% CI 2-10) were at high risk of ORB.

DISCUSSION

The prevalence of ORB in primary outcomes was low in stroke/TIA RCTs that were included in Cochrane reviews and had an identifiable trial registry record or protocol. Concerningly, we were unable to identify a trial registry record or protocol in most of our sample.

CONCLUSION

Work is needed to further reduce ORB in stroke/TIA RCTs and explore the generalisability of these findings to RCTs outside of Cochrane reviews or without a registry record or protocol, as well as to secondary outcomes.

The Automated Systematic Search Deduplicator (ASySD): a rapid, open-source, interoperable tool to remove duplicate citations in biomedical systematic reviews

BACKGROUND

Researchers performing high-quality systematic reviews search across multiple databases to identify relevant evidence. However, the same publication is often retrieved from several databases. Identifying and removing such duplicates ("deduplication") can be extremely time-consuming, but failure to remove these citations can lead to the wrongful inclusion of duplicate data. Many existing tools are not sensitive enough, lack interoperability with other tools, are not freely accessible, or are difficult to use without programming knowledge. Here, we report the performance of our Automated Systematic Search Deduplicator (ASySD), a novel tool to perform automated deduplication of systematic searches for biomedical reviews.

METHODS

We evaluated ASySD's performance on 5 unseen biomedical systematic search datasets of various sizes (1845-79,880 citations). We compared the performance of ASySD with EndNote's automated deduplication option and with the Systematic Review Assistant Deduplication Module (SRA-DM).

RESULTS

ASySD identified more duplicates than either SRA-DM or EndNote, with a sensitivity in different datasets of 0.95 to 0.99. The false-positive rate was comparable to human performance, with a specificity of > 0.99. The tool took less than 1 h to identify and remove duplicates within each dataset.

CONCLUSIONS

For duplicate removal in biomedical systematic reviews, ASySD is a highly sensitive, reliable, and time-saving tool. It is open source and freely available online as both an R package and a user-friendly web application.

Trace amine-associated receptor 1 (TAAR1) agonists for psychosis: protocol for a living systematic review and meta-analysis of human and non-human studies

BACKGROUND

There is an urgent need to develop more effective and safer antipsychotics beyond dopamine 2 receptor antagonists. An emerging and promising approach is TAAR1 agonism. Therefore, we will conduct a living systematic review and meta-analysis to synthesize and triangulate the evidence from preclinical animal experiments and clinical studies on the efficacy, safety, and underlying mechanism of action of TAAR1 agonism for psychosis.

METHODS

Independent searches will be conducted in multiple electronic databases to identify clinical and animal experimental studies comparing TAAR1 agonists with licensed antipsychotics or other control conditions in individuals with psychosis or animal models for psychosis, respectively. The primary outcomes will be overall psychotic symptoms and their behavioural proxies in animals. Secondary outcomes will include side effects and neurobiological measures. Two independent reviewers will conduct study selection, data extraction using predefined forms, and risk of bias assessment using suitable tools based on the study design. Ontologies will be developed to facilitate study identification and data extraction. Data from clinical and animal studies will be synthesized separately using random-effects meta-analysis if appropriate, or synthesis without meta-analysis. Study characteristics will be investigated as potential sources of heterogeneity. Confidence in the evidence for each outcome and source of evidence will be evaluated, considering the summary of the association, potential concerns regarding internal and external validity, and reporting biases. When multiple sources of evidence are available for an outcome, an overall conclusion will be drawn in a triangulation meeting involving a multidisciplinary team of experts. We plan trimonthly updates of the review, and any modifications in the protocol will be documented. The review will be co-produced by multiple stakeholders aiming to produce impactful and relevant results and bridge the gap between preclinical and clinical research on psychosis.

PROTOCOL REGISTRATION

PROSPERO-ID: CRD42023451628.

Designing, conducting, and reporting reproducible animal experiments

In biomedicine and many other fields, there are growing concerns around the reproducibility of research findings, with many researchers being unable to replicate their own or others' results. This raises important questions as to the validity and usefulness of much published research. In this review, we aim to engage researchers in the issue of research reproducibility and equip them with the necessary tools to increase the reproducibility of their research. We first highlight the causes and potential impact of non-reproducible research and emphasise the benefits of working reproducibly for the researcher and broader research community. We address specific targets for improvement and steps that individual researchers can take to increase the reproducibility of their work. We next provide recommendations for improving the design and conduct of experiments, focusing on in vivo animal experiments. We describe common sources of poor internal validity of experiments and offer practical guidance for limiting these potential sources of bias at different experimental stages, as well as discussing other important considerations during experimental design. We provide a list of key resources available to researchers to improve experimental design, conduct, and reporting. We then discuss the importance of open research practices such as study preregistration and the use of preprints and describe recommendations around data management and sharing. Our review emphasises the importance of reproducible work and aims to empower every individual researcher to contribute to the reproducibility of research in their field.

Systematic review and meta-analysis of studies in which burrowing behaviour was assessed in rodent models of disease-associated persistent pain

ABSTRACT

Burrowing behaviour is used to assess pain-associated behaviour in laboratory rodents. To gain insight into how models of disease-associated persistent pain and analgesics affect burrowing behaviour, we performed a systematic review and meta-analysis of studies that assessed burrowing behaviour. A systematic search in March 2020 and update in September 2020 was conducted in 4 databases. Study design characteristics and experimental data were extracted, followed by a random-effects meta-analysis. We explored the association between burrowing and monofilament-induced limb withdrawal. Dose response relationship was investigated for some analgesics. Forty-five studies were included in the meta-analysis, in which 16 model types and 14 drug classes were used. Most experiments used rat (79%) and male (72%) animals. Somatic inflammation and trauma-induced neuropathy models were associated with reduced burrowing behaviour. Analgesics (nonsteroidal anti-inflammatory drug and gabapentinoids) attenuated burrowing deficits in these models. Reporting of measures to reduce risk of bias was unclear except for randomisation which was high. There was not a correlation ( R2 = 0.1421) between burrowing and monofilament-induced limb withdrawal. Opioids, gabapentin, and naproxen showed reduced burrowing behaviour at high doses, whereas ibuprofen and celecoxib showed opposite trend. The findings indicate that burrowing could be used to assess pain-associated behaviour. We support the use of a portfolio of composite measures including spontaneous and stimulus-evoked tests. The information collected here could help in designing experiments involving burrowing assessment in models of disease-associated pain.

A protocol for the systematic review and meta-analysis of thigmotactic behaviour in the open field test in rodent models associated with persistent pain

Objective

Thigmotaxis is an innate predator avoidance behaviour of rodents and is enhanced when animals are under stress. It is characterised by the preference of a rodent to seek shelter, rather than expose itself to the aversive open area. The behaviour has been proposed to be a measurable construct that can address the impact of pain on rodent behaviour. This systematic review will assess whether thigmotaxis can be influenced by experimental persistent pain and attenuated by pharmacological interventions in rodents.

Search strategy

We will conduct search on three electronic databases to identify studies in which thigmotaxis was used as an outcome measure contextualised to a rodent model associated with persistent pain. All studies published until the date of the search will be considered.

Screening and annotation

Two independent reviewers will screen studies based on the order of (1) titles and abstracts, and (2) full texts.

Data management and reporting

For meta-analysis, we will extract thigmotactic behavioural data and calculate effect sizes. Effect sizes will be combined using a random-effects model. We will assess heterogeneity and identify sources of heterogeneity. A risk-of-bias assessment will be conducted to evaluate study quality. Publication bias will be assessed using funnel plots, Egger’s regression and trim-and-fill analysis. We will also extract stimulus-evoked limb withdrawal data to assess its correlation with thigmotaxis in the same animals. The evidence obtained will provide a comprehensive understanding of the strengths and limitations of using thigmotactic outcome measure in animal pain research so that future experimental designs can be optimised. We will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guidelines and disseminate the review findings through publication and conference presentation.

Using median survival in meta-analysis of experimental time-to-event data

BACKGROUND

Time-to-event data is frequently reported in both clinical and preclinical research spheres. Systematic review and meta-analysis is a tool that can help to identify pitfalls in preclinical research conduct and reporting that can help to improve translational efficacy. However, pooling of studies using hazard ratios (HRs) is cumbersome especially in preclinical meta-analyses including large numbers of small studies. Median survival is a much simpler metric although because of some limitations, which may not apply to preclinical data, it is generally not used in survival meta-analysis. We aimed to appraise its performance when compared with hazard ratio-based meta-analysis when pooling large numbers of small, imprecise studies.

METHODS

We simulated a survival dataset with features representative of a typical preclinical survival meta-analysis, including with influence of a treatment and a number of covariates. We calculated individual patient data-based hazard ratios and median survival ratios (MSRs), comparing the summary statistics directly and their performance at random-effects meta-analysis. Finally, we compared their sensitivity to detect associations between treatment and influential covariates at meta-regression.

RESULTS

There was an imperfect correlation between MSR and HR, although the opposing direction of treatment effects between summary statistics appeared not to be a major issue. Precision was more conservative for HR than MSR, meaning that estimates of heterogeneity were lower. There was a slight sensitivity advantage for MSR at meta-analysis and meta-regression, although power was low in all circumstances.

CONCLUSIONS

We believe we have validated MSR as a summary statistic for use in a meta-analysis of small, imprecise experimental survival studies-helping to increase confidence and efficiency in future reviews in this area. While assessment of study precision and therefore weighting is less reliable, MSR appears to perform favourably during meta-analysis. Sensitivity of meta-regression was low for this set of parameters, so pooling of treatments to increase sample size may be required to ensure confidence in preclinical survival meta-regressions.

Epidemiology and reporting characteristics of preclinical systematic reviews

In an effort to better utilize published evidence obtained from animal experiments, systematic reviews of preclinical studies are increasingly more common-along with the methods and tools to appraise them (e.g., SYstematic Review Center for Laboratory animal Experimentation [SYRCLE's] risk of bias tool). We performed a cross-sectional study of a sample of recent preclinical systematic reviews (2015-2018) and examined a range of epidemiological characteristics and used a 46-item checklist to assess reporting details. We identified 442 reviews published across 43 countries in 23 different disease domains that used 26 animal species. Reporting of key details to ensure transparency and reproducibility was inconsistent across reviews and within article sections. Items were most completely reported in the title, introduction, and results sections of the reviews, while least reported in the methods and discussion sections. Less than half of reviews reported that a risk of bias assessment for internal and external validity was undertaken, and none reported methods for evaluating construct validity. Our results demonstrate that a considerable number of preclinical systematic reviews investigating diverse topics have been conducted; however, their quality of reporting is inconsistent. Our study provides the justification and evidence to inform the development of guidelines for conducting and reporting preclinical systematic reviews.

Development and uptake of an online systematic review platform: the early years of the CAMARADES Systematic Review Facility (SyRF)

Preclinical research is a vital step in the drug discovery pipeline and more generally in helping to better understand human disease aetiology and its management. Systematic reviews (SRs) can be powerful in summarising and appraising this evidence concerning a specific research question, to highlight areas of improvements, areas for further research and areas where evidence may be sufficient to take forward to other research domains, for instance clinical trial. Guidance and tools for preclinical research synthesis remain limited despite their clear utility. We aimed to create an online end-to-end platform primarily for conducting SRs of preclinical studies, that was flexible enough to support a wide variety of experimental designs, was adaptable to different research questions, would allow users to adopt emerging automated tools and support them during their review process using best practice. In this article, we introduce the Systematic Review Facility (https://syrf.org.uk), which was launched in 2016 and designed to support primarily preclinical SRs from small independent projects to large, crowdsourced projects. We discuss the architecture of the app and its features, including the opportunity to collaborate easily, to efficiently manage projects, to screen and annotate studies for important features (metadata), to extract outcome data into a secure database, and tailor these steps to each project. We introduce how we are working to leverage the use of automation tools and allow the integration of these services to accelerate and automate steps in the systematic review workflow.

Quality and validity of large animal experiments in stroke: A systematic review

An important factor for successful translational stroke research is study quality. Low-quality studies are at risk of biased results and effect overestimation, as has been intensely discussed for small animal stroke research. However, little is known about the methodological rigor and quality in large animal stroke models, which are becoming more frequently used in the field. Based on research in two databases, this systematic review surveys and analyses the methodological quality in large animal stroke research. Quality analysis was based on the Stroke Therapy Academic Industry Roundtable and the Animals in Research: Reporting In Vivo Experiments guidelines. Our analysis revealed that large animal models are utilized with similar shortcomings as small animal models. Moreover, translational benefits of large animal models may be limited due to lacking implementation of important quality criteria such as randomization, allocation concealment, and blinded assessment of outcome. On the other hand, an increase of study quality over time and a positive correlation between study quality and journal impact factor were identified. Based on the obtained findings, we derive recommendations for optimal study planning, conducting, and data analysis/reporting when using large animal stroke models to fully benefit from the translational advantages offered by these models.

The ARRIVE guidelines 2.0: updated guidelines for reporting animal research

Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the 'ARRIVE Essential 10,' which constitutes the minimum requirement, and the 'Recommended Set,' which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.

The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research

Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the "ARRIVE Essential 10," which constitutes the minimum requirement, and the "Recommended Set," which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.

The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research

Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the "ARRIVE Essential 10," which constitutes the minimum requirement, and the "Recommended Set," which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.

The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research

Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the "ARRIVE Essential 10," which constitutes the minimum requirement, and the "Recommended Set," which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration (E&E) document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.

The ARRIVE guidelines 2.0: updated guidelines for reporting animal research

Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into two sets, the 'ARRIVE Essential 10', which constitutes the minimum requirement, and the 'Recommended Set', which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.

The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research

Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the "ARRIVE Essential 10," which constitutes the minimum requirement, and the "Recommended Set," which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.

Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0

Improving the reproducibility of biomedical research is a major challenge. Transparent and accurate reporting is vital to this process; it allows readers to assess the reliability of the findings and repeat or build upon the work of other researchers. The ARRIVE guidelines (Animal Research: Reporting In Vivo Experiments) were developed in 2010 to help authors and journals identify the minimum information necessary to report in publications describing in vivo experiments. Despite widespread endorsement by the scientific community, the impact of ARRIVE on the transparency of reporting in animal research publications has been limited. We have revised the ARRIVE guidelines to update them and facilitate their use in practice. The revised guidelines are published alongside this paper. This explanation and elaboration document was developed as part of the revision. It provides further information about each of the 21 items in ARRIVE 2.0, including the rationale and supporting evidence for their inclusion in the guidelines, elaboration of details to report, and examples of good reporting from the published literature. This document also covers advice and best practice in the design and conduct of animal studies to support researchers in improving standards from the start of the experimental design process through to publication.

UK consensus on pre-clinical vascular cognitive impairment functional outcomes assessment: Questionnaire and workshop proceedings

Assessment of outcome in preclinical studies of vascular cognitive impairment (VCI) is heterogenous. Through an ARUK Scottish Network supported questionnaire and workshop (mostly UK-based researchers), we aimed to determine underlying variability and what could be implemented to overcome identified challenges. Twelve UK VCI research centres were identified and invited to complete a questionnaire and attend a one-day workshop. Questionnaire responses demonstrated agreement that outcome assessments in VCI preclinical research vary by group and even those common across groups, may be performed differently. From the workshop, six themes were discussed: issues with preclinical models, reasons for choosing functional assessments, issues in interpretation of functional assessments, describing and reporting functional outcome assessments, sharing resources and expertise, and standardization of outcomes. Eight consensus points emerged demonstrating broadly that the chosen assessment should reflect the deficit being measured, and therefore that one assessment does not suit all models; guidance/standardisation on recording VCI outcome reporting is needed and that uniformity would be aided by a platform to share expertise, material, protocols and procedures thus reducing heterogeneity and so increasing potential for collaboration, comparison and replication. As a result of the workshop, UK wide consensus statements were agreed and future priorities for preclinical research identified.

The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research

Reproducible science requires transparent reporting. The ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) were originally developed in 2010 to improve the reporting of animal research. They consist of a checklist of information to include in publications describing in vivo experiments to enable others to scrutinise the work adequately, evaluate its methodological rigour, and reproduce the methods and results. Despite considerable levels of endorsement by funders and journals over the years, adherence to the guidelines has been inconsistent, and the anticipated improvements in the quality of reporting in animal research publications have not been achieved. Here, we introduce ARRIVE 2.0. The guidelines have been updated and information reorganised to facilitate their use in practice. We used a Delphi exercise to prioritise and divide the items of the guidelines into 2 sets, the "ARRIVE Essential 10," which constitutes the minimum requirement, and the "Recommended Set," which describes the research context. This division facilitates improved reporting of animal research by supporting a stepwise approach to implementation. This helps journal editors and reviewers verify that the most important items are being reported in manuscripts. We have also developed the accompanying Explanation and Elaboration (E&E) document, which serves (1) to explain the rationale behind each item in the guidelines, (2) to clarify key concepts, and (3) to provide illustrative examples. We aim, through these changes, to help ensure that researchers, reviewers, and journal editors are better equipped to improve the rigour and transparency of the scientific process and thus reproducibility.

Attitudes towards animal study registries and their characteristics: An online survey of three cohorts of animal researchers

OBJECTIVES

Prospective registration of animal studies has been suggested as a new measure to increase value and reduce waste in biomedical research. We sought to further explore and quantify animal researchers' attitudes and preferences regarding animal study registries (ASRs).

DESIGN

Cross-sectional online survey.

SETTING AND PARTICIPANTS

We conducted a survey with three different samples representing animal researchers: i) corresponding authors from journals with high Eigenfactor, ii) a random Pubmed sample and iii) members of the CAMARADES network.

MAIN OUTCOME MEASURES

Perceived level of importance of different aspects of publication bias, the effect of ASRs on different aspects of research as well as the importance of different research types for being registered.

RESULTS

The survey yielded responses from 413 animal researchers (response rate 7%). The respondents indicated, that some aspects of ASRs can increase administrative burden but could be outweighed by other aspects decreasing this burden. Animal researchers found it more important to register studies that involved animal species with higher levels of cognitive capabilities. The time frame for making registry entries publicly available revealed a strong heterogeneity among respondents, with the largest proportion voting for "access only after consent by the principal investigator" and the second largest proportion voting for "access immediately after registration".

CONCLUSIONS

The fact that the more senior and experienced animal researchers participating in this survey clearly indicated the practical importance of publication bias and the importance of ASRs underscores the problem awareness across animal researchers and the willingness to actively engage in study registration if effective safeguards for the potential weaknesses of ASRs are put into place. To overcome the first-mover dilemma international consensus statements on how to deal with prospective registration of animal studies might be necessary for all relevant stakeholder groups including animal researchers, academic institutions, private companies, funders, regulatory agencies, and journals.

A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guidelines (IICARus)

BACKGROUND

The ARRIVE (Animal Research: Reporting of In Vivo Experiments) guidelines are widely endorsed but compliance is limited. We sought to determine whether journal-requested completion of an ARRIVE checklist improves full compliance with the guidelines.

METHODS

In a randomised controlled trial, manuscripts reporting in vivo animal research submitted to PLOS ONE (March-June 2015) were randomly allocated to either requested completion of an ARRIVE checklist or current standard practice. Authors, academic editors, and peer reviewers were blinded to group allocation. Trained reviewers performed outcome adjudication in duplicate by assessing manuscripts against an operationalised version of the ARRIVE guidelines that consists 108 items. Our primary outcome was the between-group differences in the proportion of manuscripts meeting all ARRIVE guideline checklist subitems.

RESULTS

We randomised 1689 manuscripts (control: n = 844, intervention: n = 845), of which 1269 were sent for peer review and 762 (control: n = 340; intervention: n = 332) accepted for publication. No manuscript in either group achieved full compliance with the ARRIVE checklist. Details of animal husbandry (ARRIVE subitem 9b) was the only subitem to show improvements in reporting, with the proportion of compliant manuscripts rising from 52.1 to 74.1% (X 2 = 34.0, df = 1, p = 2.1 × 10-7) in the control and intervention groups, respectively.

CONCLUSIONS

These results suggest that altering the editorial process to include requests for a completed ARRIVE checklist is not enough to improve compliance with the ARRIVE guidelines. Other approaches, such as more stringent editorial policies or a targeted approach on key quality items, may promote improvements in reporting.

Outcome heterogeneity and bias in acute experimental spinal cord injury: A meta-analysis

OBJECTIVE

To determine whether and to what degree bias and underestimated variability undermine the predictive value of preclinical research for clinical translation.

METHODS

We investigated experimental spinal cord injury (SCI) studies for outcome heterogeneity and the impact of bias. Data from 549 preclinical SCI studies including 9,535 animals were analyzed with meta-regression to assess the effect of various study characteristics and the quality of neurologic recovery.

RESULTS

Overall, the included interventions reported a neurobehavioral outcome improvement of 26.3% (95% confidence interval 24.3-28.4). Response to treatment was dependent on experimental modeling paradigms (neurobehavioral score, site of injury, and animal species). Applying multiple outcome measures was consistently associated with smaller effect sizes compared with studies applying only 1 outcome measure. More than half of the studies (51.2%) did not report blinded assessment, constituting a likely source of evaluation bias, with an overstated effect size of 7.2%. Assessment of publication bias, which extrapolates to identify likely missing data, suggested that between 2% and 41% of experiments remain unpublished. Inclusion of these theoretical missing studies suggested an overestimation of efficacy, reducing the effect sizes by between 0.9% and 14.3%.

CONCLUSIONS

We provide empirical evidence of prevalent bias in the design and reporting of experimental SCI studies, resulting in overestimation of the effectiveness. Bias compromises the internal validity and jeopardizes the successful translation of SCI therapies from the bench to bedside.

Animal models of chemotherapy-induced peripheral neuropathy: A machine-assisted systematic review and meta-analysis

We report a systematic review and meta-analysis of research using animal models of chemotherapy-induced peripheral neuropathy (CIPN). We systematically searched 5 online databases in September 2012 and updated the search in November 2015 using machine learning and text mining to reduce the screening for inclusion workload and improve accuracy. For each comparison, we calculated a standardised mean difference (SMD) effect size, and then combined effects in a random-effects meta-analysis. We assessed the impact of study design factors and reporting of measures to reduce risks of bias. We present power analyses for the most frequently reported behavioural tests; 337 publications were included. Most studies (84%) used male animals only. The most frequently reported outcome measure was evoked limb withdrawal in response to mechanical monofilaments. There was modest reporting of measures to reduce risks of bias. The number of animals required to obtain 80% power with a significance level of 0.05 varied substantially across behavioural tests. In this comprehensive summary of the use of animal models of CIPN, we have identified areas in which the value of preclinical CIPN studies might be increased. Using both sexes of animals in the modelling of CIPN, ensuring that outcome measures align with those most relevant in the clinic, and the animal's pain contextualised ethology will likely improve external validity. Measures to reduce risk of bias should be employed to increase the internal validity of studies. Different outcome measures have different statistical power, and this can refine our approaches in the modelling of CIPN.

Identifying stroke therapeutics from preclinical models: A protocol for a novel application of network meta-analysis

Introduction: Globally, stroke is the second leading cause of death. Despite the burden of illness and death, few acute interventions are available to patients with ischemic stroke. Over 1,000 potential neuroprotective therapeutics have been evaluated in preclinical models. It is important to use robust evidence synthesis methods to appropriately assess which therapies should be translated to the clinical setting for evaluation in human studies. This protocol details planned methods to conduct a systematic review to identify and appraise eligible studies and to use a network meta-analysis to synthesize available evidence to answer the following questions: in preclinical in vivo models of focal ischemic stroke, what are the relative benefits of competing therapies tested in combination with the gold standard treatment alteplase in (i) reducing cerebral infarction size, and (ii) improving neurobehavioural outcomes? Methods: We will search Ovid Medline and Embase for articles on the effects of combination therapies with alteplase. Controlled comparison studies of preclinical in vivo models of experimentally induced focal ischemia testing the efficacy of therapies with alteplase versus alteplase alone will be identified. Outcomes to be extracted include infarct size (primary outcome) and neurobehavioural measures. Risk of bias and construct validity will be assessed using tools appropriate for preclinical studies. Here we describe steps undertaken to perform preclinical network meta-analysis to synthesise all evidence for each outcome and obtain a comprehensive ranking of all treatments. This will be a novel use of this evidence synthesis approach in stroke medicine to assess pre-clinical therapeutics. Combining all evidence to simultaneously compare mutliple therapuetics tested preclinically may provide a rationale for the clinical translation of therapeutics for patients with ischemic stroke.  Dissemination: Review findings will be submitted to a peer-reviewed journal and presented at relevant scientific meetings to promote knowledge transfer. Registration: PROSPERO number to be submitted following peer review.

Revision of the ARRIVE guidelines: rationale and scope

In 2010, the NC3Rs published the Animal Research: Reporting of In Vivo Experiments (ARRIVE) guidelines to improve the reporting of animal research. Despite considerable levels of support from the scientific community, the impact on the quality of reporting in animal research publications has been limited. This position paper highlights the strategy of an expert working group established to revise the guidelines and facilitate their uptake. The group's initial work will focus on three main areas: prioritisation of the ARRIVE items into a tiered system, development of an explanation and elaboration document, and revision of specific items.

Reproducibility of preclinical animal research improves with heterogeneity of study samples

Single-laboratory studies conducted under highly standardized conditions are the gold standard in preclinical animal research. Using simulations based on 440 preclinical studies across 13 different interventions in animal models of stroke, myocardial infarction, and breast cancer, we compared the accuracy of effect size estimates between single-laboratory and multi-laboratory study designs. Single-laboratory studies generally failed to predict effect size accurately, and larger sample sizes rendered effect size estimates even less accurate. By contrast, multi-laboratory designs including as few as 2 to 4 laboratories increased coverage probability by up to 42 percentage points without a need for larger sample sizes. These findings demonstrate that within-study standardization is a major cause of poor reproducibility. More representative study samples are required to improve the external validity and reproducibility of preclinical animal research and to prevent wasting animals and resources for inconclusive research.

Facilitating healthcare decisions by assessing the certainty in the evidence from preclinical animal studies

Laboratory animal studies are used in a wide range of human health related research areas, such as basic biomedical research, drug research, experimental surgery and environmental health. The results of these studies can be used to inform decisions regarding clinical research in humans, for example the decision to proceed to clinical trials. If the research question relates to potential harms with no expectation of benefit (e.g., toxicology), studies in experimental animals may provide the only relevant or controlled data and directly inform clinical management decisions. Systematic reviews and meta-analyses are important tools to provide robust and informative evidence summaries of these animal studies. Rating how certain we are about the evidence could provide important information about the translational probability of findings in experimental animal studies to clinical practice and probably improve it. Evidence summaries and certainty in the evidence ratings could also be used (1) to support selection of interventions with best therapeutic potential to be tested in clinical trials, (2) to justify a regulatory decision limiting human exposure (to drug or toxin), or to (3) support decisions on the utility of further animal experiments. The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach is the most widely used framework to rate the certainty in the evidence and strength of health care recommendations. Here we present how the GRADE approach could be used to rate the certainty in the evidence of preclinical animal studies in the context of therapeutic interventions. We also discuss the methodological challenges that we identified, and for which further work is needed. Examples are defining the importance of consistency within and across animal species and using GRADE's indirectness domain as a tool to predict translation from animal models to humans.

Identification and characterization of outcome measures reported in animal models of epilepsy: Protocol for a systematic review of the literature-A TASK2 report of the AES/ILAE Translational Task Force of the ILAE

Current antiseizure therapy is ineffective in approximately one third of people with epilepsy and is often associated with substantial side effects. In addition, most current therapeutic paradigms offer treatment, but not cure, and no therapies are able to modify the underlying disease, that is, can prevent or halt the process of epileptogenesis or alleviate the cognitive and psychiatric comorbidities. Preclinical research in the field of epilepsy has been extensive, but unfortunately, not all the animal models being used have been validated for their predictive value. The overall goal of TASK2 of the AES/ILAE Translational Task Force is to organize and coordinate systematic reviews on selected topics regarding animal research in epilepsy. Herein we describe our strategy. In the first part of the paper we provide an overview of the usefulness of systematic reviews and meta-analysis for preclinical research and explain the essentials for their conduct. Then we describe in detail the protocol for a first systematic review, which will focus on the identification and characterization of outcome measures reported in animal models of epilepsy. The specific goals of this study are to define systematically the phenotypic characteristics of the most commonly used animal models, and to effectively compare these with the manifestations of human epilepsy. This will provide epilepsy researchers with detailed information on the strengths and weaknesses of epilepsy models, facilitating their refinement and future research. Ultimately, this could lead to a refined use of relevant models for understanding the mechanism(s) of the epilepsies and developing novel therapies.

Letter to the editor - round table unites to tackle culture change in an effort to improve animal research reporting

A round table discussion was held during the LAVA-ESLAV-ECLAM conference on Reproducibility of Animal Studies on the 25th of September 2017 in Edinburgh. The aim of the round table was to discuss how to enhance the rate at which the quality of reporting animal research can be improved. This signed statement acknowledges the efforts that participant organizations have made towards improving the reporting of animal studies and confirms an ongoing commitment to drive further improvements, calling upon both academics and laboratory animal veterinarians to help make this cultural change.

The IMPROVE Guidelines (Ischaemia Models: Procedural Refinements Of in Vivo Experiments)

Most in vivo models of ischaemic stroke target the middle cerebral artery and a spectrum of stroke severities, from mild to substantial, can be achieved. This review describes opportunities to improve the in vivo modelling of ischaemic stroke and animal welfare. It provides a number of recommendations to minimise the level of severity in the most common rodent models of middle cerebral artery occlusion, while sustaining or improving the scientific outcomes. The recommendations cover basic requirements pre-surgery, selecting the most appropriate anaesthetic and analgesic regimen, as well as intraoperative and post-operative care. The aim is to provide support for researchers and animal care staff to refine their procedures and practices, and implement small incremental changes to improve the welfare of the animals used and to answer the scientific question under investigation. All recommendations are recapitulated in a summary poster (see supplementary information).

Optimization of large animal MI models; a systematic analysis of control groups from preclinical studies

Large animal models are essential for the development of novel therapeutics for myocardial infarction. To optimize translation, we need to assess the effect of experimental design on disease outcome and model experimental design to resemble the clinical course of MI. The aim of this study is therefore to systematically investigate how experimental decisions affect outcome measurements in large animal MI models. We used control animal-data from two independent meta-analyses of large animal MI models. All variables of interest were pre-defined. We performed univariable and multivariable meta-regression to analyze whether these variables influenced infarct size and ejection fraction. Our analyses incorporated 246 relevant studies. Multivariable meta-regression revealed that infarct size and cardiac function were influenced independently by choice of species, sex, co-medication, occlusion type, occluded vessel, quantification method, ischemia duration and follow-up duration. We provide strong systematic evidence that commonly used endpoints significantly depend on study design and biological variation. This makes direct comparison of different study-results difficult and calls for standardized models. Researchers should take this into account when designing large animal studies to most closely mimic the clinical course of MI and enable translational success.

Standardized mean differences cause funnel plot distortion in publication bias assessments

Meta-analyses are increasingly used for synthesis of evidence from biomedical research, and often include an assessment of publication bias based on visual or analytical detection of asymmetry in funnel plots. We studied the influence of different normalisation approaches, sample size and intervention effects on funnel plot asymmetry, using empirical datasets and illustrative simulations. We found that funnel plots of the Standardized Mean Difference (SMD) plotted against the standard error (SE) are susceptible to distortion, leading to overestimation of the existence and extent of publication bias. Distortion was more severe when the primary studies had a small sample size and when an intervention effect was present. We show that using the Normalised Mean Difference measure as effect size (when possible), or plotting the SMD against a sample size-based precision estimate, are more reliable alternatives. We conclude that funnel plots using the SMD in combination with the SE are unsuitable for publication bias assessments and can lead to false-positive results.

Improving our understanding of the in vivo modelling of psychotic disorders: A protocol for a systematic review and meta-analysis

Psychosis represents a set of symptoms against which current available treatments are not universally effective and are often accompanied by adverse side effects. Clinical management could potentially be improved with a greater understanding of the underlying biology and subsequently with the introduction of novel treatments. Since many clinical drug candidates are identified through in vivo modelling, a deeper understanding of the pre-clinical field, might help us understand why translation of results from animal models to inform mental health clinical practice has so far been weak. We set out to give a shallow, but broad unbiased overview of experiments looking at the in vivo modelling of psychotic disorders using a systematic review and meta-analysis. This protocol describes the exact methodology we propose to follow in order to quantitatively review both studies characterizing a model and those experiments that investigate the effects of novel therapeutic options. We are interested in assessing the prevalence of the reporting of measures to reduce risk of bias, and the internal and external validity of the animal models and outcome measures used to validate these models. This generation of strong empirical evidence has the potential to identify areas for improvement, make suggestions for future research avenues, and ultimately inform what we think we know to improve the current attrition rate between bench and bedside in psychosis research. A review like this will also support the reduction of animal numbers used in research and the refinement of experiments to maximize their value in informing the field.

Effects of MVA85A vaccine on tuberculosis challenge in animals: systematic review

Background: The existing Bacillus Calmette–Guérin (BCG) vaccination provides partial protection against tuberculosis (TB). The modified vaccinia ankara virus-expressing antigen 85A (MVA85A) aims to boost BCG immunity. We evaluated the animal evidence supporting the testing of MVA85A in humans.

Methods: Our protocol included in vivo preclinical studies of the MVA85A booster with BCG compared with BCG alone, followed by a TB challenge. We used standard methods for systematic review of animal studies, and summarized mortality, measures of pathology and lung bacterial load. The comprehensive literature search was to September 2014. Two independent investigators assessed eligibility and performed data extraction. We assessed study quality and pooled bacteria load using random effect meta-analysis.

Findings: We included eight studies in 192 animals. Three experiments were in mice, two in guinea pigs, two in macaques and one in calves. Overall, study quality was low with no randomization, baseline comparability not described and blinding not reported. For animal death (including euthanasia due to severe morbidity), studies were underpowered, and overall no benefit demonstrated. No difference was shown for lung pathology measured on an ordinal scale or bacterial load. The largest mortality trial carried out in macaques had more deaths in the MVA85A vaccine group, and was published after a trial in South Africa had started recruiting children.

Conclusions: This independent assessment of the animal data does not provide evidence to support efficacy of MVA85A as a BCG booster. More rigorous conduct and reporting of preclinical research are warranted, and we believe the results of studies should be publicly available before embarking on trials in humans, irrespective of the findings.

Systematic Review and Meta-Analysis of the Efficacy of Interleukin-1 Receptor Antagonist in Animal Models of Stroke: an Update

Interleukin-1 receptor antagonist (IL-1 RA) is an anti-inflammatory protein used clinically to treat rheumatoid arthritis and is considered a promising candidate therapy for stroke. Here, we sought to update the existing systematic review and meta-analysis of IL-1 RA in models of ischaemic stroke, published in 2009, to assess efficacy, the range of circumstances in which efficacy has been tested and whether the data appear to be confounded due to reported study quality and publication bias. We included 25 sources of data, 11 of which were additional to the original review. Overall, IL-1 RA reduced infarct volume by 36.2 % (95 % confidence interval 31.6-40.7, n = 76 comparisons from 1283 animals). Assessments for publication bias suggest 30 theoretically missing studies which reduce efficacy to 21.9 % (17.3-26.4). Efficacy was higher where IL-1 RA was administered directly into the ventricles rather than peripherally, and studies not reporting allocation concealment during the induction of ischaemia reported larger treatment effects. The preclinical data supporting IL-1 RA as a candidate therapy for ischaemic stroke have improved. The reporting of measures to reduce the risk of bias has improved substantially in this update, and studies now include the use of animals with relevant co-morbidities.

The Efficacy of Trastuzumab in Animal Models of Breast Cancer: A Systematic Review and Meta-Analysis

Background

Breast cancer is the most frequent cancers and is the second leading cause of cancer death among women. Trastuzumab is an effective treatment, the first monoclonal antibody directed against the human epidermal growth factor receptor 2 (HER2). To inform the development of other effective treatments we report summary estimates of efficacy of trastuzumab on survival and tumour volume in animal models of breast cancer.

Methods

We searched PubMed and EMBASE systematically to identify publications testing trastuzumab in animal models of breast cancer. Data describing tumour volume, median survival and animal features were extracted and we assessed quality using a 12-item checklist. We analysed the impact of study design and quality and evidence for publication bias.

Results

We included data from 83 studies reporting 169 experiments using 2076 mice. Trastuzumab treatment caused a substantial reduction in tumour growth, with tumours in treated animals growing to 32.6% of the volume of tumours in control animals (95%CI 27.8%-38.2%). Median survival was prolonged by a factor of 1.45 (1.30–1.62). Many study design and quality features accounted for between-study heterogeneity and we found evidence suggesting publication bias.

Conclusion

We have found trastuzumab to be effective in animal breast cancer models across a range of experimental circumstances. However the presence of publication bias and a low prevalence of measures to reduce bias provide a focus for future improvements in preclinical breast cancer research.

Abstract 55: The Natural Course of Myocardial Infarction in Large Animal Models; a Systematic Review and Meta-analysis

Background: Large animal models of myocardial infarction (MI) are essential for the development of novel therapeutic strategies. However, it is unclear which experimental factors are independent determinants of disease progression outcomes and the fidelity with which these models resemble MI in humans. The aim of this study was to investigate systematically what independently influences the outcome after MI in large animal models and determine which methodological factors influence primary endpoints and disease progression.

Methods: We used control animal data from two meta-analyses of large animal models of MI. We performed univariable and multivariable meta-regression to analyze if potential relevant variables were influencing outcomes; infarct size as a ratio of the area at risk (IS/AAR), as a ratio of the left ventricle (IS/LV) and ejection fraction (EF). Pre-defined independent variables were species, sex, weight, ischemia model (open vs closed), occlusion type (temporary vs permanent), occluded vessel, follow-up duration, use of co-medication, use of immunosuppression and study quality. Outcomes and variables were complemented per dataset if needed.

Results: Our analyses yielded 246 relevant studies, reporting 1500, 1221 and 775 control animals for IS/AAR, IS/LV and EF respectively. Multivariable meta-regression showed IS/AAR was influenced by species (p<0.001), sex (p=0.03), co-medication (p=0.01), occlusion type (p<0.001), occluded vessel (p=0.002) and follow-up (p=0.001). For IS/LV occlusion type (p=0.03), occluded vessel (p=0.03) and study quality (p=0.03) showed significant effects. EF measurements revealed that species (p=0.04), sex (p=0.04) and occluded vessel (p=0.05) were independent predictors. Using these variables, we can partially predict these outcomes for certain study setups.

Conclusion: Many methodological variations exist in the design of large animal MI studies. This should be taken into account when selecting a model to study therapy efficacy. We provide evidence that disease manifestation and progression greatly depend on certain biological characteristics, e.g. location of MI and sex of the patient. It is therefore possible that therapies have a different effect in specific patient populations.

From a mouse: systematic analysis reveals limitations of experiments testing interventions in Alzheimer's disease mouse models

The increasing prevalence of Alzheimer's disease (AD) poses a considerable socio‐economic challenge. Decades of experimental research have not led to the development of effective disease modifying interventions. A deeper understanding of in vivo research might provide insights to inform future in vivo research and clinical trial design. We therefore performed a systematic review and meta‐analysis of interventions tested in transgenic mouse models of AD. We searched electronically for publications testing interventions in transgenic models of AD. We extracted data for outcome, study characteristics and reported study quality and calculated summary estimates of efficacy using random effects meta‐analysis. We identified 427 publications describing 357 interventions in 55 transgenic models, involving 11,118 animals in 838 experiments. Of concern, reported study quality was relatively low; fewer than one in four publications reported the blinded assessment of outcome or random allocation to group and no study reported a sample size calculation. Additionally, there were few data for any individual intervention—only 16 interventions had outcomes described in 5 or more publications. Finally, “trim and fill” analyses suggested one in seven pathological and neurobehavioural experiments remain unpublished. Given these historical weaknesses in the in vivo modelling of AD in transgenic animals and the identified risks of bias, clinical trials that are based on claims of efficacy in animals should only proceed after a detailed critical appraisal of those animal data.

Olfactory Ensheathing Cell Transplantation in Experimental Spinal Cord Injury: Effect size and Reporting Bias of 62 Experimental Treatments: A Systematic Review and Meta-Analysis

Olfactory ensheathing cell (OEC) transplantation is a candidate cellular treatment approach for human spinal cord injury (SCI) due to their unique regenerative potential and autologous origin. The objective of this study was, through a meta-epidemiologic approach, (i) to assess the efficacy of OEC transplantation on locomotor recovery after traumatic experimental SCI and (ii) to estimate the likelihood of reporting bias and/or missing data. A study protocol was finalized before data collection. Embedded into a systematic review and meta-analysis, we conducted a literature research of databases including PubMed, EMBASE, and ISI Web of Science from 1949/01 to 2014/10 with no language restrictions, screened by two independent investigators. Studies were included if they assessed neurobehavioral improvement after traumatic experimental SCI, administrated no combined interventions, and reported the number of animals in the treatment and control group. Individual effect sizes were pooled using a random effects model. Details regarding the study design were extracted and impact of these on locomotor outcome was assessed by meta-regression. Missing data (reporting bias) was determined by Egger regression and Funnel-plotting. The primary study outcome assessed was improvement in locomotor function at the final time point of measurement. We included 49 studies (62 experiments, 1,164 animals) in the final analysis. The overall improvement in locomotor function after OEC transplantation, measured using the Basso, Beattie, and Bresnahan (BBB) score, was 20.3% (95% CI 17.8–29.5). One missing study was imputed by trim and fill analysis, suggesting only slight publication bias and reducing the overall effect to a 19.2% improvement of locomotor activity. Dose-response ratio supports neurobiological plausibility. Studies were assessed using a 9-point item quality score, resulting in a median score of 5 (interquartile range [IQR] 3–5). In conclusion, OEC transplantation exerts considerable beneficial effects on neurobehavioral recovery after traumatic experimental SCI. Publication bias was minimal and affirms the translational potential of efficacy, but safety cannot be adequately assessed. The data justify OECs as a cellular substrate to develop and optimize minimally invasive and safe cellular transplantation paradigms for the lesioned spinal cord embedded into state-of-the-art Phase I/II clinical trial design studies for human SCI.

This meta-analysis study examines the effects of transplanting olfactory ensheathing cells in rodents with experimental spinal cord injury, finding evidence for significant recovery and identifying aspects of the procedure that influence the effect size.

Spinal cord injury converts into a debilitating disease affecting millions of chronic patients worldwide. Despite increased molecular knowledge over the last decades, no causal pharmacological or cellular therapy has proven effective so far. Due to their unique regenerative capabilities and their autologous origin, olfactory ensheathing cells (OECs) constitute an appealing candidate for topical cell transplantation. In contrast to few and heterogeneous experimental reports of OEC transplantation after spinal cord injury in humans, a considerable number of preclinical studies have been conducted applying OEC transplantation in rodent models. We set out to conduct a systematic review and meta-analysis to assess preclinical efficacy of OEC transplantation. We detected a significant overall increase of functional neurological recovery in animals after OEC transplantation compared to the control group. This effect was not distorted by publication bias. We identified several specific hallmarks of the cell transplantation procedure that determine the effect size of the transplantation. Our findings delineate conditions for optimized OEC transplantation into lesioned spinal cords and its relevance for effective translation to human trials.

Cardiac Stem Cell Treatment in Myocardial Infarction: A Systematic Review and Meta-Analysis of Preclinical Studies

RATIONALE

Cardiac stem cells (CSC) therapy has been clinically introduced for cardiac repair after myocardial infarction (MI). To date, there has been no systematic overview and meta-analysis of studies using CSC therapy for MI.

OBJECTIVE

Here, we used meta-analysis to establish the overall effect of CSCs in preclinical studies and assessed translational differences between and within large and small animals in the CSC therapy field. In addition, we explored the effect of CSC type and other clinically relevant parameters on functional outcome to better predict and design future (pre)clinical studies using CSCs for MI.

METHODS AND RESULTS

A systematic search was performed, yielding 80 studies. We determined the overall effect of CSC therapy on left ventricular ejection fraction and performed meta-regression to investigate clinically relevant parameters. We also assessed the quality of included studies and possible bias. The overall effect observed in CSC-treated animals was 10.7% (95% confidence interval 9.4-12.1; P<0.001) improvement in ejection fraction compared with placebo controls. Interestingly, CSC therapy had a greater effect in small animals compared with large animals (P<0.001). Meta-regression indicated that cell type was a significant predictor for ejection fraction improvement in small animals. Minor publication bias was observed in small animal studies.

CONCLUSIONS

CSC treatment resulted in significant improvement of ejection fraction in preclinical animal models of MI compared with placebo. There was a reduction in the magnitude of effect in large compared with small animal models. Although different CSC types have overlapping culture characteristics, we observed a significant difference in their effect in post-MI animal studies.

The Missing Medians: Exclusion of Ordinal Data from Meta-Analyses

BACKGROUND

Meta-analyses are considered the gold standard of evidence-based health care, and are used to guide clinical decisions and health policy. A major limitation of current meta-analysis techniques is their inability to pool ordinal data. Our objectives were to determine the extent of this problem in the context of neurological rating scales and to provide a solution.

METHODS

Using an existing database of clinical trials of oral neuroprotective therapies, we identified the 6 most commonly used clinical rating scales and recorded how data from these scales were reported and analysed. We then identified systematic reviews of studies that used these scales (via the Cochrane database) and recorded the meta-analytic techniques used. Finally, we identified a statistical technique for calculating a common language effect size measure for ordinal data.

RESULTS

We identified 103 studies, with 128 instances of the 6 clinical scales being reported. The majority- 80%-reported means alone for central tendency, with only 13% reporting medians. In analysis, 40% of studies used parametric statistics alone, 34% of studies employed non-parametric analysis, and 26% did not include or specify analysis. Of the 60 systematic reviews identified that included meta-analysis, 88% used mean difference and 22% employed difference in proportions; none included rank-based analysis. We propose the use of a rank-based generalised odds ratio (WMW GenOR) as an assumption-free effect size measure that is easy to compute and can be readily combined in meta-analysis.

CONCLUSION

There is wide scope for improvement in the reporting and analysis of ordinal data in the literature. We hope that adoption of the WMW GenOR will have the dual effect of improving the reporting of data in individual studies while also increasing the inclusivity (and therefore validity) of meta-analyses.

Translational failure of anti-inflammatory compounds for myocardial infarction: a meta-analysis of large animal models

AIMS

Numerous anti-inflammatory drugs have been tested in large animal studies of myocardial infarction (MI). Despite positive results, translation of anti-inflammatory strategies into clinical practice has proved to be difficult. Critical disparities between preclinical and clinical study design that influence efficacy may partly be responsible for this translational failure. The aim of the present systematic review was to better understand which factors underlie the failure of transition towards the clinic.

METHODS AND RESULTS

Meta-analysis and regression of large animal studies were performed to identify sources that influenced effect size of anti-inflammatory compounds in large animal models of MI. We included 183 studies, containing 3331 large animals. Infarct size (IS) as a ratio of the area at risk (12.7%; 95% confidence interval, CI 11.1-14.4%, P < 0.001) and IS as a ratio of the left ventricle (3.9%; 95% CI 3.1-4.7%, P < 0.001) were reduced in treatment compared with control groups. Effect size was higher when outcome was assessed early after MI (P = 0.013) and where studies included only male animals (P < 0.001). Mortality in treated animals was higher in studies that blinded the investigator during the experiment (P = 0.041) and depended on the type of drug used (P < 0.001).

CONCLUSIONS

As expected, treatment with anti-inflammatory drugs leads to smaller infarct size in large animal MI models. Timing of outcome assessment, sex, and study quality are significantly associated with outcome and may explain part of the translational failure in clinical settings. Effect size depends on the type of drug used, enabling identification of compounds for future clinical testing.

Risk of Bias in Reports of In Vivo Research: A Focus for Improvement

The reliability of experimental findings depends on the rigour of experimental design. Here we show limited reporting of measures to reduce the risk of bias in a random sample of life sciences publications, significantly lower reporting of randomisation in work published in journals of high impact, and very limited reporting of measures to reduce the risk of bias in publications from leading United Kingdom institutions. Ascertainment of differences between institutions might serve both as a measure of research quality and as a tool for institutional efforts to improve research quality.

The usefulness of systematic reviews of animal experiments for the design of preclinical and clinical studies

The question of how animal studies should be designed, conducted, and analyzed remains underexposed in societal debates on animal experimentation. This is not only a scientific but also a moral question. After all, if animal experiments are not appropriately designed, conducted, and analyzed, the results produced are unlikely to be reliable and the animals have in effect been wasted. In this article, we focus on one particular method to address this moral question, namely systematic reviews of previously performed animal experiments. We discuss how the design, conduct, and analysis of future (animal and human) experiments may be optimized through such systematic reviews. In particular, we illustrate how these reviews can help improve the methodological quality of animal experiments, make the choice of an animal model and the translation of animal data to the clinic more evidence-based, and implement the 3Rs. Moreover, we discuss which measures are being taken and which need to be taken in the future to ensure that systematic reviews will actually contribute to optimizing experimental design and thereby to meeting a necessary condition for making the use of animals in these experiments justified.

The development of an online database for interventions tested in transgenic mouse models of Alzheimer's disease

Despite many efforts by the research community, Alzheimer's disease (AD) is still an incurable neurodegenerative condition that affects an estimated 44 million individuals worldwide and this figure is expected to increase to 135 million by the year 2050. As the research community currently reflects on previous endeavours, it is essential that we maximize the use of existing knowledge to inform future trials in the field. This article describes the development of a systematically identified data set relating to over 300 interventions tested in over 10,000 animals. The data set includes cohort‐level information for six structural outcomes and six behavioural assessments. We encourage others to use this dataset to inform the design of future animal experiments modelling AD and to promote effective translation to human health.

Drug repurposing: a systematic approach to evaluate candidate oral neuroprotective interventions for secondary progressive multiple sclerosis

OBJECTIVE

To develop and implement an evidence based framework to select, from drugs already licenced, candidate oral neuroprotective drugs to be tested in secondary progressive multiple sclerosis.

DESIGN

Systematic review of clinical studies of oral putative neuroprotective therapies in MS and four other neurodegenerative diseases with shared pathological features, followed by systematic review and meta-analyses of the in vivo experimental data for those interventions. We presented summary data to an international multi-disciplinary committee, which assessed each drug in turn using pre-specified criteria including consideration of mechanism of action.

RESULTS

We identified a short list of fifty-two candidate interventions. After review of all clinical and pre-clinical evidence we identified ibudilast, riluzole, amiloride, pirfenidone, fluoxetine, oxcarbazepine, and the polyunsaturated fatty-acid class (Linoleic Acid, Lipoic acid; Omega-3 fatty acid, Max EPA oil) as lead candidates for clinical evaluation.

CONCLUSIONS

We demonstrate a standardised and systematic approach to candidate identification for drug rescue and repurposing trials that can be applied widely to neurodegenerative disorders.