Designing, conducting, and reporting reproducible animal experiments

In biomedicine and many other fields, there are growing concerns around the reproducibility of research findings, with many researchers being unable to replicate their own or others' results. This raises important questions as to the validity and usefulness of much published research. In this review, we aim to engage researchers in the issue of research reproducibility and equip them with the necessary tools to increase the reproducibility of their research. We first highlight the causes and potential impact of non-reproducible research and emphasise the benefits of working reproducibly for the researcher and broader research community. We address specific targets for improvement and steps that individual researchers can take to increase the reproducibility of their work. We next provide recommendations for improving the design and conduct of experiments, focusing on in vivo animal experiments. We describe common sources of poor internal validity of experiments and offer practical guidance for limiting these potential sources of bias at different experimental stages, as well as discussing other important considerations during experimental design. We provide a list of key resources available to researchers to improve experimental design, conduct, and reporting. We then discuss the importance of open research practices such as study preregistration and the use of preprints and describe recommendations around data management and sharing. Our review emphasises the importance of reproducible work and aims to empower every individual researcher to contribute to the reproducibility of research in their field.

Animal models of chemotherapy-induced peripheral neuropathy: A machine-assisted systematic review and meta-analysis

We report a systematic review and meta-analysis of research using animal models of chemotherapy-induced peripheral neuropathy (CIPN). We systematically searched 5 online databases in September 2012 and updated the search in November 2015 using machine learning and text mining to reduce the screening for inclusion workload and improve accuracy. For each comparison, we calculated a standardised mean difference (SMD) effect size, and then combined effects in a random-effects meta-analysis. We assessed the impact of study design factors and reporting of measures to reduce risks of bias. We present power analyses for the most frequently reported behavioural tests; 337 publications were included. Most studies (84%) used male animals only. The most frequently reported outcome measure was evoked limb withdrawal in response to mechanical monofilaments. There was modest reporting of measures to reduce risks of bias. The number of animals required to obtain 80% power with a significance level of 0.05 varied substantially across behavioural tests. In this comprehensive summary of the use of animal models of CIPN, we have identified areas in which the value of preclinical CIPN studies might be increased. Using both sexes of animals in the modelling of CIPN, ensuring that outcome measures align with those most relevant in the clinic, and the animal's pain contextualised ethology will likely improve external validity. Measures to reduce risk of bias should be employed to increase the internal validity of studies. Different outcome measures have different statistical power, and this can refine our approaches in the modelling of CIPN.

Cardiac Stem Cell Treatment in Myocardial Infarction: A Systematic Review and Meta-Analysis of Preclinical Studies

RATIONALE

Cardiac stem cells (CSC) therapy has been clinically introduced for cardiac repair after myocardial infarction (MI). To date, there has been no systematic overview and meta-analysis of studies using CSC therapy for MI.

OBJECTIVE

Here, we used meta-analysis to establish the overall effect of CSCs in preclinical studies and assessed translational differences between and within large and small animals in the CSC therapy field. In addition, we explored the effect of CSC type and other clinically relevant parameters on functional outcome to better predict and design future (pre)clinical studies using CSCs for MI.

METHODS AND RESULTS

A systematic search was performed, yielding 80 studies. We determined the overall effect of CSC therapy on left ventricular ejection fraction and performed meta-regression to investigate clinically relevant parameters. We also assessed the quality of included studies and possible bias. The overall effect observed in CSC-treated animals was 10.7% (95% confidence interval 9.4-12.1; P<0.001) improvement in ejection fraction compared with placebo controls. Interestingly, CSC therapy had a greater effect in small animals compared with large animals (P<0.001). Meta-regression indicated that cell type was a significant predictor for ejection fraction improvement in small animals. Minor publication bias was observed in small animal studies.

CONCLUSIONS

CSC treatment resulted in significant improvement of ejection fraction in preclinical animal models of MI compared with placebo. There was a reduction in the magnitude of effect in large compared with small animal models. Although different CSC types have overlapping culture characteristics, we observed a significant difference in their effect in post-MI animal studies.

Risk of Bias in Reports of In Vivo Research: A Focus for Improvement

The reliability of experimental findings depends on the rigour of experimental design. Here we show limited reporting of measures to reduce the risk of bias in a random sample of life sciences publications, significantly lower reporting of randomisation in work published in journals of high impact, and very limited reporting of measures to reduce the risk of bias in publications from leading United Kingdom institutions. Ascertainment of differences between institutions might serve both as a measure of research quality and as a tool for institutional efforts to improve research quality.

Efficacy of antidepressants in animal models of ischemic stroke: a systematic review and meta-analysis

BACKGROUND AND PURPOSE

Poststroke depression is a prevalent complication of stroke with unclear pathogenesis. The benefits of antidepressants in this context and their effects on stroke recovery other than effects on mood are not clearly defined, with some studies suggesting efficacy in improving functional outcome in both depressed and nondepressed stroke patients. We have analyzed the preclinical animal data on antidepressant treatment in focal cerebral ischemia, modeled±depression, to help inform clinical trial design.

METHODS

We performed a systematic review and meta-analysis of data from experiments testing the efficacy of antidepressants versus no treatment to reduce infarct volume or improve neurobehavioral or neurogenesis outcomes in animal models of stroke. We used random-effects metaregression to test the impact of study quality and design characteristics and used trim and fill to assess publication bias.

RESULTS

We identified 44 publications describing the effects of 22 antidepressant drugs. The median quality checklist score was 5 of a possible 10 (interquartile range, 4-7). Overall, antidepressants reduced infarct volume by 27.3% (95% confidence interval, 20.7%-33.8%) and improved neurobehavioral outcomes by 53.7% (46.4%-61.1%). There was little evidence for an effect of selective serotonin reuptake inhibitors on infarct volume. For neurobehavioral outcomes there was evidence of publication bias. Selective serotonin reuptake inhibitors were the most frequently studied antidepressant subtype and improved neurobehavioral outcome by 51.8% (38.6%-64.9%) and increased neurogenesis by 2.2 SD (1.3-3.0).

CONCLUSIONS

In line with current clinical data and despite some limitations, antidepressant treatments seem to improve infarct volume and neurobehavioral outcome in animal models of ischemic stroke.

Systematic reviews and meta-analysis of preclinical studies: why perform them and how to appraise them critically

The use of systematic review and meta-analysis of preclinical studies has become more common, including those of studies describing the modeling of cerebrovascular diseases. Empirical evidence suggests that too many preclinical experiments lack methodological rigor, and this leads to inflated treatment effects. The aim of this review is to describe the concepts of systematic review and meta-analysis and consider how these tools may be used to provide empirical evidence to spur the field to improve the rigor of the conduct and reporting of preclinical research akin to their use in improving the conduct and reporting of randomized controlled trials in clinical research. As with other research domains, systematic reviews are subject to bias. Therefore, we have also suggested guidance for their conduct, reporting, and critical appraisal.

Meta-analysis of data from animal studies: a practical guide

Meta-analyses of data from human studies are invaluable resources in the life sciences and the methods to conduct these are well documented. Similarly there are a number of benefits in conducting meta-analyses on data from animal studies; they can be used to inform clinical trial design, or to try and explain discrepancies between preclinical and clinical trial results. However there are inherit differences between animal and human studies and so applying the same techniques for the meta-analysis of preclinical data is not straightforward. For example preclinical studies are frequently small and there is often substantial heterogeneity between studies. This may have an impact on both the method of calculating an effect size and the method of pooling data. Here we describe a practical guide for the meta-analysis of data from animal studies including methods used to explore sources of heterogeneity.

Systematic review and stratified meta-analysis of the efficacy of RhoA and Rho kinase inhibitors in animal models of ischaemic stroke

BACKGROUND

There is currently only one clinically approved drug, tissue plasminogen activator (tPA), for the treatment of acute ischaemic stroke. The RhoA pathway, including RhoA and its downstream effector Rho kinase (ROCK), has been identified as a possible therapeutic target. Our aim was to assess the impact of study design characteristics and study quality on reported measures of efficacy and to assess for the presence and impact of publication bias.

METHODS

We conducted a systematic review and meta-analysis on publications describing the efficacy of RhoA and ROCK inhibitors in animal models of focal cerebral ischaemia where outcome was assessed as a change in lesion size or neurobehavioural score, or both.

RESULTS

We identified 25 published papers which met our inclusion criteria. RhoA and ROCK inhibitors reduced lesion size by 37.3% in models of focal cerebral ischaemia (95% CI, 28.6% to 46.0%, 41 comparisons), and reduced neurobehavioural data by 40.5% (33.4% to 47.7%, 30 comparisons). Overall study quality was low (median=4, interquartile range 3-5) and measures to reduce bias were seldom reported. Publication bias was prevalent and associated with a substantial overstatement of efficacy for lesion size.

CONCLUSIONS

RhoA and ROCK inhibitors appear to be effective in animal models of stroke. However the low quality score, publication bias and limited number of studies are areas which need attention prior to conducting clinical trials.

Transparency in the reporting of in vivo pre-clinical pain research: The relevance and implications of the ARRIVE (Animal Research: Reporting In Vivo Experiments) guidelines

Clear reporting of research is crucial to the scientific process. Poorly designed and reported studies are damaging not only to the efforts of individual researchers, but also to science as a whole. Standardised reporting methods, such as those already established for reporting randomised clinical trials, have led to improved study design and facilitated the processes of clinical systematic review and meta-analysis.

Such standards were lacking in the pre-clinical field until the development of the ARRIVE (Animal Research: Reporting In Vivo Experiments) guidelines. These were prompted following a survey which highlighted a widespread lack of robust and consistent reporting of pre-clinical in vivo research, with reports frequently omitting basic information required for study replication and quality assessment.

The resulting twenty item checklist in ARRIVE covers all aspects of experimental design with particular emphasis on bias reduction and methodological transparency. Influential publishers and research funders have already adopted ARRIVE. Further dissemination and acknowledgement of the importance of these guidelines is vital to their widespread implementation.

Conclusions and implications

Wide implementation of the ARRIVE guidelines for reporting of in vivo preclinical research, especially pain research, are essential for a much needed increased transparency and quality in publishing such research. ARRIVE will also positively influence improvements in experimental design and quality, assist the conduct of accurate replication studies of important new findings and facilitate meta-analyses of preclinical research.

Animal models of bone cancer pain: systematic review and meta-analyses

Pain can significantly decrease the quality of life of patients with advanced cancer. Current treatment strategies often provide inadequate analgesia and unacceptable side effects. Animal models of bone cancer pain are used in the development of novel pharmacological approaches. Here we conducted a systematic review and meta-analysis of publications describing in vivo modelling of bone cancer pain in which behavioural, general health, macroscopic, histological, biochemical, or electrophysiological outcomes were reported and compared to appropriate controls. In all, 150 publications met our inclusion criteria, describing 38 different models of bone cancer pain. Reported methodological quality was low; only 31% of publications reported blinded assessment of outcome, and 11% reported random allocation to group. No publication reported a sample size calculation. Studies that reported measures to reduce bias reported smaller differences in behavioural outcomes between tumour-bearing and control animals, and studies that presented a statement regarding a conflict of interest reported larger differences in behavioural outcomes. Larger differences in behavioural outcomes were reported in female animals, when cancer cells were injected into either the tibia or femur, and when MatLyLu prostate or Lewis Lung cancer cells were used. Mechanical-evoked pain behaviours were most commonly reported; however, the largest difference was observed in spontaneous pain behaviours. In the spinal cord astrocyte activation and increased levels of Substance P receptor internalisation, c-Fos, dynorphin, tumor necrosis factor-α and interleukin-1β have been reported in bone cancer pain models, suggesting several potential therapeutic targets. However, the translational impact of animal models on clinical pain research could be enhanced by improving methodological quality.

Dysregulation of glucocorticoid metabolism in murine obesity: comparable effects of leptin resistance and deficiency

In obese humans, metabolism of glucocorticoids by 11 beta-hydroxysteroid dehydrogenase type 1 (11 beta-HSD1) and A-ring reduction (by 5 alpha- and 5 beta-reductases) is dysregulated in a tissue specific manner. These changes have been recapitulated in leptin resistant obese Zucker rats but were not observed in high-fat fed Wistar rats. Recent data from mouse models suggest that such discrepancies may reflect differences in leptin signalling. We therefore compared glucocorticoid metabolism in murine models of leptin deficiency and resistance. Male ob/ob and db/db mice and their respective littermate controls (n=10-12/group) were studied at the age of 12 weeks. Enzyme activities and mRNA expression were quantified in snap-frozen tissues. The patterns of altered pathways of steroid metabolism in obesity were similar in ob/ob and db/db mice. In liver, 5 beta-reductase activity and mRNA were increased and 11 beta-HSD1 decreased in obese mice, whereas 5 alpha-reductase 1 (5 alpha R1) mRNA was not altered. In visceral adipose depots, 5 beta-reductase was not expressed, 11 beta-HSD1 activity was increased and 5 alpha R1 mRNA was not altered in obesity. By contrast, in subcutaneous adipose tissue 11 beta-HSD1 and 5 alpha R1 mRNA were decreased. Systematic differences were not found between ob/ob and db/db murine models of obesity, suggesting that variations in leptin signalling through the short splice variant of the Ob receptor do not contribute to dysregulation of glucocorticoid metabolism.