The place of experimental design and statistics in the 3Rs

Reporting and reproducibility: Proteomics of fish models in environmental toxicology and ecotoxicology

Environmental toxicology and ecotoxicology research efforts are employing proteomics with fish models as New Approach Methodologies, along with in silico, in vitro and other omics techniques to elucidate hazards of toxicants and toxins. We performed a critical review of toxicology studies with fish models using proteomics and reported fundamental parameters across experimental design, sample preparation, mass spectrometry, and bioinformatics of fish, which represent alternative vertebrate models in environmental toxicology, and routinely studied animals in ecotoxicology. We observed inconsistencies in reporting and methodologies among experimental designs, sample preparations, data acquisitions and bioinformatics, which can affect reproducibility of experimental results. We identified a distinct need to develop reporting guidelines for proteomics use in environmental toxicology and ecotoxicology, increased QA/QC throughout studies, and method optimization with an emphasis on reducing inconsistencies among studies. Several recommendations are offered as logical steps to advance development and application of this emerging research area to understand chemical hazards to public health and the environment.

Refinement of Animal Experiments: Replacing Traumatic Methods of Laboratory Animal Marking with Non-Invasive Alternatives

Reliable methods for identifying rodents play an important role in ensuring the success of preclinical studies. However, animal identification remains a trivial laboratory routine that is not often discussed, despite the fact that more than 6 million rodents are used in animal studies each year. Currently, there are extensive regulations in place to ensure adequate anesthesia and to reduce animal suffering during experiments. At the same time, not enough attention is paid to the comfort of rodents during routine identification procedures, which can be painful and cause some complications. In order to achieve the highest ethical standards in laboratory research, we must minimize animal discomfort during the identification phase. In this article, we discuss traumatic methods of identification and describe several painless methods for marking in long-term experimental studies. The use of non-traumatic and non-invasive methods requires the renewal of marks as they fade and additional handling of the rodents. Laboratory personnel must be trained in stress-minimizing handling techniques to make mark renewal less stressful.

Transparency in Non-Technical Project Summaries to Promote the Three Rs in Respiratory Disease Research

Non-Technical Project Summaries (NTS) are legal documents that were first introduced by the Directive 2010/63/EU to enhance transparency within scientific animal experimentation. Researchers intending to conduct biological research on animal models must fulfil the NTS requirements by outlining their proposed use of animals and how they plan to implement the Three Rs (replacement, reduction and refinement of animal use) in their experiments. This study outlines a novel systematic analysis approach that enables the assessment of NTS transparency based on the accuracy of reporting of certain Three Rs-specific information. This potentially customisable strategy could help toward the development of practical guidelines for use by Animal Welfare and Ethical Review Bodies (AWERBs) in establishments conducting animal research, in the process of scrutinising NTS during their pre-submission review of proposed licence applications. This could help to identify gaps in reporting of Three Rs-specific information relating to the planned animal experiments, which represents a remarkable step toward achieving greater openness in scientific communication. This study supports the concept that NTS transparency can promote the implementation of non-animal alternatives in fields where this is currently lacking, such as respiratory disease research. Although NTS were originally conceived as informative documents for a lay audience, we can conclude that data in NTS can be successfully used as a basis for systematic analysis. By reviewing the NTS, the experimental limitations of the currently available replacement strategies can also be highlighted, potentially pinpointing where there is a need for future method development.

Between two stools: preclinical research, reproducibility, and statistical design of experiments

Translation of animal-based preclinical research is hampered by poor validity and reproducibility issues. Unfortunately, preclinical research has ‘fallen between the stools’ of competing study design traditions. Preclinical studies are often characterised by small sample sizes, large variability, and ‘problem’ data. Although Fisher-type designs with randomisation and blocking are appropriate and have been vigorously promoted, structured statistically-based designs are almost unknown. Traditional analysis methods are commonly misapplied, and basic terminology and principles of inference testing misinterpreted. Problems are compounded by the lack of adequate statistical training for researchers, and failure of statistical educators to account for the unique demands of preclinical research. The solution is a return to the basics: statistical education tailored to non-statistician investigators, with clear communication of statistical concepts, and curricula that address design and data issues specific to preclinical research. Statistics curricula should focus on statistics as process: data sampling and study design before analysis and inference. Properly-designed and analysed experiments are a matter of ethics as much as procedure. Shifting the focus of statistical education from rote hypothesis testing to sound methodology will reduce the numbers of animals wasted in noninformative experiments and increase overall scientific quality and value of published research.

Imaging in experimental models of diabetes

Translational medicine, experimental medicine and experimental animal models, in particular mice and rats, represent a multidisciplinary field that has made it possible to achieve, in the last decades, important scientific progress. In this review, we have summarized the most frequently used imaging animal models, such as ultrasound (US), micro-CT, MRI and the optical imaging methods, and their main implications in diagnostic and therapeutic fields, with a particular focus on diabetes mellitus, a multifactorial disease extremely widespread among the general population.

Biological sex, not reproductive cycle, influences peripheral blood immune cell prevalence in mice

KEY POINTS

Traditionally the female sex, compared with the male sex, has been perceived as having greater variability in many physiological traits, including within the immune system. We investigated effects of biological sex and the female reproductive cycle on numbers of circulating leukocytes in C57BL/6J mice. We show that biological sex, but not female reproductive cyclicity, has a significant effect on peripheral blood immune cell prevalence and variability, and that sex differences were not consistent amongst common inbred laboratory mouse strains. We found that male C57BL/6J mice, compared with female mice, have greater variability in peripheral blood immunophenotype, and that this was influenced by body weight. We created summary tables for researchers to facilitate experiment planning and sample size calculations for peripheral immune cells that consider the effects of biological sex.

ABSTRACT

Immunophenotyping (i.e. quantifying the number and types of circulating leukocytes) is used to characterize immune changes during health and disease, and in response to pharmacological and other interventions. Despite the importance of biological sex in immune function, there is considerable uncertainty amongst researchers as to the extent to which biological sex or the female reproductive cycle influence blood immunophenotype. We quantified circulating leukocytes by multicolour flow cytometry in young C57BL/6J mice and assessed the effects of the reproductive cycle, biological sex, and other experimental and biological factors on data variability. We found that there are no significant effects of the female reproductive cycle on the prevalence of peripheral blood B cells, NK cells, CD4⁺ T cells, CD8⁺ T cells, monocytes, or neutrophils. Immunophenotype composition and variability do not significantly change between stages of the female reproductive cycle. There are, however, sex-specific differences in immune cell prevalence, with fewer monocytes, neutrophils, and NK cells in female mice. Surprisingly, immunophenotype is more variable in male mice, and weight is a significant contributing factor. We provide tools for researchers to perform a priori sample size calculations for two-group and factorial analyses. We show that immunophenotype varies between inbred mouse strains, and that using equal sample sizes of male and female mice is not always appropriate for within-sex evaluations of immune cell populations in peripheral blood.

Experimental Models as Refined Translational Tools for Breast Cancer Research

Breast cancer is one of the most common cancers worldwide, which makes it a very impactful malignancy in the society. Breast cancers can be classified through different systems based on the main tumor features and gene, protein, and cell receptors expression, which will determine the most advisable therapeutic course and expected outcomes. Multiple therapeutic options have already been proposed and implemented for breast cancer treatment. Nonetheless, their use and efficacy still greatly depend on the tumor classification, and treatments are commonly associated with invasiveness, pain, discomfort, severe side effects, and poor specificity. This has demanded an investment in the research of the mechanisms behind the disease progression, evolution, and associated risk factors, and on novel diagnostic and therapeutic techniques. However, advances in the understanding and assessment of breast cancer are dependent on the ability to mimic the properties and microenvironment of tumors in vivo, which can be achieved through experimentation on animal models. This review covers an overview of the main animal models used in breast cancer research, namely in vitro models, in vivo models, in silico models, and other models. For each model, the main characteristics, advantages, and challenges associated to their use are highlighted.

Using human in vitro transcriptome analysis to build trustworthy machine learning models for prediction of animal drug toxicity

During the development of new drugs or compounds there is a requirement for preclinical trials, commonly involving animal tests, to ascertain the safety of the compound prior to human trials. Machine learning techniques could provide an in-silico alternative to animal models for assessing drug toxicity, thus reducing expensive and invasive animal testing during clinical trials, for drugs that are most likely to fail safety tests. Here we present a machine learning model to predict kidney dysfunction, as a proxy for drug induced renal toxicity, in rats. To achieve this, we use inexpensive transcriptomic profiles derived from human cell lines after chemical compound treatment to train our models combined with compound chemical structure information. Genomics data due to its sparse, high-dimensional and noisy nature presents significant challenges in building trustworthy and transparent machine learning models. Here we address these issues by judiciously building feature sets from heterogenous sources and coupling them with measures of model uncertainty achieved through Gaussian Process based Bayesian models. We combine the use of insight into the feature-wise contributions to our predictions with the use of predictive uncertainties recovered from the Gaussian Process to improve the transparency and trustworthiness of the model.

Polygonogram and isobolographic analysis of interactions between various novel antiepileptic drugs in the 6-Hz corneal stimulation-induced seizure model in mice

Pharmacotherapy with two antiepileptic drugs in combination is usually prescribed to epilepsy patients with refractory seizures. The choice of antiepileptic drugs in combination should be based on synergistic cooperation of the drugs with respect to suppression of seizures. The selection of synergistic interactions between antiepileptic drugs is challenging issue for physicians, especially, if 25 antiepileptic drugs are currently available and approved to treat epilepsy patients. The aim of this study was to determine all possible interactions among 5 second-generation antiepileptic drugs (gabapentin (GBP), lacosamide (LCM), levetiracetam (LEV), pregabalin (PGB) and retigabine (RTG)) in the 6-Hz corneal stimulation-induced seizure model in adult male albino Swiss mice. The anticonvulsant effects of 10 various two-drug combinations of antiepileptic drugs were evaluated with type I isobolographic analysis associated with graphical presentation of polygonogram to visualize the types of interactions. Isobolographic analysis revealed that 7 two-drug combinations of LEV+RTG, LEV+LCM, GBP+RTG, PGB+LEV, GBP+LEV, PGB+RTG, PGB+LCM were synergistic in the 6-Hz corneal stimulation-induced seizure model in mice. The additive interaction was observed for the combinations of GBP+LCM, GBP+PGB, and RTG+LCM in this seizure model in mice. The most beneficial combination, offering the highest level of synergistic suppression of seizures in mice was that of LEV+RTG, whereas the most additive combination that protected the animals from seizures was that reporting additivity for RTG+LCM. The strength of interaction for two-drug combinations can be arranged from the synergistic to the additive, as follows: LEV+RTG > LEV+LCM > GBP+RTG > PGB+LEV > GBP+LEV > PGB+RTG > PGB+LCM > GBP+LCM > GBP+PGB > RTG+LCM.

A refinement to the formalin test in mice

The constant refinement of tests used in animal research is crucial for the scientific community. This is particularly true for the field of pain research, where ethical standards are notably sensitive. The formalin test is widely used in pain research and some of its mechanisms resemble those underlying clinical pain in humans. Immediately upon injection, formalin triggers two waves (an early and a late phase) of strong, nociceptive behaviour, characterised by licking, biting, lifting and shaking the injected paw of the animal. Although well characterised at the behaviour level, since its proposal over four decades ago, there has not been any significant refinement to the formalin test, especially those combining minimisation of animal distress and preservation of behavioural outcomes of the test.  Here, we propose a modified and improved method for the formalin test. We show that anaesthetising the animal with the inhalable anaesthetic sevoflurane at the time of the injection can produce reliable, robust and reproducible results whilst animal distress during the initial phase is reduced. Importantly, our results were validated by pharmacological suppression of the behaviour during the late phase of the test with gabapentin, the anaesthetic showing no interference with the drug. In addition, we demonstrate that this is also a useful method to screen for changes in pain behaviour in response to formalin in transgenic lines.

A refinement to the formalin test in mice

The constant refinement of tests used in animal research is crucial for the scientific community. This is particularly true for the field of pain research, where ethical standards are notably sensitive. The formalin test is widely used in pain research and some of its mechanisms resemble those underlying clinical pain in humans. Immediately upon injection, formalin triggers two waves (an early and a late phase) of strong, nociceptive behaviour, characterised by licking, biting, lifting and shaking the injected paw of the animal. Although well characterised at the behaviour level, since its proposal over four decades ago, there has not been any significant refinement to the formalin test, especially those combining minimisation of animal distress and preservation of behavioural outcomes of the test.  Here, we propose a modified and improved method for the formalin test. We show that anaesthetising the animal with the inhalable anaesthetic sevoflurane at the time of the injection can produce reliable, robust and reproducible results whilst animal distress during the initial phase is reduced. Importantly, our results were validated by pharmacological suppression of the behaviour during the late phase of the test with gabapentin, the anaesthetic showing no interference with the drug. In addition, we demonstrate that this is also a useful method to screen for changes in pain behaviour in response to formalin in transgenic lines.

Welfare and Scientific Considerations of Tattooing and Ear Tagging for Mouse Identification

Ear tagging is perceived as less painful or stressful than tattooing and therefore is generally considered less harmful or costly to welfare. However, ear tags are more difficult to read than tattoos and can fall out, and mice usually require restraint for the tag numbers to be read accurately. We assessed the welfare and scientific implications of tattooing by using a commercial device compared with restraint in a device versus ear tagging. Male and female BALB/c mice (n = 32) underwent procedures after 1 wk of tail or nonaversive (tunnel) handling to determine whether tunnel handling reduced anxiety. Pain was evaluated using both the Mouse Grimace Scale (MGS) and manual and automated behavior analyses; light-dark preference testing and voluntary interaction with the handler's hand were used to assess anxiety. Tail inflammation after tattooing was quantified using bioluminescent imaging, and ear tag and tattoo misidentification rates were estimated from volunteer staff records. Tunnel handling reduced anxiety compared with tail handling. According to the MGS, tattooing was not more painful than ear tagging but caused significant tail inflammation and more agitation and anxiety. However, all tattoos were read correctly without handling, whereas all ear tagged mice needed restraint, and at least 25% of the tag codes were misread. Handling stress together with identification errors at this rate represent potentially serious concerns regarding the scientific integrity of data from studies using ear tagging. These concerns are unlikely to arise with tattooing. Although tattooing was stressful, so were restraint and ear tagging. However, considering the other major advantages of tattooing, the total costs associated with tattooing were not substantially greater than for ear tagging.

The applied impact of 'naïve' statistical modelling of clustered observations of motion data in injury biomechanics research

OBJECTIVES

Appropriate statistical analysis of clustered data necessitates accounting for within-participant effects to ensure results are repeatable and translatable to real-world applications. This study aimed to compare statistical output and injury risk interpretation differences from two statistical regression models built from a clinical movement sidestepping database. A "naïve" regression model, which does not account for within-participant effects, was compared with an appropriately applied mixed effects model.

DESIGN

Comparative study.

METHODS

Three-dimensional unplanned sidestepping joint angle data (trunk, hip, and knee) from 35 males (112 observations) were used to model peak knee valgus moments and anterior cruciate ligament injury risk during the impact phase of stance. Both statistical models were cross-validated using a k-fold analysis.

RESULTS

The naïve regression returned inflated goodness of fit statistics (R²=0.50), which was evident following cross-validation (predicted R²=0.43). Following cross-validation, the mixed effects model (predicted R²=0.40) explained a similar amount of variance, despite containing three less predictors. The naïve model produced inaccurate parameter estimates, overestimating the effects of certain kinematic parameters by as much as 79 %.

CONCLUSIONS

A regression model naïvely applied to clustered observations of sidestepping data resulted in erroneous parameter estimates and goodness of fit statistics which have the potential to mislead future research and real-world applications. It is important for sport and clinical scientists to use statistically appropriate mixed effects models when modelling clustered motion capture data for injury biomechanics research to protect the translatability of the findings.

Researchers' attitudes to the 3Rs-An upturned hierarchy?

Animal use in biomedical research is generally justified by its potential benefits to the health of humans, or other animals, or the environment. However, ethical acceptability also requires scientists to limit harm to animals in their research. Training in laboratory animal science (LAS) helps scientists to do this by promoting best practice and the 3Rs. This study evaluated scientists' awareness and application of the 3Rs, and their approach to other ethical issues in animal research. It was based on an online survey of participants in LAS courses held in eight venues in four European countries: Portugal (Porto, Braga), Germany (Munich, Heidelberg), Switzerland (Basel, Lausanne, Zurich), and Denmark (Copenhagen). The survey questions were designed to assess general attitudes to animal use in biomedical research, Replacement alternatives, Reduction and Refinement conflicts, and harm-benefit analysis. The survey was conducted twice: immediately before the course ('BC', N = 310) and as a follow-up six months after the course ('AC', N = 127). While courses do appear to raise awareness of the 3Rs, they had no measurable effect on the existing low level of belief that animal experimentation can be fully replaced by non-animal methods. Most researchers acknowledged ethical issues with their work and reported that they discussed these with their peers. The level of an animal's welfare, and especially the prevention of pain, was regarded as the most pressing ethical issue, and as more important than the number of animals used or the use of animals as such. Refinement was considered more feasible than Replacement, as well as more urgent, and was also favoured over Reduction. Respondents in the survey reversed the 'hierarchy' of the 3Rs proposed by their architects, Russell and Burch, prioritizing Refinement over Reduction, and Reduction over Replacement. This ordering may conflict with the expectations of the public and regulators.

Learning process and improvement of point-of-care ultrasound technique for subxiphoid visualization of the inferior vena cava

BACKGROUND

Medical residents' training in ultrasonography usually follows the recommendations of the American College of Emergency Physicians (ACEP), even though these do not provide specific technical guidelines. Adequate training is considered to require 25 practical iterations in the majority of ultrasound procedures. However, the effectiveness of this approach has not been verified experimentally. We set out to determine the number of repetitions required for an acceptable ultrasound procedure of the inferior vena cava (IVC), as an important and emerging ultrasound procedure in cardiology.

METHODS

Using three human models, each of eight medical residents in the Emergency Medicine (EM) Program at the Universidad del Rosario performed 25 iterations of the recommended procedure, with image quality evaluation by an EM physician expert in the technique. Logistic regression analysis was used to determine the lowest number of repetitions required to achieve an adjusted probability of success of 80 and 90 %, respectively.

RESULTS

We obtained 200 ultrasound images. The percentage success by each resident ranged from 52 to 96 %. There was no statistical significance in the relation between gender and success (p = 0.83), but there was an association between year of residency and success (p < 0.001). The average time taken for each procedure was 17.3 s (SD 8.1); there was no association between the time taken and either repetition number or image quality. We demonstrate that eleven repetitions are required to achieve acceptable image quality in 80 %, and that 21 repetitions are required to achieve acceptable image quality in 90 %.

CONCLUSIONS

This is the first study to formally evaluate the effectiveness of recommended training in ultrasound techniques. Our findings demonstrate that training comprising 25 procedural repetitions is easily sufficient to achieve optimal image quality, and they also provide empiric knowledge toward elucidating the times and minimum repetitions needed to acquire and improve ultrasonographic technique in novice operators to a level which fulfills quality requirements for interpretation.

Longitudinal imaging of the ageing mouse

Several non-invasive imaging techniques are used to investigate the effect of pathologies and treatments over time in mouse models. Each preclinical in vivo technique provides longitudinal and quantitative measurements of changes in tissues and organs, which are fundamental for the evaluation of alterations in phenotype due to pathologies, interventions and treatments. However, it is still unclear how these imaging modalities can be used to study ageing with mice models. Almost all age related pathologies in mice such as osteoporosis, arthritis, diabetes, cancer, thrombi, dementia, to name a few, can be imaged in vivo by at least one longitudinal imaging modality. These measurements are the basis for quantification of treatment effects in the development phase of a novel treatment prior to its clinical testing. Furthermore, the non-invasive nature of such investigations allows the assessment of different tissue and organ phenotypes in the same animal and over time, providing the opportunity to study the dysfunction of multiple tissues associated with the ageing process. This review paper aims to provide an overview of the applications of the most commonly used in vivo imaging modalities used in mouse studies: micro-computed-tomography, preclinical magnetic-resonance-imaging, preclinical positron-emission-tomography, preclinical single photon emission computed tomography, ultrasound, intravital microscopy, and whole body optical imaging.

Analyzing Clustered Data: Why and How to Account for Multiple Observations Nested within a Study Participant?

A conventional study design among medical and biological experimentalists involves collecting multiple measurements from a study subject. For example, experiments utilizing mouse models in neuroscience often involve collecting multiple neuron measurements per mouse to increase the number of observations without requiring a large number of mice. This leads to a form of statistical dependence referred to as clustering. Inappropriate analyses of clustered data have resulted in several recent critiques of neuroscience research that suggest the bar for statistical analyses within the field is set too low. We compare naïve analytical approaches to marginal, fixed-effect, and mixed-effect models and provide guidelines for when each of these models is most appropriate based on study design. We demonstrate the influence of clustering on a between-mouse treatment effect, a within-mouse treatment effect, and an interaction effect between the two. Our analyses demonstrate that these statistical approaches can give substantially different results, primarily when the analyses include a between-mouse treatment effect. In a novel analysis from a neuroscience perspective, we also refine the mixed-effect approach through the inclusion of an aggregate mouse-level counterpart to a within-mouse (neuron level) treatment as an additional predictor by adapting an advanced modeling technique that has been used in social science research and show that this yields more informative results. Based on these findings, we emphasize the importance of appropriate analyses of clustered data, and we aim for this work to serve as a resource for when one is deciding which approach will work best for a given study.

Interim analysis: A rational approach of decision making in clinical trial

Interim analysis of especially sizeable trials keeps the decision process free of conflict of interest while considering cost, resources, and meaningfulness of the project. Whenever necessary, such interim analysis can also call for potential termination or appropriate modification in sample size, study design, and even an early declaration of success. Given the extraordinary size and complexity today, this rational approach helps to analyze and predict the outcomes of a clinical trial that incorporate what is learned during the course of a study or a clinical development program. Such approach can also fill the gap by directing the resources toward relevant and optimized clinical trials between unmet medical needs and interventions being tested currently rather than fulfilling only business and profit goals.

The design and statistical analysis of animal experiments: introduction to this issue

Animal research has made major contributions to the health and welfare of humans and domestic animals. Immunization, first developed against rabies and anthrax by Pasteur using dogs, sheep, and rabbits, is now used to control many infectious diseases. The first drug, Salvarsan, was developed by Ehrlich using rabbits infected with the organism causing syphilis. This was the forerunner of the many drugs developed by the pharmaceutical industry today. The discovery of vitamins using rats has almost eliminated diseases such as scurvy and rickets; and hormones, such as insulin discovered following work in dogs, have helped to control many metabolic diseases. These and many other advances have enabled physicians to treat a wide range of human diseases. But many diseases continue to cause suffering.

Mammalian Genotyping Using Acoustic Droplet Ejection for Enhanced Data Reproducibility, Superior Throughput, and Minimized Cross-Contamination

Genetically engineered animal models are major tools of a drug discovery pipeline because they facilitate understanding of the molecular and biochemical basis of disease. These highly complex models of human disease often require increasingly convoluted genetic analysis. With growing needs for throughput and consistency, we find that traditional aspiration-and-dispense liquid-handling robots no longer have the required speed, quality, or reproducibility.We present an adaptation and installation of an acoustic droplet ejection (ADE) liquid-handling system for ultra-high-throughput screening of genetically engineered models. An ADE system is fully integrated with existing laboratory processes and platforms to facilitate execution of PCR and quantitative PCR (qPCR) reactions. Such a configuration permits interrogation of highly complex genetic models in a variety of backgrounds. Our findings demonstrate that a single ADE system replaces 8-10 traditional liquid-handling robots while increasing quality and reproducibility.We demonstrate significant improvements achieved by transitioning to an ADE device: extremely low detectable cross-contamination in PCR and qPCR despite extensive use, greatly increased data reproducibility (large increases in data quality and Cq consistency), lowered reaction volumes for large cost savings, and nearly a magnitude increase in speed per instrument. We show several comparisons between traditional- and ADE-based pipetting for a qPCR-based workflow.

Experimental design and analysis and their reporting: new guidance for publication in BJP

Linked Editorials

This Editorial is part of a series. To view the other Editorials in this series, visit: http://onlinelibrary.wiley.com/doi/10.1111/bph.12956/abstract; http://onlinelibrary.wiley.com/doi/10.1111/bph.12954/abstract; http://onlinelibrary.wiley.com/doi/10.1111/bph.12955/abstract and http://onlinelibrary.wiley.com/doi/10.1111/bph.13112/abstract