The methodological quality of animal research in critical care: the public face of science

Intraperitoneal drug delivery systems releasing cytostatic agents to target gastro-intestinal peritoneal metastases in laboratory animals: a systematic review

For peritoneal metastases (PM), there are few curative treatment options, and they are only available for a select patient group. Recently, new therapies have been developed to deliver intraperitoneal chemotherapy for a prolonged period, suitable for a larger patient group. These drug delivery systems (DDSs) seem promising in the experimental setting. Many types of DDSs have been explored in a variety of animal models, using different cytostatics. This review aimed to provide an overview of animal studies using DDSs containing cytostatics for the treatment of gastro-intestinal PM and identify the most promising therapeutic combinations. The review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and Systematic Review Center for Laboratory Animal Experimentation (SYRCLE) guidelines. The 35 studies included revealed similar results: using a cytostatic-loaded DDS to treat PM resulted in a higher median survival time (MST) and a lower intraperitoneal tumor load compared to no treatment or treatment with a ‘free’ cytostatic or an unloaded DDS. In 65% of the studies, the MST was significantly longer and in 24% the tumor load was significantly lower in the animals treated with cytostatic-loaded DDS. The large variety of experimental setups made it impossible to identify the most promising DDS-cytostatic combination. In most studies, the risk of bias was unclear due to poor reporting. Future studies should focus more on improving the clinical relevance of the experiments, standardizing the experimental study setup, and improving their methodological quality and reporting.

Beneficial effects of citrulline enteral administration on sepsis-induced T cell mitochondrial dysfunction

Since sepsis induces a sustained immunosuppression responsible for secondary infections acquisition and late mortality, restoring immune function would result in a better outcome. Given the role of arginine deficiency in T cell dysfunction, the evaluation of restoring arginine availability in sepsis has to be explored. Using an animal model of sepsis, we demonstrated that increasing arginine availability enhanced mitochondrial T cell function and decreased sepsis-induced immunosuppression.

Severe sepsis induces a sustained immune dysfunction associated with poor clinical behavior. In particular, lymphopenia along with increased lymphocyte apoptosis and decreased lymphocyte proliferation, enhanced circulating regulatory T cells (Treg), and the emergence of myeloid-derived suppressor cells (MDSCs) have all been associated with persistent organ dysfunction, secondary infections, and late mortality. The mechanisms involved in MDSC-mediated T cell dysfunction during sepsis share some features with those described in malignancies such as arginine deprivation. We hypothesized that increasing arginine availability would restore T cell function and decrease sepsis-induced immunosuppression. Using a mouse model of sepsis based on cecal ligation and puncture and secondary pneumonia triggered by methicillin-resistant Staphylococcus aureus inoculation, we demonstrated that citrulline administration was more efficient than arginine in increasing arginine plasma levels and restoring T cell mitochondrial function and proliferation while reducing sepsis-induced Treg and MDSC expansion. Because there is no specific therapeutic strategy to restore immune function after sepsis, we believe that our study provides evidence for developing citrulline-based clinical studies in sepsis.

A Study to Assess the Quality of Reporting of Animal Research Studies Published in PubMed Indexed Journals: A Retrospective, Cross-Sectional Content Analysis

Objectives: A complete and concise pre-clinical experimental research gives detailed information about the disease-specific model, prevents duplication, and saves animal life, money, as well as time. It will also allow readers to effectively interpret and evaluate the work and ensure that others can replicate the experiments described. The present study was conducted to assess the adequacy of animal details provided in published experimental animal studies.

Methods: All in vivo studies published as full-text articles in two PubMed indexed journals (one Indian and one international) from January 2011 to December 2019 and satisfying the inclusion norms were included. A checklist consisting of 27 discrete items subdivided under three domains, viz. animal details, disease model, and guidelines, was used. Every article was assessed by two investigators independently for determining the reporting quality.

Results: One hundred and seventy-seven studies satisfied the inclusion criteria. Age or age range was reported in 20.34% of the articles in the Indian journal and 5.88% articles in the international journal (p=0.019). Housing and husbandry details were reported in all the articles published in the international journal and 82.7% of the articles in the Indian journal (p=0.001). The disease/pathology studied was given in 70.62% of articles published in the Indian journal and 86.27% of articles published in the international journal (p=0.029). None of the studies provided details of genetic modification, health status, sample size calculation, steps taken to minimize bias, and implementation of randomization.

Conclusion: There is a need for optimal reporting of certain relevant animal details, disease models, experimental procedures, sample size calculation, and adherence to guidelines by the researchers for which the reporting was found to be sub-par to improve reproducibility and validity of animal research.

Preclinical Research Reporting in Shock: Room for Improvement

ABSTRACT

The ARRIVE (Animals in Research: Reporting In Vivo Experiments) guidelines were endorsed by the Shock Society in 2012, but to date there has been no systematic evaluation of research reporting quality for Shock. We systematically assessed 100 randomly selected animal-based research articles published between 2014 and 2018 for reporting quality and statistical practice, compared with 40 pre-ARRIVE studies. More than half of surveyed papers omitted verifiable ethical oversight information and basic animal descriptive information. Few papers reported best-practice methods, such as sample size justification (10%), randomization (43%), randomization method (7%), blinding (23%). Only one paper reported effect sizes to interpret study results. Most troubling was inadequate reporting of welfare-related information (anesthesia, analgesia, humane endpoints, euthanasia). Almost a decade after ARRIVE endorsement, our findings show that reporting deficiencies have persisted with little sign of correction. There is a clear need for investigators to increase transparency of research methods reporting, and drastically improve skills in experimental design. Improvement in standards and greater attention paid to reporting will lead to improvement in reproducibility, replicability, and research quality. It is incumbent upon the research community to improve reporting practices; accurate and transparent reporting is integral to producing rigorous and ethical science.

Inadequate awareness of adherence to ARRIVE guidelines, regarding reporting quality of hernia models repaired with meshes: a systematic review

PURPOSE

Research papers involving animal studies often display poor reporting standards, leading to lower study reproducibility. We aim to determine the difference in reporting animal studies regarding abdominal wall hernia repair with mesh placement, before and after the publication of ARRIVE-2010 (Animal Research: Reporting of In Vivo Experiments) guidelines. Furthermore, we aim to present the most up-to-date reporting quality using the updated ARRIVE-2020 as criteria.

METHODS

All animal studies concerning hernia repair with meshes were systematically searched. Articles published in the 5 years leading up to the ARRIVE-2010 (pre-ARRIVE) and articles within the last 5 years until the updated ARRIVE 2.0 (post-ARRIVE) were compared for overall species and specific species separately. Articles published last year were evaluated for presenting fully reported (sub)items.

RESULTS

The number of fully reported (sub)items per article was on average significantly higher for pre-ARRIVE than post-ARRIVE for overall species (mean (SD) = 14.0 (2.8) vs. 12.6 (2.5), P < 0.001). The same applies to rabbit (mean (SD) = 14.8 (2.6) vs. 12.6 (2.6), P = 0.001) and pig studies (mean (SD) = 14.5 (2.7) vs. 11.6 (2.6), P = 0.004), with no significance in rat studies (mean (SD) = 13.6 (2.9) vs. 12.9 (2.3), P = 0.076). Significance was found in several (sub)items between pre-ARRIVE and post-ARRIVE (n = 7, 3, 8, and 3 for overall species, rat, rabbit, and pig studies, respectively).

CONCLUSION

General reporting quality of animal experiments has been improved markedly by ARRIVE guidelines. However, more improvements are required considering the arrival of ARRIVE 2.0 guidelines.

Gap Analysis of Swine-Based Hemostasis Research: "Houses of Brick or Mansions of Straw?"

INTRODUCTION

Hemorrhage control is the top priority in far-forward care. Preclinical studies are essential for determining safety and efficacy before novel therapeutics can be tested in humans. Unfortunately, poor methodological quality jeopardizes translational potential.

METHODS

We systematically reviewed 136 recent publications describing swine models of hemostasis and hemorrhage reduction to assess compliance with established standards for scientific reporting. Quality measures were summarized by descriptive statistics; randomization was assessed by using baseline group differences to test the uniform distribution assumption for observed P-values.

RESULTS

Most articles did not report information essential to assess study validity and reliability of experimental results. Studies claiming random allocation showed clear evidence of systematic bias. Sample sizes were small, but nearly all studies reported statistically significant effects in the direction of "benefit." Excessive hypothesis testing increased the risk of false positives.

CONCLUSIONS

Methodological quality was poor. Although funding agencies actively promote good scientific practice, investigators have been slow to comply. Poorly executed and reported animal research is an ethical and translational issue, wasting animals and potentially harming patients. To properly assess the therapeutic benefit of novel interventions, investigators must rely less on rote hypothesis testing, develop skills in experimental design and quantitative analysis, and comply with best-practice reporting guidelines.

Premise for Standardized Sepsis Models

Sepsis morbidity and mortality exacts a toll on patients and contributes significantly to healthcare costs. Preclinical models of sepsis have been used to study disease pathogenesis and test new therapies, but divergent outcomes have been observed with the same treatment even when using the same sepsis model. Other disorders such as diabetes, cancer, malaria, obesity, and cardiovascular diseases have used standardized, preclinical models that allow laboratories to compare results. Standardized models accelerate the pace of research and such models have been used to test new therapies or changes in treatment guidelines. The National Institutes of Health mandated that investigators increase data reproducibility and the rigor of scientific experiments and has also issued research funding announcements about the development and refinement of standardized models. Our premise is that refinement and standardization of preclinical sepsis models may accelerate the development and testing of potential therapeutics for human sepsis, as has been the case with preclinical models for other disorders. As a first step toward creating standardized models, we suggest standardizing the technical standards of the widely used cecal ligation and puncture model and creating a list of appropriate organ injury and immune dysfunction parameters. Standardized sepsis models could enhance reproducibility and allow comparison of results between laboratories and may accelerate our understanding of the pathogenesis of sepsis.

Quality of design and reporting of animal research in peritoneal dialysis: A scoping review

The concerns about reproducibility and validity of animal studies are partly related to poor experimental design and reporting. Here, we undertook a scoping review of the literature to determine the extent and quality of reporting of animal studies on peritoneal dialysis (PD). Online databases were searched to identify 567 relevant original articles published between 1979 and 2018. These were analyzed with respect to bibliographic parameters and general aspects of animal experimentation. A subgroup of 120 studies was analyzed in detail in terms of the impact on the reporting quality of the Animal Research: Reporting of In Vivo Experiments (ARRIVE) guidelines for animal studies. The number of animal studies on PD increased continuously over the years with a thematic shift toward long-term preservation of the peritoneum as a dialyzing organ. There were significant deficiencies in research design with the lack of sample size estimation, randomization, and blinding being the commonest shortcomings. The description of animal numbers, housing conditions, use of medication, and statistical analysis was incomplete. The introduction in 2010 of the ARRIVE guidelines produced very little improvement in the completeness of reporting regardless of journal impact factor. The animal studies on PD suffer from deficits in experimental protocols and transparent reporting. These drawbacks need to be corrected to ensure high-quality and much-needed animal research in PD.

Do oncology researchers adhere to reproducible and transparent principles? A cross-sectional survey of published oncology literature

OBJECTIVES

As much as 50%-90% of research is estimated to be irreproducible, costing upwards of $28 billion in USA alone. Reproducible research practices are essential to improving the reproducibility and transparency of biomedical research, such as including preregistering studies, publishing a protocol, making research data and metadata publicly available, and publishing in open access journals. Here we report an investigation of key reproducible or transparent research practices in the published oncology literature.

DESIGN

We performed a cross-sectional analysis of a random sample of 300 oncology publications published from 2014 to 2018. We extracted key reproducibility and transparency characteristics in a duplicative fashion by blinded investigators using a pilot tested Google Form.

PRIMARY OUTCOME MEASURES

The primary outcome of this investigation is the frequency of key reproducible or transparent research practices followed in published biomedical and clinical oncology literature.

RESULTS

Of the 300 publications randomly sampled, 296 were analysed for reproducibility characteristics. Of these 296 publications, 194 contained empirical data that could be analysed for reproducible and transparent research practices. Raw data were available for nine studies (4.6%). Five publications (2.6%) provided a protocol. Despite our sample including 15 clinical trials and 7 systematic reviews/meta-analyses, only 7 included a preregistration statement. Less than 25% (65/194) of publications provided an author conflict of interest statement.

CONCLUSION

We found that key reproducibility and transparency characteristics were absent from a random sample of published oncology publications. We recommend required preregistration for all eligible trials and systematic reviews, published protocols for all manuscripts, and deposition of raw data and metadata in public repositories.

Methodological standards, quality of reporting and regulatory compliance in animal research on amyotrophic lateral sclerosis: a systematic review

Objectives

The amyotrophic lateral sclerosis (ALS) research community was one of the first to adopt methodology guidelines to improve preclinical research reproducibility. We here present the results of a systematic review to investigate how the standards in this field changed over the 10-year period during which the guidelines were first published (2007) and updated (2010).

Methods

We searched for papers reporting ALS research on SOD1 (superoxide dismutase 1) mice published between 2005 and 2015 on the ISI Web of Science database, resulting in a sample of 569 papers to review, after triage. Two scores-one for methodological quality, one for regulatory compliance-were built from weighted sums of separate sets of items, and subjected to multivariable regression analysis, to assess how these related to publication year, type of study, country of origin and journal.

Results

Reporting standards improved over time. Of papers published after the first ALS guidelines were made public, fewer than 9% referred specifically to these. Of key research parameters, only three (genetic background, number of transgenes and group size) were reported in >50% of the papers. Information on housing conditions, randomisation and blinding was absent in over two-thirds of the papers. Group size was among the best reported parameters, but the majority reported using fewer than the recommended sample size and only two studies clearly justified group size.

Conclusions

General methodological standards improved gradually over a period of 8-10 years, but remained generally comparable with related fields with no specific guidelines, except with regard to severity. Only 11% of ALS studies were classified in the highest severity level (animals allowed to reach death or moribund stages), substantially below the proportion in studies of comparable neurodegenerative diseases such as Huntington's. The existence of field-specific guidelines, although a welcome indication of concern, seems insufficient to ensure adherence to high methodological standards. Other mechanisms may be required to improve methodological and welfare standards.

A study to assess the methodological quality of in vivo animal experiments published in Indian journal of pharmacology: A retrospective, cross-sectional, observational study

BACKGROUND

Good reporting of experimental research is a vital part of research process. Although the reporting guidelines such as Animal research: Reporting in vivo experiments (ARRIVE) require the authors to stick to a standard format, they do not guarantee study reports' validity. For assessing the study reports validity, critical appraisal tools are required.

OBJECTIVE

The evaluation of the methodological quality of in vivo animal studies of Indian Journal of Pharmacology (IJP) published between 2011 and 2017 through critical appraisal was the primary objective. The secondary objective was to evaluate the adherence of selected studies to the ARRIVE guidelines.

MATERIALS AND METHODS

All in vivo animal studies published as full-text articles in IJP from January 2011 to December 2017 and satisfying the inclusion norms were included. A checklist based on the underlying principles of ARRIVE statement was applied to in vivo animal research (AR) published in IJP. For critical appraisal of reports, risk-of-bias domains were also applied on studies from in vivo AR.

RESULTS

One hundred and sixty-one studies satisfied the inclusion criteria. Seventy-three articles (45.34%, 95% confidence interval [CI]: 0.37-0.53) discussed about randomization procedure. Eighteen articles (11.18%, 95% CI: 0.07-0.16) gave details about blinding when assessing results. None of the studies provided details of sample size calculation. Eight articles (4.97%, 95% CI: 0.02-0.09) commented on the study limitations.

CONCLUSION

It was found that adherence to only some criteria of ARRIVE guidelines was subpar. There is a need for optimal reporting of random distribution of animals to experimental groups, concealment of allocation, blinded outcome evaluation, computation of sample size, and attrition of animals for improving the validity of AR.

The reporting quality of acupuncture for neurogenesis in experimental ischemic stroke study

Background

There is increasing evidence demonstrating the highly inadequate reporting of preclinical research in multiple scientific publications. The purpose of this study is to systematically investigate the reporting quality of acupuncture for neurogenesis in animal models of acute ischemic stroke.

Methods

We searched eight databases, including PubMed, EMBASE, CINAHL, AMED, Chinese National Knowledge Infrastructure, VIP information database, Wanfang data Information Site, and Chinese Biomedical Literature Database. The methodological quality of included studies was assessed by using the CAMARADES 10-item checklist. The STRICTA statement was modified to gear to animal acupuncture research. The reporting quality was assessed according to the ARRIVE guidelines and the modified STRICTA statement. Data were analyzed with descriptive statistics.

Results

Ultimately, 44 studies containing 2,411 subjects were identified. The overall compliance with the CAMARADES 10-item checklist has a mean of 4.3. The reporting quality indicated that the overall compliance with ARRIVE guidelines has a mean of 59.9% and with the modified STRICTA statement a mean of 71.8%. The findings suggest that the reporting quality of acupuncture for preclinical stroke was generally poor.

Conclusions

Full compliance with ARRIVE guidelines and/or modified STRICTA statement in designing, conducting and reporting preclinical acupuncture research is urgently needed in the future.

The importance of selection and reporting of the sex of experimental animals

Biased use of males and females in animal studies or omitting specific details of the sex of animals used in publications limits reproducibility, hampers the pace and likelihood of new discoveries and invites adverse events in ensuing clinical research. Hence unbiased use of males and females in animal studies and specific reporting of animal details are increasingly required by funding bodies and scientific journals worldwide. An analysis to determine how males and females are used in animal studies involving the Commonwealth Scientific and Industrial Research Organisation (CSIRO) was undertaken as part of a process to review and further support best practice. In the study 178 publications that contain animal studies and include CSIRO researchers published between January 2014 and December 2016 were analysed for the sex of animals used. The overall sex distribution was males only 26.4% (47/178), females only 15.7% (28/178), males and females 18.0% (32/178) and sex of animals unspecified 39.9% (71/178). Reasons for this distribution include species biology, farming practices and commercial relevance. Although including sex as an experimental variable provides the most information, using both sexes in an animal study requires careful consideration and planning. Furthermore, there are valid biological and experimental reasons why sex distribution in a study may not be balanced. Biological reasons include cases where the severity of disease in a given model differs between males and females, superior husbandry or production traits in one sex and hermaphroditic species that change sex with age. Examples where experiments can only be undertaken in one sex of animal include animal models of female breast cancer, female reproductive traits, male fertility studies and post-castration welfare outcomes. Where there is no biological or experimental reason for using a single sex of animal, future studies should obtain an estimate of sex effect either from the literature or with a pilot study, and experiments should be planned and reported accordingly.

Part I: Minimum Quality Threshold in Preclinical Sepsis Studies (MQTiPSS) for Study Design and Humane Modeling Endpoints

Preclinical animal studies are mandatory before new treatments can be tested in clinical trials. However, their use in developing new therapies for sepsis has been controversial because of limitations of the models and inconsistencies with the clinical conditions. In consideration of the revised definition for clinical sepsis and septic shock (Sepsis-3), a Wiggers-Bernard Conference was held in Vienna in May 2017 to propose standardized guidelines on preclinical sepsis modeling. The participants conducted a literature review of 260 most highly cited scientific articles on sepsis models published between 2003 and 2012. The review showed, for example, that mice were used in 79% and euthanasia criteria were defined in 9% of the studies. Part I of this report details the recommendations for study design and humane modeling endpoints that should be addressed in sepsis models. The first recommendation is that survival follow-up should reflect the clinical time course of the infectious agent used in the sepsis model. Furthermore, it is recommended that therapeutic interventions should be initiated after the septic insult replicating clinical care. To define an unbiased and reproducible association between a new treatment and outcome, a randomization and blinding of treatments as well as inclusion of all methodological details in scientific publications is essential. In all preclinical sepsis studies, the high standards of animal welfare must be implemented. Therefore, development and validation of specific criteria for monitoring pain and distress, and euthanasia of septic animals, as well as the use of analgesics are recommended. A set of four considerations is also proposed to enhance translation potential of sepsis models. Relevant biological variables and comorbidities should be included in the study design and sepsis modeling should be extended to mammalian species other than rodents. In addition, the need for source control (in case of a defined infection focus) should be considered. These recommendations and considerations are proposed as "best practices" for animal models of sepsis that should be implemented.

Animal experimental research design in critical care

Background

Limited translational success in critical care medicine is thought to be in part due to inadequate methodology, study design, and reporting in preclinical studies. The purpose of this study was to compare reporting of core features of experimental rigor: blinding, randomization, and power calculations in critical care medicine animal experimental research. We hypothesized that these study design characteristics were more frequently reported in 2015 versus 2005.

Methods

We performed an observational bibliometric study to grade manuscripts on blinding, randomization, and power calculations. Chi-square tests and logistic regression were used for analysis. Inter-rater agreement was assessed using kappa and Gwet’s AC1.

Results

A total of 825 articles from seven journals were included. In 2005, power estimations were reported in 2%, randomization in 35%, and blinding in 20% (n = 482). In 2015, these metrics were included in 9, 47, and 36% of articles (n = 343). The increase in proportion for the metrics tested was statistically significant (p < 0.001, p = 0.002, and p < 0.001).

Conclusions

Only a minority of published manuscripts in critical care medicine journals reported on recommended study design steps to increase rigor. Routine justification for the presence or absence of blinding, randomization, and power calculations should be considered to better enable readers to assess potential sources of bias.

Could diet composition modulate pathological outcomes in schistosomiasis mansoni? A systematic review of in vivo preclinical evidence

Schistosomiasis and malnutrition are often overlapped in poor communities, resulting in disproportionately high mortality rates. Currently, fragmented data make it difficult to define the relationship between diet and schistosomiasis. Thus, we systematically review the preclinical evidence on the impact of diet in Schistosoma mansoni infection. From a structured search, we recovered 27 original articles. All studies used mice and most of them investigated hypoproteic (70.37%), hyperlipidic (22.22%) or vitamin-deficient (7.41%) diets. Diets based on carbohydrate, zinc or milk supplementation were investigated at a reduced frequency (3.70% each). Hypoproteic diets attenuated parasitic load and granulomatous inflammation, but also reduced host resistance to S. mansoni infection, determining higher mortality rates. By stimulating steatohepatitis, parasitic load and granulomatous inflammation, hyperlipidic diets increase organ damage and mortality in infected animals. Although a high-sugar diet and vitamin restriction potentiate and zinc supplementation attenuates S. mansoni infection, the current evidence for these diets remains inconclusive. Analysis of methodological quality indicated that the current evidence is at high risk of bias due to incomplete characterization of the experimental design, diet composition and treatment protocols. From the bias analysis, we report methodological limitations that should be considered to avoid systematic reproduction of inconsistent and poorly reproducible experimental designs.

The Weak Spots in Contemporary Science (and How to Fix Them)

In this review, the author discusses several of the weak spots in contemporary science, including scientific misconduct, the problems of post hoc hypothesizing (HARKing), outcome switching, theoretical bloopers in formulating research questions and hypotheses, selective reading of the literature, selective citing of previous results, improper blinding and other design failures, p-hacking or researchers' tendency to analyze data in many different ways to find positive (typically significant) results, errors and biases in the reporting of results, and publication bias. The author presents some empirical results highlighting problems that lower the trustworthiness of reported results in scientific literatures, including that of animal welfare studies. Some of the underlying causes of these biases are discussed based on the notion that researchers are only human and hence are not immune to confirmation bias, hindsight bias, and minor ethical transgressions. The author discusses solutions in the form of enhanced transparency, sharing of data and materials, (post-publication) peer review, pre-registration, registered reports, improved training, reporting guidelines, replication, dealing with publication bias, alternative inferential techniques, power, and other statistical tools.

Postmarketing withdrawal of human medicinal products because of adverse reactions in animals: a systematic review and analysis

PURPOSE

We have identified human medicinal products for which animal data were used as evidence for withdrawal, determined whether the adverse reactions were reported in humans, established whether confirmatory human studies were conducted, and explored the withdrawal patterns over time.

METHODS

We searched the World Health Organization's Consolidated List of [Medicinal] Products, drug regulatory authorities' websites, PubMed, Google Scholar, and selected textbooks to identify medicinal products withdrawn from 1950 to June 2016. We included medicinal products for which animal data were specifically reported as a reason for withdrawal. We used a checklist adapted from the International Agency for Research on Cancer criteria to rate the evidence.

RESULTS

In 37 cases, evidence from animals was the reason given for withdrawal between 1963 and 2000. Evidence of carcinogenicity was cited in 23 cases (62%). Limited evidence for harms occasioned withdrawal in over 80% of cases. In 11 cases (30%), the adverse drug reactions were subsequently reported in humans. In 5 instances (14%), formal studies were conducted in humans. The median interval to withdrawal following reports of adverse reactions was 2 years (IQR = 1-9 y).

CONCLUSIONS

Regulatory authorities and drug manufacturers are likely to withdraw medicinal products quickly from the market when animal experiments suggest increased risks of cancers or congenital malformations. Human studies are seldom conducted when harms are suspected in animals. Future research should explore better methods of extrapolating harms data from animal research to humans.

Publication bias in animal research presented at the 2008 Society of Critical Care Medicine Conference

Background

To determine a direct measure of publication bias by determining subsequent full-paper publication (P) of studies reported in animal research abstracts presented at an international conference (A).

Methods

We selected 100 random (using a random-number generator) A from the 2008 Society of Critical Care Medicine Conference. Using a data collection form and study manual, we recorded methodology and result variables from A. We searched PubMed and EMBASE to June 2015, and DOAJ and Google Scholar to May 2017 to screen for subsequent P. Methodology and result variables were recorded from P to determine changes in reporting from A. Predictors of P were examined using Fisher’s Exact Test.

Results

62% (95% CI 52–71%) of studies described in A were subsequently P after a median 19 [IQR 9–33.3] months from conference presentation. Reporting of studies in A was of low quality: randomized 27% (the method of randomization and allocation concealment not described), blinded 0%, sample-size calculation stated 0%, specifying the primary outcome 26%, numbers given with denominators 6%, and stating number of animals used 47%. Only being an orally presented (vs. poster presented) A (14/16 vs. 48/84, p = 0.025) predicted P. Reporting of studies in P was of poor quality: randomized 39% (the method of randomization and allocation concealment not described), likely blinded 6%, primary outcome specified 5%, sample size calculation stated 0%, numbers given with denominators 34%, and number of animals used stated 56%. Changes in reporting from A to P occurred: from non-randomized to randomized 19%, from non-blinded to blinded 6%, from negative to positive outcomes 8%, from having to not having a stated primary outcome 16%, and from non-statistically to statistically significant findings 37%. Post-hoc, using publication data, P was predicted by having positive outcomes (published 62/62, unpublished 33/38; p = 0.003), or statistically significant results (published 58/62, unpublished 20/38; p < 0.001).

Conclusions

Only 62% (95% CI 52–71%) of animal research A are subsequently P; this was predicted by oral presentation of the A, finally having positive outcomes, and finally having statistically significant results. Publication bias is prevalent in critical care animal research.

Electronic supplementary material

The online version of this article (doi:10.1186/s13104-017-2574-0) contains supplementary material, which is available to authorized users.

Hyperoxia provokes a time- and dose-dependent inflammatory response in mechanically ventilated mice, irrespective of tidal volumes

Background

Mechanical ventilation and hyperoxia have the potential to independently promote lung injury and inflammation. Our purpose was to study both time- and dose-dependent effects of supplemental oxygen in an experimental model of mechanically ventilated mice.

Methods

Healthy male C57Bl/6J mice, aged 9–10 weeks, were intraperitoneally anesthetized and randomly assigned to the mechanically ventilated group or the control group. In total, 100 mice were tracheotomized and mechanically ventilated for either 8 or 12 h after allocation to different settings for the applied fractions of inspired oxygen (FiO₂, 30, 50, or 90%) and tidal volumes (7.5 or 15 ml/kg). After euthanisation arterial blood, bronchoalveolar lavage fluid (BALf) and tissues were collected for analyses.

Results

Mechanical ventilation significantly increased the lung injury score (P < 0.05), mean protein content (P < 0.001), and the mean number of cells (P < 0.01), including neutrophils in BALf (P < 0.001). In mice ventilated for 12 h, a significant increase in TNF-α, IFN-γ, IL-1β, IL-10, and MCP-1 (P < 0.01) was observed with 90% FiO₂, whereas IL-6 showed a decreasing trend (P for trend = 0.03) across FiO₂ groups. KC, MIP-2, and sRAGE were similar between FiO₂ groups. HMGB-1 was significantly higher in BALf of mechanically ventilated mice compared to controls and showed a gradual increase in expression with increasing FiO₂. Cytokine and chemokine levels in BALf did not markedly differ between FiO₂ groups after 8 h of ventilation. Differences between the tidal volume groups were small and did not appear to significantly interact with the oxygen levels.

Conclusions

We demonstrated a severe vascular leakage and a pro-inflammatory pulmonary response in mechanically ventilated mice, which was enhanced by severe hyperoxia and longer duration of mechanical ventilation. Prolonged ventilation with high oxygen concentrations induced a time-dependent immune response characterized by elevated levels of neutrophils, cytokines, and chemokines in the pulmonary compartment.

Electronic supplementary material

The online version of this article (doi:10.1186/s40635-017-0142-5) contains supplementary material, which is available to authorized users.

Impact of Stroke Therapy Academic Industry Roundtable (STAIR) Guidelines on Peri-Anesthesia Care for Rat Models of Stroke: A Meta-Analysis Comparing the Years 2005 and 2015

Numerous studies using rats in stroke models have failed to translate into successful clinical trials in humans. The Stroke Therapy Academic Industry Roundtable (STAIR) has produced guidelines on the rodent stroke model for preclinical trials in order to promote the successful translation of animal to human studies. These guidelines also underline the importance of anaesthetic and monitoring techniques. The aim of this literature review is to document whether anaesthesia protocols (i.e., choice of agents, mode of ventilation, physiological support and monitoring) have been amended since the publication of the STAIR guidelines in 2009. A number of articles describing the use of a stroke model in adult rats from the years 2005 and 2015 were randomly selected from the PubMed database and analysed for the following parameters: country where the study was performed, strain of rats used, technique of stroke induction, anaesthetic agent for induction and maintenance, mode of intubation and ventilation, monitoring techniques, control of body temperature, vascular accesses, and administration of intravenous fluids and analgesics. For each parameter (stroke, induction, maintenance, monitoring), exact chi-square tests were used to determine whether or not proportions were significantly different across year and p values were corrected for multiple comparisons. An exact p-test was used for each parameter to compare the frequency distribution of each value followed by a Bonferroni test. The level of significant set at < 0.05. Results show that there were very few differences in the anaesthetic and monitoring techniques used between 2005 and 2015. In 2015, significantly more studies were performed in China and significantly fewer studies used isoflurane and nitrous oxide. The most striking finding is that the vast majority of all the studies from both 2005 and 2015 did not report the use of ventilation; measurement of blood gases, end-tidal carbon dioxide concentration, or blood pressure; or administration of intravenous fluids or analgesics. The review of articles published in 2015 showed that the STAIR guidelines appear to have had no effect on the anaesthetic and monitoring techniques in rats undergoing experimental stroke induction, despite the publication of said guidelines in 2009.

Expectations for the methodology and translation of animal research: a survey of the general public, medical students and animal researchers in North America

To determine what are considered acceptable standards for animal research (AR) methodology and translation rate to humans, a validated survey was sent to: a) a sample of the general public, via Sampling Survey International (SSI; Canada), Amazon Mechanical Turk (AMT; USA), a Canadian city festival (CF) and a Canadian children's hospital (CH); b) a sample of medical students (two first-year classes); and c) a sample of scientists (corresponding authors and academic paediatricians). There were 1379 responses from the general public sample (SSI, n = 557; AMT, n = 590; CF, n = 195; CH, n = 102), 205/330 (62%) medical student responses, and 23/323 (7%, too few to report) scientist responses. Asked about methodological quality, most of the general public and medical student respondents expect that: AR is of high quality (e.g. anaesthesia and analgesia are monitored, even overnight, and 'humane' euthanasia, optimal statistical design, comprehensive literature review, randomisation and blinding, are performed), and costs and difficulty are not acceptable justifications for lower quality (e.g. costs of expert consultation, or more laboratory staff). Asked about their expectations of translation to humans (of toxicity, carcinogenicity, teratogenicity and treatment findings), most expect translation more than 60% of the time. If translation occurred less than 20% of the time, a minority disagreed that this would "significantly reduce your support for AR". Medical students were more supportive of AR, even if translation occurred less than 20% of the time. Expectations for AR are much higher than empirical data show to have been achieved.

Pain and Laboratory Animals: Publication Practices for Better Data Reproducibility and Better Animal Welfare

Scientists who perform major survival surgery on laboratory animals face a dual welfare and methodological challenge: how to choose surgical anesthetics and post-operative analgesics that will best control animal suffering, knowing that both pain and the drugs that manage pain can all affect research outcomes. Scientists who publish full descriptions of animal procedures allow critical and systematic reviews of data, demonstrate their adherence to animal welfare norms, and guide other scientists on how to conduct their own studies in the field. We investigated what information on animal pain management a reasonably diligent scientist might find in planning for a successful experiment. To explore how scientists in a range of fields describe their management of this ethical and methodological concern, we scored 400 scientific articles that included major animal survival surgeries as part of their experimental methods, for the completeness of information on anesthesia and analgesia. The 400 articles (250 accepted for publication pre-2011, and 150 in 2014–15, along with 174 articles they reference) included thoracotomies, craniotomies, gonadectomies, organ transplants, peripheral nerve injuries, spinal laminectomies and orthopedic procedures in dogs, primates, swine, mice, rats and other rodents. We scored articles for Publication Completeness (PC), which was any mention of use of anesthetics or analgesics; Analgesia Use (AU) which was any use of post-surgical analgesics, and Analgesia Completeness (a composite score comprising intra-operative analgesia, extended post-surgical analgesia, and use of multimodal analgesia). 338 of 400 articles were PC. 98 of these 338 were AU, with some mention of analgesia, while 240 of 338 mentioned anesthesia only but not post-surgical analgesia. Journals’ caliber, as measured by their 2013 Impact Factor, had no effect on PC or AU. We found no effect of whether a journal instructs authors to consult the ARRIVE publishing guidelines published in 2010 on PC or AC for the 150 mouse and rat articles in our 2014–15 dataset. None of the 302 articles that were silent about analgesic use included an explicit statement that analgesics were withheld, or a discussion of how pain management or untreated pain might affect results. We conclude that current scientific literature cannot be trusted to present full detail on use of animal anesthetics and analgesics. We report that publication guidelines focus more on other potential sources of bias in experimental results, under-appreciate the potential for pain and pain drugs to skew data, and thus mostly treat pain management as solely an animal welfare concern, in the jurisdiction of animal care and use committees. At the same time, animal welfare regulations do not include guidance on publishing animal data, even though publication is an integral part of the cycle of research and can affect the welfare of animals in studies building on published work, leaving it to journals and authors to voluntarily decide what details of animal use to publish. We suggest that journals, scientists and animal welfare regulators should revise current guidelines and regulations, on treatment of pain and on transparent reporting of treatment of pain, to improve this dual welfare and data-quality deficiency.

Bad Science: Cause and Consequence

Scientific progress is dependent on accumulation of quality data with appropriate data analysis. Unfortunately, there are a troubling number of accounts describing an inability to replicate published work. Some explanations are lack of access to proprietary reagents and equipment, or lack of expertise and know how. However, it is clear that there are many publications that are fatally flawed, and it is difficult to ascertain which ones they are, but there are clues. Many articles are improperly controlled, resulting in false-positive or -negative results. Reagents and procedures are used without verifying their specificity. There is also confirmation bias, a tendency to seek and find conclusions that we like, which is exacerbated by faithful acceptance by readers of the publication record without assessment of merit. These and other issues have slowed progress, resulted in waste of scarce funds, and even put patients at risk when clinical decisions are made according to flawed data. Solving these and related problems requires recognition of the problem and better training. We also need to take personal responsibility for not only our own work, but also for the accuracy of information in the scientific domain.

Risk of Bias in Reports of In Vivo Research: A Focus for Improvement

The reliability of experimental findings depends on the rigour of experimental design. Here we show limited reporting of measures to reduce the risk of bias in a random sample of life sciences publications, significantly lower reporting of randomisation in work published in journals of high impact, and very limited reporting of measures to reduce the risk of bias in publications from leading United Kingdom institutions. Ascertainment of differences between institutions might serve both as a measure of research quality and as a tool for institutional efforts to improve research quality.

Expectations for methodology and translation of animal research: a survey of health care workers

Background

Health care workers (HCW) often perform, promote, and advocate use of public funds for animal research (AR); therefore, an awareness of the empirical costs and benefits of animal research is an important issue for HCW. We aim to determine what health-care-workers consider should be acceptable standards of AR methodology and translation rate to humans.

Methods

After development and validation, an e-mail survey was sent to all pediatricians and pediatric intensive care unit nurses and respiratory-therapists (RTs) affiliated with a Canadian University. We presented questions about demographics, methodology of AR, and expectations from AR. Responses of pediatricians and nurses/RTs were compared using Chi-square, with P < .05 considered significant.

Results

Response rate was 44/114(39%) (pediatricians), and 69/120 (58%) (nurses/RTs). Asked about methodological quality, most respondents expect that: AR is done to high quality; costs and difficulty are not acceptable justifications for low quality; findings should be reproducible between laboratories and strains of the same species; and guidelines for AR funded with public money should be consistent with these expectations. Asked about benefits of AR, most thought that there are sometimes/often large benefits to humans from AR, and disagreed that “AR rarely produces benefit to humans.” Asked about expectations of translation to humans (of toxicity, carcinogenicity, teratogenicity, and treatment findings), most: expect translation >40% of the time; thought that misleading AR results should occur <21% of the time; and that if translation was to occur <20% of the time, they would be less supportive of AR. There were few differences between pediatricians and nurses/RTs.

Conclusions

HCW have high expectations for the methodological quality of, and the translation rate to humans of findings from AR. These expectations are higher than the empirical data show having been achieved. Unless these areas of AR significantly improve, HCW support of AR may be tenuous.