Study Design Rigor in Animal-Experimental Research Published in Anesthesia Journals

Methodological rigour in preclinical urology: a systematic review reporting research quality over a 14-year period

OBJECTIVE

To investigate the prevalence and trends of essential study design elements in preclinical urological studies, as well as key factors that may improve methodological rigour, as the demand for methodological rigour in preclinical studies is increasing since research reproducibility and transparency in the medico-scientific field are being questioned.

METHODS AND RESULTS

PubMed was searched to include preclinical urological studies published between July 2007 to June 2021. A total of 3768 articles met the inclusion criteria. Data on study design elements and animal models used were collected. Citation density was also examined as a surrogate marker of study influence. We performed an analysis of the prevalence of seven critical study design elements and temporal patterns over 14 years. Randomisation was reported in 50.0%, blinding in 15.0%, sample size estimation in 1.0%, inclusion of both sexes in 6.3%, statistical analysis in 97.1%, housing and husbandry in 47.7%, and inclusion/exclusion criteria in 5.0%. Temporal analysis showed that the implementation of these study design elements has increased, except for inclusion of both sexes and inclusion/exclusion criteria. Reporting study design elements were associated with increased citation density in randomisation and statistical analysis.

CONCLUSIONS

The risk of bias is prevalent in 14-year publications describing preclinical urological research, and the quality of methodological rigour is barely related to the citation density of the article. Yet five study design elements (randomisation, blinding, sample size estimation, statistical analysis, and housing and husbandry) proposed by both the National Institutes of Health and Animal Research: Reporting of In Vivo Experiments guidelines have been either well reported or are being well reported over time.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42022233125.

A Study to Assess the Quality of Reporting of Animal Research Studies Published in PubMed Indexed Journals: A Retrospective, Cross-Sectional Content Analysis

Objectives: A complete and concise pre-clinical experimental research gives detailed information about the disease-specific model, prevents duplication, and saves animal life, money, as well as time. It will also allow readers to effectively interpret and evaluate the work and ensure that others can replicate the experiments described. The present study was conducted to assess the adequacy of animal details provided in published experimental animal studies.

Methods: All in vivo studies published as full-text articles in two PubMed indexed journals (one Indian and one international) from January 2011 to December 2019 and satisfying the inclusion norms were included. A checklist consisting of 27 discrete items subdivided under three domains, viz. animal details, disease model, and guidelines, was used. Every article was assessed by two investigators independently for determining the reporting quality.

Results: One hundred and seventy-seven studies satisfied the inclusion criteria. Age or age range was reported in 20.34% of the articles in the Indian journal and 5.88% articles in the international journal (p=0.019). Housing and husbandry details were reported in all the articles published in the international journal and 82.7% of the articles in the Indian journal (p=0.001). The disease/pathology studied was given in 70.62% of articles published in the Indian journal and 86.27% of articles published in the international journal (p=0.029). None of the studies provided details of genetic modification, health status, sample size calculation, steps taken to minimize bias, and implementation of randomization.

Conclusion: There is a need for optimal reporting of certain relevant animal details, disease models, experimental procedures, sample size calculation, and adherence to guidelines by the researchers for which the reporting was found to be sub-par to improve reproducibility and validity of animal research.

Biomonitoring and Digital Data Technology as an Opportunity for Enhancing Animal Study Translation

The failure of animal studies to translate to effective clinical therapeutics has driven efforts to identify underlying cause and develop solutions that improve the reproducibility and translatability of preclinical research. Common issues revolve around study design, analysis, and reporting as well as standardization between preclinical and clinical endpoints. To address these needs, recent advancements in digital technology, including biomonitoring of digital biomarkers, development of software systems and database technologies, as well as application of artificial intelligence to preclinical datasets can be used to increase the translational relevance of preclinical animal research. In this review, we will describe how a number of innovative digital technologies are being applied to overcome recurring challenges in study design, execution, and data sharing as well as improving scientific outcome measures. Examples of how these technologies are applied to specific therapeutic areas are provided. Digital technologies can enhance the quality of preclinical research and encourage scientific collaboration, thus accelerating the development of novel therapeutics.

Acupuncture for experimental cerebral ischemia/reperfusion injury: a systematic review of methodology and reporting quality

OBJECTIVES

Preclinical research is essential to the advancement of science but susceptible to insufficient reporting and methodological shortcomings, which compromise internal validity. We aimed to systematically assess the methodological and reporting quality of studies conducted on acupuncture for experimental cerebral ischemia/reperfusion injury (CIRI).

METHODS

A comprehensive search in six databases was performed for animal research concerning acupuncture for CIRI. Two authors independently selected articles, extracted data, and assessed the methodological and reporting quality of identified articles using the Systematic Review Center for Laboratory Animal Experimentation (SYRCLE) tool, and Animal Research: Reporting In Vivo Experiments (ARRIVE) guideline, respectively.

RESULTS

A total of 24 studies were identified. Only 1 article (4%) achieved a decent overall rating in using SYRCLE (percentage of items with "low risk" ⩾50%). Of the 22 items on the SYRCLE tool, only 8 items (37%) were rated as "low risk" of bias in more than 50% of the included studies. Of the 39 items of ARRIVE, 20 (51%) items were rated as "low risk" in more than 50% of the included studies.

CONCLUSIONS

The methodological and reporting quality of included studies was generally low, which demands further improvement. These findings should inform the development of evidence-based guidelines for future preclinical research assessing the effect of acupuncture on CIRI.

Evaluation of Preclinical and Clinical Studies Published in Medical Journals of Bosnia and Herzegovina: Methodology Issues

INTRODUCTION

Results of preclinical and clinical studies in medicine could be trusted only if their design and statistical analysis were appropriate.

AIM

The aim of our study was to investigate whether preclinical and clinical studies published in medical journals of Bosnia and Herzegovina satisfy basic requirements for appropriate design and statistical interpretation of data.

METHODS

Preclinical and clinical studies published in medical journals of Bosnia and Herzegovina were retrieved from the PubMed database, and the sample for analysis was randomly chosen from the retrieved publications. Implementation rate of basic principles of experimental design (local control, randomization and replication) and rate of the most common errors in design of clinical/observational studies was established by careful reading of the sampled publications and their checking against predefined criteria.

RESULTS

Our study showed that only a minority of experimental preclinical studies had basic principles of design completely implemented (7%), while implementation rate of single aspects of appropriate experimental design varied from as low as 12% to as high as 77%. Only one of the clinical/observational studies had none of the errors searched for (2%), and specific errors rates varied from 10% to 89%. Average impact factor of the surveyed studies was around one, and average publication date recent, less than 5 years ago.

CONCLUSION

Prevalence of preclinical studies that did not follow completely basic principles of research design, and that of clinical/observational studies with errors are high, raising suspicion to validity of their results. If incorrect and not protected against bias, results of published studies may adversely influence future research.

Evaluation of Published Preclinical Experimental Studies in Medicine: Methodology Issues

INTRODUCTION

Inappropriate design of experimental studies in medicine inevitably leads to inaccurate or false results, which serve as basis for erroneous and biased conclusions.

AIM

The aim of our study was to investigate prevalence of implementing basic principles of experimental design (local control, replication and randomization) in preclinical experimental studies, performed either on animals in vivo, or animal/human material in vitro.

MATERIAL AND METHODS

Preclinical experimental studies were retrieved from the PubMed database, and the sample for analysis was randomly chosen from the retrieved publications. Implementation rate of basic experimental research principles (local control, randomization and replication) was established by careful reading of the sampled publications and their checking against predefined criteria.

RESULTS

Our study showed that only a minority of experimental preclinical studies had basic principles of design completely implemented (7%), while implementation rate of single aspects of appropriate experimental design varied from as low as 9% to maximum 86%. Average impact factor of the surveyed studies was high, and publication date relatively recent, suggesting generalizability of our results to highly ranked contemporary journals.

CONCLUSION

Prevalence of experimental preclinical studies that did not implement completely basic principles of research design is high, raising suspicion to validity of their results. If incorrect and biased, results of published studies may mislead authors of future studies and cause conduction of fruitless research that will waste precious resources.

Animal experimental research design in critical care

Background

Limited translational success in critical care medicine is thought to be in part due to inadequate methodology, study design, and reporting in preclinical studies. The purpose of this study was to compare reporting of core features of experimental rigor: blinding, randomization, and power calculations in critical care medicine animal experimental research. We hypothesized that these study design characteristics were more frequently reported in 2015 versus 2005.

Methods

We performed an observational bibliometric study to grade manuscripts on blinding, randomization, and power calculations. Chi-square tests and logistic regression were used for analysis. Inter-rater agreement was assessed using kappa and Gwet’s AC1.

Results

A total of 825 articles from seven journals were included. In 2005, power estimations were reported in 2%, randomization in 35%, and blinding in 20% (n = 482). In 2015, these metrics were included in 9, 47, and 36% of articles (n = 343). The increase in proportion for the metrics tested was statistically significant (p < 0.001, p = 0.002, and p < 0.001).

Conclusions

Only a minority of published manuscripts in critical care medicine journals reported on recommended study design steps to increase rigor. Routine justification for the presence or absence of blinding, randomization, and power calculations should be considered to better enable readers to assess potential sources of bias.

The study design elements employed by researchers in preclinical animal experiments from two research domains and implications for automation of systematic reviews

Systematic reviews are increasingly using data from preclinical animal experiments in evidence networks. Further, there are ever-increasing efforts to automate aspects of the systematic review process. When assessing systematic bias and unit-of-analysis errors in preclinical experiments, it is critical to understand the study design elements employed by investigators. Such information can also inform prioritization of automation efforts that allow the identification of the most common issues. The aim of this study was to identify the design elements used by investigators in preclinical research in order to inform unique aspects of assessment of bias and error in preclinical research. Using 100 preclinical experiments each related to brain trauma and toxicology, we assessed design elements described by the investigators. We evaluated Methods and Materials sections of reports for descriptions of the following design elements: 1) use of comparison group, 2) unit of allocation of the interventions to study units, 3) arrangement of factors, 4) method of factor allocation to study units, 5) concealment of the factors during allocation and outcome assessment, 6) independence of study units, and 7) nature of factors. Many investigators reported using design elements that suggested the potential for unit-of-analysis errors, i.e., descriptions of repeated measurements of the outcome (94/200) and descriptions of potential for pseudo-replication (99/200). Use of complex factor arrangements was common, with 112 experiments using some form of factorial design (complete, incomplete or split-plot-like). In the toxicology dataset, 20 of the 100 experiments appeared to use a split-plot-like design, although no investigators used this term. The common use of repeated measures and factorial designs means understanding bias and error in preclinical experimental design might require greater expertise than simple parallel designs. Similarly, use of complex factor arrangements creates novel challenges for accurate automation of data extraction and bias and error assessment in preclinical experiments.