Improving transparency and scientific rigor in academic publishing

Assessing the transparency in reporting of clinical trials investigating manual therapy interventions for low back pain: A methodological review

RATIONALE

Low back pain (LBP) is a common condition with a significant societal burden. Manual therapy is an effective treatment for LBP and recommended in clinical practice guidelines. While the quantity of literature supporting the use of manual therapy is large, the methodological quality and transparency of this collective work are unclear.

AIMS AND OBJECTIVES

Explore the transparency in reporting of clinical trials assessing manual therapy interventions in patients with LBP by comparing planned components in the trial registration with what was reported in the published manuscript.

METHODS

Three databases were searched to identify trials assessing the treatment effect of manual therapy for LBP from January 2005 to May 2023. Studies were included if the manual therapy consisted of thrust manipulations, mobilizations or muscle energy techniques.

RESULTS

From 4462 studies initially identified, 167 studies remained in the final review after title, abstract and full-text review. Only 87 (52.1%) of the 167 studies were registered (n = 57 prospectively and n = 30 retrospectively). Primary outcomes in the publications were identical to the registration in 54 (62.1%) of the registered trials. Secondary outcomes in the publication were identical to the registration in 27 (31.0%) of the registered trials. The CONSORT reporting guideline was referenced in only 19 (21.8%) trials. Multiple discrepancies between registration and publication were noted for primary and secondary outcomes. All trials had eligibility criteria in the registration that matched their corresponding manuscript, while only four (4.6%) trial registrations addressed any type of statistical analysis plan.

CONCLUSION

Approximately half of the trials were not registered. Of those registered, only half were registered prospectively. Substantial discrepancies existed between registered and published outcomes that were never addressed by the authors, raising questions about potential bias. Transparency can be improved through more stringent requirements during manuscript submission to journals, and better reporting of the rationale for discrepancies between registration and publication.

The Elephant in the Room: Recognition and Documentation of Personnel Practices That Confound Reproducibility

The ability to apply findings from animal studies efficiently and effectively is predicated on an understanding of biology and pathobiology, how that biology relates to the human systems being modeled, and how the studies are conducted and reported. This overview discusses various factors in research within the animal environment (referred to as extrinsic factors) that the NIH now expects to be documented to foster replicability in science and expand interpretations of study outcomes. Specifically, an important extrinsic factor in research with animals is that of individual personnel who perform handling practices, participate in research interactions, and share an overall presence in the housing facility with animals, all of which can confound reproducibility efforts in biomedical science. An improved understanding of the influences and behaviors of animal research personnel on animal responses is critical with regard to research results and the interpretation of data collected from animal models of biomedical disease.

Standardization of zebrafish drug testing parameters for muscle diseases

Skeletal muscular diseases predominantly affect skeletal and cardiac muscle, resulting in muscle weakness, impaired respiratory function and decreased lifespan. These harmful outcomes lead to poor health-related quality of life and carry a high healthcare economic burden. The absence of promising treatments and new therapies for muscular disorders requires new methods for candidate drug identification and advancement in animal models. Consequently, the rapid screening of drug compounds in an animal model that mimics features of human muscle disease is warranted. Zebrafish are a versatile model in preclinical studies that support developmental biology and drug discovery programs for novel chemical entities and repurposing of established drugs. Due to several advantages, there is an increasing number of applications of the zebrafish model for high-throughput drug screening for human disorders and developmental studies. Consequently, standardization of key drug screening parameters, such as animal husbandry protocols, drug compound administration and outcome measures, is paramount for the continued advancement of the model and field. Here, we seek to summarize and explore critical drug treatment and drug screening parameters in the zebrafish-based modeling of human muscle diseases. Through improved standardization and harmonization of drug screening parameters and protocols, we aim to promote more effective drug discovery programs.

Gamification and education: A pragmatic approach with two examples of implementation

Leveraging elements of game design and theories of human motivation, gamification provides a variety of techniques to engage learners in novel ways. Our Clinical and Translational Science Award created the software platform (Kaizen-Education©) to deliver gamified educational content in 2012. Here, we explore two novel use cases of this platform to provide practical insights for leveraging these methods in educational settings: (1) national training in rigor, reproducibility, and transparency and (2) attainment of learner competency (n = 7) as a gauge of curricular effectiveness across Master of Public Health degree tracks (n = 5). Data were captured in real time during player interaction with Kaizen-Education© to provide descriptive analyses of player engagement in both implementation examples. We then assessed item analysis to assess knowledge gain and competency attainment. We have just begun to leverage the potential for gamification to engage learners, enhance knowledge acquisition, and document completion of training, across various learning environments. We encourage a systematic approach to gamification applying insights from self-determination theory to learners and learning environments, a methodical approach to game design and rigorous analysis after implementation to generate evidence-based insights to maximize educational return for time invested.

Opinion: Independent third-party entities as a model for validation of commercial antibodies

A vast array of commercial antibodies covers a large percentage of human gene products, but determining which among them is most appropriate for any given application is challenging. This leads to use of non-specific antibodies that contributes to issues with reproducibility. It is our opinion that the community of scientists who use commercial antibodies in their biomedical research would benefit from third-party antibody characterization entities that use standardized operating procedures to assess and compare antibody performance. Ideally, such entities would follow the principles of open science, such that all antibodies against any given protein target would be tested in parallel, and all data generated released to the public domain without bias. Furthermore, there should be no financial incentive for the entity beyond cost-recovery. Such non-profit organizations, combined with other scientific efforts, could catalyse new discoveries by providing scientists with better validated antibody tools.

Improving transparency and scientific rigor in academic publishing

Progress in basic and clinical research is slowed when researchers fail to provide a complete and accurate report of how a study was designed, executed, and the results analyzed. Publishing rigorous scientific research involves a full description of the methods, materials, procedures, and outcomes. Investigators may fail to provide a complete description of how their study was designed and executed because they may not know how to accurately report the information or the mechanisms are not in place to facilitate transparent reporting. Here, we provide an overview of how authors can write manuscripts in a transparent and thorough manner. We introduce a set of reporting criteria that can be used for publishing, including recommendations on reporting the experimental design and statistical approaches. We also discuss how to accurately visualize the results and provide recommendations for peer reviewers to enhance rigor and transparency. Incorporating transparency practices into research manuscripts will significantly improve the reproducibility of the results by independent laboratories. SIGNIFICANCE: Failure to replicate research findings often arises from errors in the experimental design and statistical approaches. By providing a full account of the experimental design, procedures, and statistical approaches, researchers can address the reproducibility crisis and improve the sustainability of research outcomes. In this piece, we discuss the key issues leading to irreproducibility and provide general approaches to improving transparency and rigor in reporting, which could assist in making research more reproducible.