1
|
Héroux M, Diong J, Bye E, Fisher G, Robertson L, Butler A, Gandevia S. Poor statistical reporting, inadequate data presentation and spin persist despite Journal awareness and updated Information for Authors. F1000Res 2023; 12:1483. [PMID: 38434651 PMCID: PMC10905014 DOI: 10.12688/f1000research.142841.1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 11/13/2023] [Indexed: 03/05/2024] Open
Abstract
Sound reporting of research results is fundamental to good science. Unfortunately, poor reporting is common and does not improve with editorial educational strategies. We investigated whether publicly highlighting poor reporting at a journal can lead to improved reporting practices. We also investigated whether reporting practices that are required or strongly encouraged in journal Information for Authors are enforced by journal editors and staff. A 2016 audit highlighted poor reporting practices in the Journal of Neurophysiology. In August 2016 and 2018, the American Physiological Society updated the Information for Authors, which included the introduction of several required or strongly encouraged reporting practices. We audited Journal of Neurophysiology papers published in 2019 and 2020 (downloaded through the library of the University of New South Wales) on reporting items selected from the 2016 audit, the newly introduced reporting practices, and items from previous audits. Summary statistics (means, counts) were used to summarize audit results. In total, 580 papers were audited. Compared to results from the 2016 audit, several reporting practices remained unchanged or worsened. For example, 60% of papers erroneously reported standard errors of the mean, 23% of papers included undefined measures of variability, 40% of papers failed to define a statistical threshold for their tests, and when present, 64% of papers with p-values between 0.05 and 0.1 misinterpreted them as statistical trends. As for the newly introduced reporting practices, required practices were consistently adhered to by 34 to 37% of papers, while strongly encouraged practices were consistently adhered to by 9 to 26% of papers. Adherence to the other audited reporting practices was comparable to our previous audits. Publicly highlighting poor reporting practices did little to improve research reporting. Similarly, requiring or strongly encouraging reporting practices was only partly effective. Although the present audit focused on a single journal, this is likely not an isolated case. Stronger, more strategic measures are required to improve poor research reporting.
Collapse
Affiliation(s)
- Martin Héroux
- School of Biomedical Sciences, University of New South Wales, Sydney, New South Wales, 2052, Australia
- Neuroscience Research Australia, Sydney, NSW, 2031, Australia
| | - Joanna Diong
- Neuroscience Research Australia, Sydney, NSW, 2031, Australia
- School of Biomedical Sciences, The University of Sydney, Sydney, New South Wales, 2006, Australia
| | - Elizabeth Bye
- School of Biomedical Sciences, University of New South Wales, Sydney, New South Wales, 2052, Australia
- Neuroscience Research Australia, Sydney, NSW, 2031, Australia
| | - Georgia Fisher
- Neuroscience Research Australia, Sydney, NSW, 2031, Australia
- Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney, New South Wales, 2109, Australia
| | - Lucy Robertson
- Neuroscience Research Australia, Sydney, NSW, 2031, Australia
- School of Clinical Medicine, University of New South Wales, Sydney, New South Wales, 2031, Australia
| | - Annie Butler
- School of Biomedical Sciences, University of New South Wales, Sydney, New South Wales, 2052, Australia
- Neuroscience Research Australia, Sydney, NSW, 2031, Australia
| | - Simon Gandevia
- Neuroscience Research Australia, Sydney, NSW, 2031, Australia
- School of Clinical Medicine, University of New South Wales, Sydney, New South Wales, 2031, Australia
| |
Collapse
|
2
|
TSUSHIMA E. Interpreting Results from Statistical Hypothesis Testing: Understanding the Appropriate P-value. Phys Ther Res 2022; 25:49-55. [DOI: 10.1298/ptr.r0019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2022] [Accepted: 03/02/2022] [Indexed: 11/23/2022]
Affiliation(s)
- Eiki TSUSHIMA
- Graduate School of Health Sciences, Hirosaki University, Japan
| |
Collapse
|