Practical Effects of Lowering the P Value in Neurosurgery: Restricting Evidence-Based Medicine to Big Business

Misinterpretations of Null Hypothesis Significance Testing Results Near the P-value Threshold: An Estimation of The Prevalence of Such a Statistical Error In The Neurosurgical Literature

INTRODUCTION

Although statistical errors have been shown to be prevalent in the medical literature there has been a paucity of studies focusing on the prevalence of such errors in neurosurgery. In this article, we audit the contemporary neurosurgical literature for a common statistical mistake, namely, misinterpretations of null hypothesis significance testing results near the p-value threshold by conveying the idea of a "trend".

METHODS

PubMed/Medline was used to identify all articles published in six major neurosurgical journals between 2000 and 2020. The abstracts of these articles were extracted and scrutinized to determine when the word "trend" was used to express "near-statistical significance".

RESULTS

A total of 45,244 articles were included. Of those 461 (1.02% [0.86-1.18]) employed the word "trend" to indicate near statistical significance, a total of 3.8 [2.93-4.75] errors per issue per year. The error under study occurred more frequently in JNS Spine and less frequently in Acta Neurochirurgica (p=0.007). On an annual basis, there was no linear correlation between the total number of papers published per journal and the number of errors in that journal (r=0.34, p = 0.01).

CONCLUSION

Misinterpretations of null hypothesis significance testing results near the p-value threshold is present in at least 1% of the neurosurgical literature. While we believe that most statistical errors in medicine in general and neurosurgery in particular may be unintentional in nature, additional measures should be enacted to prevent the subsequent adoption of such methodological practices amongst future researchers.