The p value wars (again)

Sonographic, Demographic, and Clinical Characteristics of Pre- and Postmenopausal Women with Endometrial Cancer; Results from a Post Hoc Analysis of the IETA4 (International Endometrial Tumor Analysis) Multicenter Cohort

In this study, we conducted a comparative analysis of demographic, histopathological, and sonographic characteristics between pre- and postmenopausal women diagnosed with endometrial cancer, while also examining sonographic and anthropometric features in 'low' and 'intermediate/high-risk' cases, stratified by menopausal status. Our analysis, based on data from the International Endometrial Tumor Analysis (IETA) 4 cohort comprising 1538 women (161 premenopausal, 1377 postmenopausal) with biopsy-confirmed endometrial cancer, revealed that premenopausal women, compared to their postmenopausal counterparts, exhibited lower parity (median 1, IQR 0-2 vs. 1, IQR 1-2, p = 0.001), a higher family history of colon cancer (16% vs. 7%, p = 0.001), and smaller waist circumferences (median 92 cm, IQR 82-108 cm vs. 98 cm, IQR 87-112 cm, p = 0.002). Premenopausal women more often had a regular endometrial-myometrial border (39% vs. 23%, p < 0.001), a visible endometrial midline (23% vs. 11%, p < 0.001), and undefined tumor (73% vs. 84%, p = 0.001). Notably, despite experiencing a longer duration of abnormal uterine bleeding (median 5 months, IQR 3-12 vs. 3 months, 2-6, p < 0.001), premenopausal women more often had 'low' risk disease (78% vs. 46%, p < 0.001). Among sonographic and anthropometric features, only an irregular endometrial-myometrial border was associated with 'intermediate/high' risk in premenopausal women. Conversely, in postmenopausal women, multiple features correlated with 'intermediate/high' risk disease. Our findings emphasize the importance of considering menopausal status when evaluating sonographic features in women with endometrial cancer.

Poor statistical reporting, inadequate data presentation and spin persist despite Journal awareness and updated Information for Authors

Sound reporting of research results is fundamental to good science. Unfortunately, poor reporting is common and does not improve with editorial educational strategies. We investigated whether publicly highlighting poor reporting at a journal can lead to improved reporting practices. We also investigated whether reporting practices that are required or strongly encouraged in journal Information for Authors are enforced by journal editors and staff. A 2016 audit highlighted poor reporting practices in the Journal of Neurophysiology. In August 2016 and 2018, the American Physiological Society updated the Information for Authors, which included the introduction of several required or strongly encouraged reporting practices. We audited Journal of Neurophysiology papers published in 2019 and 2020 (downloaded through the library of the University of New South Wales) on reporting items selected from the 2016 audit, the newly introduced reporting practices, and items from previous audits. Summary statistics (means, counts) were used to summarize audit results. In total, 580 papers were audited. Compared to results from the 2016 audit, several reporting practices remained unchanged or worsened. For example, 60% of papers erroneously reported standard errors of the mean, 23% of papers included undefined measures of variability, 40% of papers failed to define a statistical threshold for their tests, and when present, 64% of papers with p-values between 0.05 and 0.1 misinterpreted them as statistical trends. As for the newly introduced reporting practices, required practices were consistently adhered to by 34 to 37% of papers, while strongly encouraged practices were consistently adhered to by 9 to 26% of papers. Adherence to the other audited reporting practices was comparable to our previous audits. Publicly highlighting poor reporting practices did little to improve research reporting. Similarly, requiring or strongly encouraging reporting practices was only partly effective. Although the present audit focused on a single journal, this is likely not an isolated case. Stronger, more strategic measures are required to improve poor research reporting.

Phylogenetic diversity analysis of shotgun metagenomic reads describes gut microbiome development and treatment effects in the post-weaned pig

Intensive farming practices can increase exposure of animals to infectious agents against which antibiotics are used. Orally administered antibiotics are well known to cause dysbiosis. To counteract dysbiotic effects, numerous studies in the past two decades sought to understand whether probiotics are a valid tool to help re-establish a healthy gut microbial community after antibiotic treatment. Although dysbiotic effects of antibiotics are well investigated, little is known about the effects of intramuscular antibiotic treatment on the gut microbiome and a few studies attempted to study treatment effects using phylogenetic diversity analysis techniques. In this study we sought to determine the effects of two probiotic- and one intramuscularly administered antibiotic treatment on the developing gut microbiome of post-weaning piglets between their 3^rd and 9^th week of life. Shotgun metagenomic sequences from over 800 faecal time-series samples derived from 126 post-weaning piglets and 42 sows were analysed in a phylogenetic framework. Differences between individual hosts such as breed, litter, and age, were found to be important contributors to variation in the community composition. Host age was the dominant factor in shaping the gut microbiota of piglets after weaning. The post-weaning pig gut microbiome appeared to follow a highly structured developmental program with characteristic post-weaning changes that can distinguish hosts that were born as little as two days apart in the second month of life. Treatment effects of the antibiotic and probiotic treatments were found but were subtle and included a higher representation of Mollicutes associated with intramuscular antibiotic treatment, and an increase of Lactobacillus associated with probiotic treatment. The discovery of correlations between experimental factors and microbial community composition is more commonly addressed with OTU-based methods and rarely analysed via phylogenetic diversity measures. The latter method, although less intuitive than the former, suffers less from library size normalization biases, and it proved to be instrumental in this study for the discovery of correlations between microbiome composition and host-, and treatment factors.

A systematic review and meta-analysis of the efficacy of intermittent theta burst stimulation (iTBS) on cognitive enhancement

Intermittent theta-burst stimulation (iTBS) has been used to focally regulate excitability of neural cortex over the past decade - however there is little consensus on the generalizability of effects reported in individual studies. Many studies use small sample sizes (N < 30), and there is a considerable amount of methodological heterogeneity in application of the stimulation itself. This systematic meta-analysis aims to consolidate the extant literature and determine if up-regulatory theta-burst stimulation reliably enhances cognition through measurable behavior. Results show that iTBS - when compared to suitable control conditions - may enhance cognition when outlier studies are removed, but also that there is a significant amount of heterogeneity across studies. Significant contributors to between-study heterogeneity include location of stimulation and method of navigation to the stimulation site. Surprisingly, the type of cognitive domain investigated was not a significant contributor of heterogeneity. The findings of this meta-analysis demonstrate that standardization of iTBS is urgent and necessary to determine if neuroenhancement of particular cognitive faculties are reliable and robust, and measurable through observable behavior.

Why we habitually engage in null-hypothesis significance testing: A qualitative study

Background

Null Hypothesis Significance Testing (NHST) is the most familiar statistical procedure for making inferences about population effects. Important problems associated with this method have been addressed and various alternatives that overcome these problems have been developed. Despite its many well-documented drawbacks, NHST remains the prevailing method for drawing conclusions from data. Reasons for this have been insufficiently investigated. Therefore, the aim of our study was to explore the perceived barriers and facilitators related to the use of NHST and alternative statistical procedures among relevant stakeholders in the scientific system.

Methods

Individual semi-structured interviews and focus groups were conducted with junior and senior researchers, lecturers in statistics, editors of scientific journals and program leaders of funding agencies. During the focus groups, important themes that emerged from the interviews were discussed. Data analysis was performed using the constant comparison method, allowing emerging (sub)themes to be fully explored. A theory substantiating the prevailing use of NHST was developed based on the main themes and subthemes we identified.

Results

Twenty-nine interviews and six focus groups were conducted. Several interrelated facilitators and barriers associated with the use of NHST and alternative statistical procedures were identified. These factors were subsumed under three main themes: the scientific climate, scientific duty, and reactivity. As a result of the factors, most participants feel dependent in their actions upon others, have become reactive, and await action and initiatives from others. This may explain why NHST is still the standard and ubiquitously used by almost everyone involved.

Conclusion

Our findings demonstrate how perceived barriers to shift away from NHST set a high threshold for actual behavioral change and create a circle of interdependency between stakeholders. By taking small steps it should be possible to decrease the scientific community’s strong dependence on NHST and p-values.

Multiplicity Eludes Peer Review: The Case of COVID-19 Research

Multiplicity arises when data analysis involves multiple simultaneous inferences, increasing the chance of spurious findings. It is a widespread problem frequently ignored by researchers. In this paper, we perform an exploratory analysis of the Web of Science database for COVID-19 observational studies. We examined 100 top-cited COVID-19 peer-reviewed articles based on p-values, including up to 7100 simultaneous tests, with 50% including >34 tests, and 20% > 100 tests. We found that the larger the number of tests performed, the larger the number of significant results (r = 0.87, p < 10⁻⁶). The number of p-values in the abstracts was not related to the number of p-values in the papers. However, the highly significant results (p < 0.001) in the abstracts were strongly correlated (r = 0.61, p < 10⁻⁶) with the number of p < 0.001 significances in the papers. Furthermore, the abstracts included a higher proportion of significant results (0.91 vs. 0.50), and 80% reported only significant results. Only one reviewed paper addressed multiplicity-induced type I error inflation, pointing to potentially spurious results bypassing the peer-review process. We conclude the need to pay special attention to the increased chance of false discoveries in observational studies, including non-replicated striking discoveries with a potentially large social impact. We propose some easy-to-implement measures to assess and limit the effects of multiplicity.

Improving target assessment in biomedical research: the GOT-IT recommendations

Academic research plays a key role in identifying new drug targets, including understanding target biology and links between targets and disease states. To lead to new drugs, however, research must progress from purely academic exploration to the initiation of efforts to identify and test a drug candidate in clinical trials, which are typically conducted by the biopharma industry. This transition can be facilitated by a timely focus on target assessment aspects such as target-related safety issues, druggability and assayability, as well as the potential for target modulation to achieve differentiation from established therapies. Here, we present recommendations from the GOT-IT working group, which have been designed to support academic scientists and funders of translational research in identifying and prioritizing target assessment activities and in defining a critical path to reach scientific goals as well as goals related to licensing, partnering with industry or initiating clinical development programmes. Based on sets of guiding questions for different areas of target assessment, the GOT-IT framework is intended to stimulate academic scientists' awareness of factors that make translational research more robust and efficient, and to facilitate academia-industry collaboration.