The Practical Alternative to the p Value Is the Correctly Used p Value

Beyond 'statistical significance': A nontechnical primer of Bayesian statistics and Bayes factors for health researchers

RATIONALE

Hypothesis testing is integral to health research and is commonly completed through frequentist statistics focused on computing p values. p Values have been long criticized for offering limited information about the relationship of variables and strength of evidence concerning the plausibility, presence and certainty of associations among variables. Bayesian statistics is a potential alternative for inference-making. Despite emerging discussion on Bayesian statistics across various disciplines, the uptake of Bayesian statistics in health research is still limited.

AIM

To offer a primer on Bayesian statistics and Bayes factors for health researchers to gain preliminary knowledge of its use, application and interpretation in health research.

METHODS

Theoretical and empirical literature on Bayesian statistics and methods were used to develop this methodological primer.

CONCLUSIONS

Using Bayesian statistics in health research without a careful and complete understanding of its underlying philosophy and differences from frequentist testing, estimation and interpretation methods can result in similar ritualistic use as done for p values.

IMPLICATIONS

Health researchers should supplement frequentists statistics with Bayesian statistics when analysing research data. The overreliance on p values for clinical decisions making should be avoided. Bayes factors offer a more intuitive measure of assessing the strength of evidence for null and alternative hypothesis.

Understanding p-values and significance

P-values combined with estimates of effect size are used to assess the importance of experimental results. However, their interpretation can be invalidated by selection bias when testing multiple hypotheses, fitting multiple models or even informally selecting results that seem interesting after observing the data. We offer an introduction to principled uses of p-values (targeted at the non-specialist) and identify questionable practices to be avoided.

Student radiographers' knowledge and experience of lateral hip X-ray positioning: A survey

INTRODUCTION

The horizontal beam lateral (HBL) position technique for X-ray imaging has been used for nearly a century; however, this can be challenging for the patient and the practitioner, as it potentially compromises patient dignity. This study explores student radiographers' knowledge and experience of lateral hip positions and their impact on diagnostic quality and patient dignity.

METHOD

A cross-sectional mixed-method online survey of undergraduate diagnostic radiography students was completed. Likert scale assessments, rank ordering questions, and free-test qualitative responses were utilised for questions on knowledge and experience of different positioning, ease to obtain, patient dignity, diagnostic quality, and need for repeats. Data analysis included descriptive statistics and cross-tabulation non-parametric analysis against variables of age, gender and year of study.

RESULTS

Responses were received by n = 42/158 students, a response rate of 27%. The HBL position was the most commonly repeated image (76.6%); the qualitative themes included HBL image quality issues and difficulty in the HBL positioning for elderly or frail patients, often in discomfort and pain. Analysis of student responses to perceived patient dignity in positioning identified 73.8% found the HBL undignified, and 85.7% agreed the Clements-Nakayama (CN) position would be more dignified for patients. The diagnostic image quality of the HBL position (64.2%) was compared to the CN alternative axiolateral (66.6%). Comparison of ease of obtaining the correct position for HBL (47.6%) was higher than CN position (28.6%); this could be due to the lack of experience n = 3/42 (7.1%) of this position.

CONCLUSION

Overall, student radiographers' experience and knowledge of various lateral hip positions observed in clinical practice was good. The CN position scored high for diagnostic image (66.6%) and dignity for the patient (85.7%), over the often repeated HBL position (76.6%), which scored lower for image quality (64.2%) and dignity (76.6%).

IMPLICATIONS FOR PRACTICE

Radiographers should advocate for professional autonomy and explore alternative positioning techniques. Further investigation into the CN position's utilisation, image quality and radiation dose in England is recommended.

Misinterpretation of statistical nonsignificance as a sign of potential bias: Hydroxychloroquine as a case study

The term "statistical significance," ubiquitous in the medical literature, is often misinterpreted, as is the "p-value" from which it stems. This article explores the implications of results that are numerically positive (e.g., those in the treatment arm do better on average) but not statistically significant. This lack of statistical significance is sometimes interpreted as strong, even decisive, evidence against an effect without due consideration of other factors. Three influential articles on hydroxychloroquine (HCQ) as a treatment for COVID-19 are illustrative. They all involve numerically positive results that were not statistically significant that were misinterpreted as strong evidence against HCQ's efficacy. These and related considerations raise concerns regarding the reliability of academic/medical reasoning around COVID-19 treatments, as well as more generally, and regarding the potential for bias stemming from conflicts of interest.

The Impact of a Vocal Loading Task on Voice Characteristics of Female Speakers With Benign Vocal Fold Lesions

OBJECTIVES

To examine the effect of a vocal loading task on measures of vocal structure and function in females with benign vocal fold lesions (BVFLs) and determine if change is observed in voice and lesion characteristics.

STUDY DESIGN

Prospective cohort study.

METHODS

Twenty-eight (n = 28) female subjects with phonotraumatic BVFLs completed a vocal loading task of 30 minutes of reading aloud at 75-85 dBA. Multidimensional voice evaluation was completed pre- and post-load, including audio and videostroboscopy recordings and images for expert perceptual ratings and acoustic and aerodynamic evaluation. Subjects also scored themselves using a 10 cm visual analogue scale for Perceived Phonatory Effort, and completed the Evaluation of Ability to Voice Easily, a 12 item self-report scale of current perceived speaking voice function. An exploratory rather than confirmatory approach to data analysis was adopted. The direction and magnitude of the change scores (pre- to post-load) for each individual, across a wide variety of instrumental and self-report measures, were assessed against a Minimal Clinically Important Difference criteria.

RESULTS

Observations of change and the direction of change in vocal response of individuals with BVFLs to 30 minutes of loud vocal load was variable. Minimal to no change was noted for participants pre- to post-load as rated perceptually, for auditory and videostroboscopy samples. For most instrumental measures, change was shown for many participants including an overall improvement in aerodynamic and acoustic measures of function and efficiency post-load for 20 participants (77%) and decline in function for 4 participants (15%). Self-reported effort and vocal function post-load was multidirectional with similar numbers of participants reporting no change, improved function or a decline.

CONCLUSION

Subjects with BVFLs demonstrate change in vocal function following 30 minutes of vocal load. While this change can be variable and multidirectional, overall improvement was observed in instrumental measures of function and efficiency for most participants. Some participants perceived this change to be an increase in effort, some a reduction in effort and some perceived no change. Improved vocal function despite relative lesion stability can seemingly occur after loading in some pathological voices.

Exploring a multimodal approach for utilizing digital biomarkers for childhood mental health screening

Background

Depression and anxiety are prevalent mental health concerns among children and adolescents. The application of conventional assessment methods, such as survey questionnaires to children, may lead to self-reporting issues. Digital biomarkers provide extensive data, reducing bias in mental health self-reporting, and significantly influence patient screening. Our primary objectives were to accurately assess children's mental health and to investigate the feasibility of using various digital biomarkers.

Methods

This study included a total of 54 boys and girls aged between 7 to 11 years. Each participant's mental state was assessed using the Depression, Anxiety, and Stress Scale. Subsequently, the subjects participated in digital biomarker collection tasks. Heart rate variability (HRV) data were collected using a camera sensor. Eye-tracking data were collected through tasks displaying emotion-face stimuli. Voice data were obtained by recording the participants' voices while they engaged in free speech and description tasks.

Results

Depressive symptoms were positively correlated with low frequency (LF, 0.04-0.15 Hz of HRV) in HRV and negatively associated with eye-tracking variables. Anxiety symptoms had a negative correlation with high frequency (HF, 0.15-0.40 Hz of HRV) in HRV and a positive association with LF/HF. Regarding stress, eye-tracking variables indicated a positive correlation, while pNN50, which represents the proportion of NN50 (the number of pairs of successive R-R intervals differing by more than 50 milliseconds) divided by the total number of NN (R-R) intervals, exhibited a negative association. Variables identified for childhood depression included LF and the total time spent looking at a sad face. Those variables recognized for anxiety were LF/HF, heart rate (HR), and pNN50. For childhood stress, HF, LF, and Jitter showed different correlation patterns between the two grade groups.

Discussion

We examined the potential of multimodal biomarkers in children, identifying features linked to childhood depression, particularly LF and the Sad.TF:time. Anxiety was most effectively explained by HRV features. To explore reasons for non-replication of previous studies, we categorized participants by elementary school grades into lower grades (1st, 2nd, 3rd) and upper grades (4th, 5th, 6th).

Conclusion

This study confirmed the potential use of multimodal digital biomarkers for children's mental health screening, serving as foundational research.

The fragility index: how robust are the outcomes of head and neck cancer randomised, controlled trials?

BACKGROUND

The fragility index represents the minimum number of patients required to convert an outcome from statistically significant to insignificant. This report assesses the fragility index of head and neck cancer randomised, controlled trials.

METHODS

Studies were extracted from PubMed/Medline, Scopus, Embase and Cochrane databases.

RESULTS

Overall, 123 randomised, controlled trials were included. The sample size and fragility index medians (interquartile ranges) were 103 (56-213) and 2 (0-5), respectively. The fragility index exceeded the number of patients lost to follow up in 42.3 per cent (n = 52) of studies. A higher fragility index correlated with higher sample size (r = 0.514, p < 0.001), number of events (r = 0.449, p < 0.001) and statistical significance via p-value (r = -0.367, p < 0.001).

CONCLUSION

Head and neck cancer randomised, controlled trials demonstrated low fragility index values, in which statistically significant results could be nullified by altering the outcomes of just two patients, on average. Future head and neck oncology randomised, controlled trials should report the fragility index in order to provide insight into statistical robustness.

Testing delayed, gradual, and temporary treatment effects in randomized single-case experiments: A general response function framework

Randomization tests represent a class of significance tests to assess the statistical significance of treatment effects in randomized single-case experiments. Most applications of single-case randomization tests concern simple treatment effects: immediate, abrupt, and permanent changes in the level of the outcome variable. However, researchers are confronted with delayed, gradual, and temporary treatment effects; in general, with "response functions" that are markedly different from single-step functions. We here introduce a general framework that allows specifying a test statistic for a randomization test based on predicted response functions that is sensitive to a wide variety of data patterns beyond immediate and sustained changes in level: different latencies (degrees of delay) of effect, abrupt versus gradual effects, and different durations of the effect (permanent or temporary). There may be reasonable expectations regarding the kind of effect (abrupt or gradual), entailing a different focal data feature (e.g., level or slope). However, the exact amount of latency and the exact duration of a temporary effect may not be known a priori, justifying an exploratory approach studying the effect of specifying different latencies or delayed effects and different durations for temporary effects. We provide illustrations of the proposal with real data, and we present a user-friendly freely available web application implementing it.

Assumption-checking rather than (just) testing: The importance of visualization and effect size in statistical diagnostics

Statistical methods generally have assumptions (e.g., normality in linear regression models). Violations of these assumptions can cause various issues, like statistical errors and biased estimates, whose impact can range from inconsequential to critical. Accordingly, it is important to check these assumptions, but this is often done in a flawed way. Here, I first present a prevalent but problematic approach to diagnostics-testing assumptions using null hypothesis significance tests (e.g., the Shapiro-Wilk test of normality). Then, I consolidate and illustrate the issues with this approach, primarily using simulations. These issues include statistical errors (i.e., false positives, especially with large samples, and false negatives, especially with small samples), false binarity, limited descriptiveness, misinterpretation (e.g., of p-value as an effect size), and potential testing failure due to unmet test assumptions. Finally, I synthesize the implications of these issues for statistical diagnostics, and provide practical recommendations for improving such diagnostics. Key recommendations include maintaining awareness of the issues with assumption tests (while recognizing they can be useful), using appropriate combinations of diagnostic methods (including visualization and effect sizes) while recognizing their limitations, and distinguishing between testing and checking assumptions. Additional recommendations include judging assumption violations as a complex spectrum (rather than a simplistic binary), using programmatic tools that increase replicability and decrease researcher degrees of freedom, and sharing the material and rationale involved in the diagnostics.

Extraordinary Claims in the Literature on High-Intensity Interval Training (HIIT): I. Bonafide Scientific Revolution or a Looming Crisis of Replication and Credibility?

The literature on high-intensity interval training (HIIT) contains claims that, if true, could revolutionize the science and practice of exercise. This critical analysis examines two varieties of claims: (i) HIIT is effective in improving various indices of fitness and health, and (ii) HIIT is as effective as more time-consuming moderate-intensity continuous exercise. Using data from two recent systematic reviews as working examples, we show that studies in both categories exhibit considerable weaknesses when judged through the prism of fundamental statistical principles. Predominantly, small-to-medium effects are investigated in severely underpowered studies, thus greatly increasing the risk of both type I and type II errors of statistical inference. Studies in the first category combine the volatility of estimates associated with small samples with numerous dependent variables analyzed without consideration of the inflation of the type I error rate. Studies in the second category inappropriately use the p > 0.05 criterion from small studies to support claims of 'similar' or 'comparable' effects. It is concluded that the situation in the HIIT literature is reminiscent of the research climate that led to the replication crisis in psychology. As in psychology, this could be an opportunity to reform statistical practices in exercise science.

The "P"-Value: The Primary Alphabet of Research Revisited

Each research roves around the P value. A value less than 0.05 is considered to be statistically significant. Very few researchers are aware of the history, real-world significance, statistical insight, and in-depth criticism about this monumental alphabet of research. This article will provide detailed insight into the most common molecule of research which will be rewarding for the young students and researchers in the primary world of research. It is not a simple value; it is the longest and broadest description of research squeezed to a number for the ground level worker to the principal investigator. The present review will provide a detailed and unique insight into the P value which would be rewarding for the primary care physicians toward translating research into their clinical practice.

The epistemic and pragmatic function of dichotomous claims based on statistical hypothesis tests

Researchers commonly make dichotomous claims based on continuous test statistics. Many have branded the practice as a misuse of statistics and criticize scientists for the widespread application of hypothesis tests to tentatively reject a hypothesis (or not) depending on whether a p-value is below or above an alpha level. Although dichotomous claims are rarely explicitly defended, we argue they play an important epistemological and pragmatic role in science. The epistemological function of dichotomous claims consists in transforming data into quasibasic statements, which are tentatively accepted singular facts that can corroborate or falsify theoretical claims. This transformation requires a prespecified methodological decision procedure such as Neyman-Pearson hypothesis tests. From the perspective of methodological falsificationism these decision procedures are necessary, as probabilistic statements (e.g., continuous test statistics) cannot function as falsifiers of substantive hypotheses. The pragmatic function of dichotomous claims is to facilitate scrutiny and criticism among peers by generating contestable claims, a process referred to by Popper as “conjectures and refutations.” We speculate about how the surprisingly widespread use of a 5% alpha level might have facilitated this pragmatic function. Abandoning dichotomous claims, for example because researchers commonly misuse p-values, would sacrifice their crucial epistemic and pragmatic functions.

Case-based teaching of experimental design - contributions for meaningful learning

This article argues the need for education and training of researchers carrying out animal studies on the fundamentals of experimental design (ED), as a key means of improving the reliability and reproducibility of preclinical results. The current landscape in ED education in Europe is presented, and we make the case for dedicated tutor-guided teaching of ED. With less than a day dedicated to it in many courses effective techniques for communicating key issues are needed. We have developed two approaches that transfer to experimental design teaching the case-study, problem-solving techniques known to be effective in other fields. They use realistic research scenarios to provoke discussion and engage learning. In one the scenario is for group discussion or informal or formal assessment with subsequent tutor-led discussion of key points. For this each scenario needs a clear statement of the purpose of the research study, simplified text outlining the comparisons and procedures, and a statement of the outcome measure. In the other approach, the scenario is used with freely-available software with a good graphical output to explore the sizing of experiments and the use of both sexes. Trainee feedback and informal assessment show that these approaches can make for interesting and memorable sessions and offer a useful contribution to improvement in experimental design teaching so that it produces meaningful learning that can translate into better practice.

Editorial perspective: Bayesian statistical methods are useful for researchers in child and adolescent mental health

Bayesian statistical approaches offer nuanced, detailed, and intuitive analyses, even with small sample sizes. Although these qualities are highly relevant for researchers in child and adolescent mental health, Bayesian methods are still quite rarely employed. This editorial perspective will briefly describe what is different about Bayesian statistical methods, discuss some of the ways they may benefit research in our field, and provide an introduction to how Bayesian statistics are employed in practical research.

Statistical Analysis in the Presence of Spatial Autocorrelation: Selected Sampling Strategy Effects

Fundamental to most classical data collection sampling theory development is the random drawings assumption requiring that each targeted population member has a known sample selection (i.e., inclusion) probability. Frequently, however, unrestricted random sampling of spatially autocorrelated data is impractical and/or inefficient. Instead, randomly choosing a population subset accounts for its exhibited spatial pattern by utilizing a grid, which often provides improved parameter estimates, such as the geographic landscape mean, at least via its precision. Unfortunately, spatial autocorrelation latent in these data can produce a questionable mean and/or standard error estimate because each sampled population member contains information about its nearby members, a data feature explicitly acknowledged in model-based inference, but ignored in design-based inference. This autocorrelation effect prompted the development of formulae for calculating an effective sample size (i.e., the equivalent number of sample selections from a geographically randomly distributed population that would yield the same sampling error) estimate. Some researchers recently challenged this and other aspects of spatial statistics as being incorrect/invalid/misleading. This paper seeks to address this category of misconceptions, demonstrating that the effective geographic sample size is a valid and useful concept regardless of the inferential basis invoked. Its spatial statistical methodology builds upon the preceding ingredients.

Replication concerns in sports and exercise science: a narrative review of selected methodological issues in the field

Known methodological issues such as publication bias, questionable research practices and studies with underpowered designs are known to decrease the replicability of study findings. The presence of such issues has been widely established across different research fields, especially in psychology. Their presence raised the first concerns that the replicability of study findings could be low and led researchers to conduct large replication projects. These replication projects revealed that a significant portion of original study findings could not be replicated, giving rise to the conceptualization of the replication crisis. Although previous research in the field of sports and exercise science has identified the first warning signs, such as an overwhelming proportion of significant findings, small sample sizes and lack of data availability, their possible consequences for the replicability of our field have been overlooked. We discuss the consequences of the above issues on the replicability of our field and offer potential solutions to improve replicability.

Better Inference in Neuroscience: Test Less, Estimate More

Null-hypothesis significance testing (NHST) has become the main tool of inference in neuroscience, and yet evidence suggests we do not use this tool well: tests are often planned poorly, conducted unfairly, and interpreted invalidly. This editorial makes the case that in addition to reforms to increase rigor we should test less, reserving NHST for clearly confirmatory contexts in which the researcher has derived a quantitative prediction, can provide the inputs needed to plan a quality test, and can specify the criteria not only for confirming their hypothesis but also for rejecting it. A reduction in testing would be accompanied by an expansion of the use of estimation [effect sizes and confidence intervals (CIs)]. Estimation is more suitable for exploratory research, provides the inputs needed to plan strong tests, and provides important contexts for properly interpreting tests.

More value from less food? Effects of epicurean labeling on moderate eating in the United States and in France

Emerging research has shown that sensory-based interventions (e.g., inviting people to mindfully focus on the multisensory aspects of eating) can be a viable alternative to nutrition-based interventions (e.g., nutrition labeling) to encourage moderate eating. We contribute to this literature in two ways. First, we propose a novel and simple sensory-based intervention to increase the appeal of moderate food portions in commercial settings, epicurean labeling, which consists in emphasizing the aesthetic, multisensory properties of the food when describing it on menus or packages. Second, we show theory-relevant cross-cultural differences in the effectiveness of this intervention between the United States and France, two food cultures at the opposite ends of the hedonic-utilitarian food attitude spectrum. We report the results of a multi-day field experiment at a French cafeteria showing that epicurean labeling, unlike nutrition labeling, reduces intake while increasing the perceived monetary value of the meal thanks to higher savoring. We then show in a matched cross-national online experiment that epicurean labeling is more effective in France than in the United States. We provide additional evidence of this cross-cultural variation in a study of 9154 food products sold in supermarkets in both countries. We find that epicurean labeling is more prevalent, but also more likely to be associated with smaller portions in France than in the United States. While sensory-based interventions are a promising alternative to nutrition-based interventions, it is necessary to develop business-friendly interventions that can be implemented in everyday life, as well as to consider cultural factors that can modulate their effectiveness.

Questionable Research Practices, Low Statistical Power, and Other Obstacles to Replicability: Why Preclinical Neuroscience Research Would Benefit from Registered Reports

Replicability, the degree to which a previous scientific finding can be repeated in a distinct set of data, has been considered an integral component of institutionalized scientific practice since its inception several hundred years ago. In the past decade, large-scale replication studies have demonstrated that replicability is far from favorable, across multiple scientific fields. Here, I evaluate this literature and describe contributing factors including the prevalence of questionable research practices (QRPs), misunderstanding of p-values, and low statistical power. I subsequently discuss how these issues manifest specifically in preclinical neuroscience research. I conclude that these problems are multifaceted and difficult to solve, relying on the actions of early and late career researchers, funding sources, academic publishers, and others. I assert that any viable solution to the problem of substandard replicability must include changing academic incentives, with adoption of registered reports being the most immediately impactful and pragmatic strategy. For animal research in particular, comprehensive reporting guidelines that document potential sources of sensitivity for experimental outcomes is an essential addition.

Money Does Not Always Buy Happiness, but Are Richer People Less Happy in Their Daily Lives? It Depends on How You Analyze Income

Do people who have more money feel happier during their daily activities? Some prior research has found no relationship between income and daily happiness when treating income as a continuous variable in OLS regressions, although results differ between studies. We re-analyzed existing data from the United States and Germany, treating household income as a categorical variable and using lowess and spline regressions to explore nonlinearities. Our analyses reveal that these methodological decisions change the results and conclusions about the relationship between income and happiness. In American and German diary data from 2010 to 2015, results for the continuous treatment of income showed a null relationship with happiness, whereas the categorization of income showed that some of those with higher incomes reported feeling less happy than some of those with lower incomes. Lowess and spline regressions suggested null results overall, and there was no evidence of a relationship between income and happiness in Experience Sampling Methodology (ESM) data. Not all analytic approaches generate the same results, which may contribute to explaining discrepant results in existing studies about the correlates of happiness. Future research should be explicit about their approaches to measuring and analyzing income when studying its relationship with subjective well-being, ideally testing different approaches, and making conclusions based on the pattern of results across approaches.

Simple and statistically sound recommendations for analysing physical theories

Physical theories that depend on many parameters or are tested against data from many different experiments pose unique challenges to statistical inference. Many models in particle physics, astrophysics and cosmology fall into one or both of these categories. These issues are often sidestepped with statistically unsound ad hoc methods, involving intersection of parameter intervals estimated by multiple experiments, and random or grid sampling of model parameters. Whilst these methods are easy to apply, they exhibit pathologies even in low-dimensional parameter spaces, and quickly become problematic to use and interpret in higher dimensions. In this article we give clear guidance for going beyond these procedures, suggesting where possible simple methods for performing statistically sound inference, and recommendations of readily-available software tools and standards that can assist in doing so. Our aim is to provide any physicists lacking comprehensive statistical training with recommendations for reaching correct scientific conclusions, with only a modest increase in analysis burden. Our examples can be reproduced with the code publicly available at Zenodo.

Applied Bayesian Approaches for Research in Motor Neuron Disease

Statistical evaluation of empirical data is the basis of the modern scientific method. Available tools include various hypothesis tests for specific data structures, as well as methods that are used to quantify the uncertainty of an obtained result. Statistics are pivotal, but many misconceptions arise due to their complexity and difficult-to-acquire mathematical background. Even though most studies rely on a frequentist interpretation of statistical readouts, the application of Bayesian statistics has increased due to the availability of easy-to-use software suites and an increased outreach favouring this topic in the scientific community. Bayesian statistics take our prior knowledge together with the obtained data to express a degree of belief how likely a certain event is. Bayes factor hypothesis testing (BFHT) provides a straightforward method to evaluate multiple hypotheses at the same time and provides evidence that favors the null hypothesis or alternative hypothesis. In the present perspective, we show the merits of BFHT for three different use cases, including a clinical trial, basic research as well as a single case study. Here we show that Bayesian statistics is a viable addition of a scientist's statistical toolset, which can help to interpret data.

Don’t Be Fooled by Randomness: Valid p-Values for Single Molecule Microscopy

The human mind shows extraordinary capability at recognizing patterns, while at the same time tending to underestimate the natural scope of random processes. Taken together, this easily misleads researchers in judging whether the observed characteristics of their data are of significance or just the outcome of random effects. One of the best tools to assess whether observed features fall into the scope of pure randomness is statistical significance testing, which quantifies the probability to falsely reject a chosen null hypothesis. The central parameter in this context is the p-value, which can be calculated from the recorded data sets. In case of p-values smaller than the level of significance, the null hypothesis is rejected, otherwise not. While significance testing has found widespread application in many sciences including the life sciences, it is hardly used in (bio-)physics. We propose here that significance testing provides an important and valid addendum to the toolbox of quantitative (single molecule) biology. It allows to support a quantitative judgement (the hypothesis) about the data set with a probabilistic assessment. In this manuscript we describe ways for obtaining valid p-values in two selected applications of single molecule microscopy: (i) Nanoclustering in single molecule localization microscopy. Previously, we developed a method termed 2-CLASTA, which allows to calculate a valid p-value for the null hypothesis of an underlying random distribution of molecules of interest while circumventing overcounting issues. Here, we present an extension to this approach, yielding a single overall p-value for data pooled from multiple cells or experiments. (ii) Single molecule trajectories. Data from a single molecule trajectory are inherently correlated, thus prohibiting a direct analysis via conventional statistical tools. Here, we introduce a block permutation test, which yields a valid p-value for the analysis and comparison of single molecule trajectory data. We exemplify the approach based on FRET trajectories.

Towards better hypothesis tests in oxytocin research: Evaluating the validity of auxiliary assumptions

Various factors have been attributed to the inconsistent reproducibility of human oxytocin research in the cognitive and behavioral sciences. These factors include small sample sizes, a lack of pre-registered studies, and the absence of overarching theoretical frameworks that can account for oxytocin's effects over a broad range of contexts. While there have been efforts to remedy these issues, there has been very little systematic scrutiny of the role of auxiliary assumptions, which are claims that are not central for testing a hypothesis but nonetheless critical for testing theories. For instance, the hypothesis that oxytocin increases the salience of social cues is predicated on the assumption that intranasally administered oxytocin increases oxytocin levels in the brain. Without robust auxiliary assumptions, it is unclear whether a hypothesis testing failure is due to an incorrect hypothesis or poorly supported auxiliary assumptions. Consequently, poorly supported auxiliary assumptions can be blamed for hypothesis failure, thereby safeguarding theories from falsification. In this article, I will evaluate the body of evidence for key auxiliary assumptions in human behavioral oxytocin research in terms of theory, experimental design, and statistical inference, and highlight assumptions that require stronger evidence. Strong auxiliary assumptions will leave hypotheses vulnerable for falsification, which will improve hypothesis testing and consequently advance our understanding of oxytocin's role in cognition and behavior.

Learning to love the null

Children's behaviour and mental health has the power to surprise us, readers and authors of the Journal of Child Psychology and Psychiatry and laymen alike, if not for the endless variation among people, then for the ever-changing context in which they develop. The hypothetico-deductive method in combination with null-hypothesis significance testing has turned surprise into scientific knowledge. Null effects may in themselves also be surprising and informative, but appear less well represented in the literature. This editorial highlights emerging methodological practices for studying null effects in the most informative way.

Augmenting salivation, but not evaluations, through subliminal conditioning of eating-related words

Correlating eating-related words (CS) with positively valenced words (US+) may augment eating-associated motivational responses (e.g., preingestive salivation) with minimal CS knowledge. We tested this claim using a subliminal conditioning procedure, where CS and US were presented under subliminal and supraliminal visual conditions. Three groups of Brazilian undergraduates (N = 69) viewed eating-related words (CS) or their scrambled counterparts (non-CS) followed by positive (US+) or neutral (US-) words. A free-selection visibility check confirmed that subliminally presented CS and non-CS had not been detected by any group. Participants exposed to CS/US+ pairings produced significantly more saliva relative to participants exposed to CS/US- and non-CS/US+ pairings. Reliable induction of salivation, coupled with null outcomes across evaluation measures, suggests that affective information related to eating can subliminally augment preingestive salivation with minimal deliberation.

Statistical Inferences Using Effect Sizes in Human Endothelial Function Research

INTRODUCTION

Magnitudes of change in endothelial function research can be articulated using effect size statistics. Effect sizes are commonly used in reference to Cohen's seminal guidelines of small (d = 0.2), medium (d = 0.5), and large (d = 0.8). Quantitative analyses of effect size distributions across various research disciplines have revealed values differing from Cohen's original recommendations. Here we examine effect size distributions in human endothelial function research, and the magnitude of small, medium, and large effects for macro and microvascular endothelial function.

METHODS

Effect sizes reported as standardized mean differences were extracted from meta research available for endothelial function. A frequency distribution was constructed to sort effect sizes. The 25th, 50th, and 75th percentiles were used to derive small, medium, and large effects. Group sample sizes and publication year from primary studies were also extracted to observe any potential trends, related to these factors, in effect size reporting in endothelial function research.

RESULTS

Seven hundred fifty-two effect sizes were extracted from eligible meta-analyses. We determined small (d = 0.28), medium (d = 0.69), and large (d = 1.21) effects for endothelial function that corresponded to the 25th, 50th, and 75th percentile of the data distribution.

CONCLUSION

Our data indicate that direct application of Cohen's guidelines would underestimate the magnitude of effects in human endothelial function research. This investigation facilitates future a priori power analyses, provides a practical guiding benchmark for the contextualization of an effect when no other information is available, and further encourages the reporting of effect sizes in endothelial function research.