Most of the time, P is an unreliable marker, so we need no exact cut-off

Effects of Shinrin-Yoku Retreat on Mental Health: a Pilot Study in Fukushima, Japan

Shinrin-yoku (forest bathing) is a cost-effective healing practice that has recently attracted the interest of social scientists who have attributed it, in part, to mental health benefits. Japanese university students suffer from high rates of mental health problems, and the number of suicides remains high despite the total number of suicides in Japan decreasing. Effective mental health approaches which increase mental wellbeing and self-compassion, and reduce associated deficits, such as loneliness, are sought after for Japanese students; however, healthful treatment has not been identified to date. Accordingly, this pre-post-pilot study evaluated the levels of mental wellbeing, self-compassion, and loneliness among 25 Japanese undergraduate students who participated in a three-day shinrin-yoku retreat in Fukushima. Measurements were taken prior, straight after, and two weeks post-intervention. One-way ANOVA with Tukey post hoc analysis revealed that the mean scores of self-compassion, common humanity, and mindfulness increased statistically significantly from pre-retreat to follow-up. The mean scores of mental wellbeing and loneliness did not statistically significantly change. The positive effects on self-compassion indicate that shinrin-yoku retreat should be evaluated within a larger sample and in a shorter time frame to establish optimal shinrin-yoku parameters in this arena.

The earth is flat (p > 0.05): significance thresholds and the crisis of unreplicable research

The widespread use of 'statistical significance' as a license for making a claim of a scientific finding leads to considerable distortion of the scientific process (according to the American Statistical Association). We review why degrading p-values into 'significant' and 'nonsignificant' contributes to making studies irreproducible, or to making them seem irreproducible. A major problem is that we tend to take small p-values at face value, but mistrust results with larger p-values. In either case, p-values tell little about reliability of research, because they are hardly replicable even if an alternative hypothesis is true. Also significance (p ≤ 0.05) is hardly replicable: at a good statistical power of 80%, two studies will be 'conflicting', meaning that one is significant and the other is not, in one third of the cases if there is a true effect. A replication can therefore not be interpreted as having failed only because it is nonsignificant. Many apparent replication failures may thus reflect faulty judgment based on significance thresholds rather than a crisis of unreplicable research. Reliable conclusions on replicability and practical importance of a finding can only be drawn using cumulative evidence from multiple independent studies. However, applying significance thresholds makes cumulative knowledge unreliable. One reason is that with anything but ideal statistical power, significant effect sizes will be biased upwards. Interpreting inflated significant results while ignoring nonsignificant results will thus lead to wrong conclusions. But current incentives to hunt for significance lead to selective reporting and to publication bias against nonsignificant findings. Data dredging, p-hacking, and publication bias should be addressed by removing fixed significance thresholds. Consistent with the recommendations of the late Ronald Fisher, p-values should be interpreted as graded measures of the strength of evidence against the null hypothesis. Also larger p-values offer some evidence against the null hypothesis, and they cannot be interpreted as supporting the null hypothesis, falsely concluding that 'there is no effect'. Information on possible true effect sizes that are compatible with the data must be obtained from the point estimate, e.g., from a sample average, and from the interval estimate, such as a confidence interval. We review how confusion about interpretation of larger p-values can be traced back to historical disputes among the founders of modern statistics. We further discuss potential arguments against removing significance thresholds, for example that decision rules should rather be more stringent, that sample sizes could decrease, or that p-values should better be completely abandoned. We conclude that whatever method of statistical inference we use, dichotomous threshold thinking must give way to non-automated informed judgment.