The evidence interval and the Bayesian evidence value: On a unified theory for Bayesian hypothesis testing and interval estimation

Interval estimation is one of the most frequently used methods in statistical science, employed to provide a range of credible values a parameter is located in after taking into account the uncertainty in the data. However, while this interpretation only holds for Bayesian interval estimates, these suffer from two problems. First, Bayesian interval estimates can include values which have not been corroborated by observing the data. Second, Bayesian interval estimates and hypothesis tests can yield contradictory conclusions. In this paper a new theory for Bayesian hypothesis testing and interval estimation is presented. A new interval estimate is proposed, the Bayesian evidence interval, which is inspired by the Pereira-Stern theory of the full Bayesian significance test (FBST). It is shown that the evidence interval is a generalization of existing Bayesian interval estimates, that it solves the problems of standard Bayesian interval estimates and that it unifies Bayesian hypothesis testing and parameter estimation. The Bayesian evidence value is introduced, which quantifies the evidence for the (interval) null and alternative hypothesis. Based on the evidence interval and the evidence value, the (full) Bayesian evidence test (FBET) is proposed as a new, model-independent Bayesian hypothesis test. Additionally, a decision rule for hypothesis testing is derived which shows the relationship to a widely used decision rule based on the region of practical equivalence and Bayesian highest posterior density intervals and to the e-value in the FBST. In summary, the proposed method is universally applicable, computationally efficient, and while the evidence interval can be seen as an extension of existing Bayesian interval estimates, the FBET is a generalization of the FBST and contains it as a special case. Together, the theory developed provides a unification of Bayesian hypothesis testing and interval estimation and is made available in the R package fbst.

Statistical significance and its critics: practicing damaging science, or damaging scientific practice?

While the common procedure of statistical significance testing and its accompanying concept of p-values have long been surrounded by controversy, renewed concern has been triggered by the replication crisis in science. Many blame statistical significance tests themselves, and some regard them as sufficiently damaging to scientific practice as to warrant being abandoned. We take a contrary position, arguing that the central criticisms arise from misunderstanding and misusing the statistical tools, and that in fact the purported remedies themselves risk damaging science. We argue that banning the use of p-value thresholds in interpreting data does not diminish but rather exacerbates data-dredging and biasing selection effects. If an account cannot specify outcomes that will not be allowed to count as evidence for a claim-if all thresholds are abandoned-then there is no test of that claim. The contributions of this paper are: To explain the rival statistical philosophies underlying the ongoing controversy; To elucidate and reinterpret statistical significance tests, and explain how this reinterpretation ameliorates common misuses and misinterpretations; To argue why recent recommendations to replace, abandon, or retire statistical significance undermine a central function of statistics in science: to test whether observed patterns in the data are genuine or due to background variability.

Confidence Distribution for the Ability Parameter of the Rasch Model

In this paper, we consider the Rasch model and suggest novel point estimators and confidence intervals for the ability parameter. They are based on a proposed confidence distribution (CD) whose construction has required to overcome some difficulties essentially due to the discrete nature of the model. When the number of items is large, the computations due to the combinatorics involved become heavy, and thus, we provide first- and second-order approximations of the CD. Simulation studies show the good behavior of our estimators and intervals when compared with those obtained through other standard frequentist and weakly informative Bayesian procedures. Finally, using the expansion of the expected length of the suggested interval, we are able to identify reasonable values of the sample size which lead to a desired length of the interval.