A Frequentist Alternative to Significance Testing, p-Values, and Confidence Intervals

Inferential Statistics Is an Unfit Tool for Interpreting Data

Null hypothesis significance testing is a commonly used tool for making statistical inferences in empirical studies, but its use has always been controversial. In this manuscript, I argue that even more problematic is that significance testing, and other abstract statistical benchmarks, often are used as tools for interpreting study data. This is problematic because interpreting data requires domain knowledge of the scientific topic and sensitivity to the study context, something that significance testing and other purely statistical approaches are not. By using simple examples, I demonstrate that researchers must first use their domain knowledge—professional expertise, clinical experience, practical insight—to interpret the data in their study and then use inferential statistics to provide some reasonable estimates about what can be generalized from the study data. Moving beyond the current focus on abstract statistical benchmarks will encourage researchers to measure their phenomena in more meaningful ways, transparently convey their data, and communicate their intellectual reasons for interpreting the data as they do, a shift that will better foster a scientific forum for cumulative science.

Statistical Assumptions and Reproducibility in Psychology: Data Mining Based on Open Science

The failures of reproducibility in psychology (or other social sciences) can be investigated by tracing their logical chains, from statistical hypothesis to their conclusion. This research starts with the normality hypothesis, the homoscedasticity hypothesis, and the robust hypothesis and uses the R language to simulate and analyze the original data of 100 studies in Estimating the Reproducibility of Psychological Science to explore the influence of the premise hypothesis on statistical methods on the reproducibility of psychological research. The results indicated the following: (1) the answer to the question about psychological studies being repeatable or not relates to the fields to which the subjects belonged, (2) not all the psychological variables meet the normal distribution hypothesis, (3) the t-test is a more robust tool for psychological research than the Analysis of Variance (ANOVA), and (4) the robustness of ANOVA is independent of the normality and variance congruence of the analyzed data. This study made us realize that the repeatable study factors in psychology are more complex than we expected them to be.

Beyond p < .05: a critical review of new Bayesian proposals for assessing the p-value

This paper reviews recent contributions from a Bayesian-oriented perspective, after the ASA statement on p-values (2016). We classify proposals that (i) supplement the p-value; (ii) modify the p-value itself. In the first group, we review the Bayes factor, the False Positive risk, the rejection odds and the analysis of credibility from both Matthews' and Held's point of view. We also put forth and discuss a new index of credibility, about which we conduct a delimited simulation study. In the second group, we discuss Gannon's modification of the p-value based on the Bayes factor and the second-generation p-value. The theory is illustrated with two case studies on pharmacotherapy in infectious diseases. Contemporary authors still refer to the p-value as a statistical indicator but have abandoned the perspective of evaluating p-values with fixed thresholds. Statistical societies worldwide should target new strategies to disseminate the debate on p-values in all applied fields of knowledge, as well as they may promote the use of different statistical procedures to supplement p-values.

Lessons Learned About Designing and Conducting Studies From HRI Experts

The field of human-robot interaction (HRI) research is multidisciplinary and requires researchers to understand diverse fields including computer science, engineering, informatics, philosophy, psychology, and more disciplines. However, it is hard to be an expert in everything. To help HRI researchers develop methodological skills, especially in areas that are relatively new to them, we conducted a virtual workshop, Workshop Your Study Design (WYSD), at the 2021 International Conference on HRI. In this workshop, we grouped participants with mentors, who are experts in areas like real-world studies, empirical lab studies, questionnaire design, interview, participatory design, and statistics. During and after the workshop, participants discussed their proposed study methods, obtained feedback, and improved their work accordingly. In this paper, we present 1) Workshop attendees’ feedback about the workshop and 2) Lessons that the participants learned during their discussions with mentors. Participants’ responses about the workshop were positive, and future scholars who wish to run such a workshop can consider implementing their suggestions. The main contribution of this paper is the lessons learned section, where the workshop participants contributed to forming this section based on what participants discovered during the workshop. We organize lessons learned into themes of 1) Improving study design for HRI, 2) How to work with participants - especially children -, 3) Making the most of the study and robot’s limitations, and 4) How to collaborate well across fields as they were the areas of the papers submitted to the workshop. These themes include practical tips and guidelines to assist researchers to learn about fields of HRI research with which they have limited experience. We include specific examples, and researchers can adapt the tips and guidelines to their own areas to avoid some common mistakes and pitfalls in their research.

Technical Analysis of Tourism Price Process in the Eurozone

This study is a specific contribution to investigating normalities in prices to a well-established cointegrated vector autoregressive model (VAR). While the role of prices in computational economics has been investigated, the real prices vis-à-vis nominal prices in the decision process has been neglected. The paper investigates the transition from nominal to real time-series of prices without losing information in the data set when deflating or de-seasonalizing. The likelihood approach is based on careful specifications of the (co)integration characteristics of tourism prices. The results confirm that the transmission of tourism prices in the Eurozone positively impacts Slovenian tourism prices when the spatial consolidated cointegrated VAR model is used. The theoretical-conceptual and empirical contribution is twofold: first, the study develops and empirically applies bona fide divisor of normality consolidation for time-series in levels instead of routinely utilised inflation integers, and second, the study introduces perfection of prices on a long-run time-series treatment.

Why we habitually engage in null-hypothesis significance testing: A qualitative study

Background

Null Hypothesis Significance Testing (NHST) is the most familiar statistical procedure for making inferences about population effects. Important problems associated with this method have been addressed and various alternatives that overcome these problems have been developed. Despite its many well-documented drawbacks, NHST remains the prevailing method for drawing conclusions from data. Reasons for this have been insufficiently investigated. Therefore, the aim of our study was to explore the perceived barriers and facilitators related to the use of NHST and alternative statistical procedures among relevant stakeholders in the scientific system.

Methods

Individual semi-structured interviews and focus groups were conducted with junior and senior researchers, lecturers in statistics, editors of scientific journals and program leaders of funding agencies. During the focus groups, important themes that emerged from the interviews were discussed. Data analysis was performed using the constant comparison method, allowing emerging (sub)themes to be fully explored. A theory substantiating the prevailing use of NHST was developed based on the main themes and subthemes we identified.

Results

Twenty-nine interviews and six focus groups were conducted. Several interrelated facilitators and barriers associated with the use of NHST and alternative statistical procedures were identified. These factors were subsumed under three main themes: the scientific climate, scientific duty, and reactivity. As a result of the factors, most participants feel dependent in their actions upon others, have become reactive, and await action and initiatives from others. This may explain why NHST is still the standard and ubiquitously used by almost everyone involved.

Conclusion

Our findings demonstrate how perceived barriers to shift away from NHST set a high threshold for actual behavioral change and create a circle of interdependency between stakeholders. By taking small steps it should be possible to decrease the scientific community’s strong dependence on NHST and p-values.

Psychological Determinants of COVID-19 Vaccine Acceptance among Healthcare Workers in Kuwait: A Cross-Sectional Study Using the 5C and Vaccine Conspiracy Beliefs Scales

Acceptance of coronavirus disease 2019 (COVID-19) vaccination appears as a decisive factor necessary to control the ongoing pandemic. Healthcare workers (HCWs) are among the highest risk groups for infection. The current study aimed to evaluate COVID-19 vaccine acceptance among HCWs in Kuwait, with identification of the psychological determinants of COVID-19 vaccine hesitancy. The study was conducted using an online anonymous survey distributed between 18 March 2021 and 29 March 2021. The sampling strategy was convenience-based depending on chain-referral sampling. Psychological determinants of COVID-19 vaccine acceptance were assessed using the 5C subscales and the Vaccine Conspiracy Beliefs Scale (VCBS). The total number of study participants was 1019, with the largest group being physicians (28.7%), pharmacists (20.2%), dentists (16.7%), and nurses (12.5%). The overall rate for COVID-19 vaccine acceptance was 83.3%, with 9.0% who were not willing to accept vaccination and 7.7% who were unsure. The highest rate for COVID-19 vaccine acceptance was seen among dentists (91.2%) and physicians (90.4%), while the lowest rate was seen among nurses (70.1%; p < 0.001). A higher level of COVID-19 vaccine hesitancy was found among females, participants with a lower educational level, and HCWs in the private sector. A preference for mRNA vaccine technology and Pfizer-BioNTech COVID-19 vaccine was found among the majority of participants (62.6% and 69.7%, respectively). COVID-19 vaccine hesitancy was significantly linked to the embrace of vaccine conspiracy beliefs. The highest 5C psychological predictors of COVID-19 vaccine acceptance were high levels of collective responsibility and confidence, and lower levels of constraints and calculation. The VCBS and 5C subscales (except the calculation subscale) showed acceptable levels of predicting COVID-19 vaccine acceptance based on receiver operating characteristic analyses. The participants who depended on social media platforms, TV programs, and news releases as their main sources of knowledge about COVID-19 vaccines showed higher rates of COVID-19 vaccine hesitancy. An overall satisfactory level of COVID-19 vaccine acceptance was seen among HCWs in Kuwait, which was among the highest rates reported globally. However; higher levels of vaccine hesitancy were observed among certain groups (females, nurses and laboratory workers, HCWs in the private sector), which should be targeted with more focused awareness programs. HCWs in Kuwait can play a central role in educating their patients and the general public about the benefits of COVID-19 vaccination to halt the spread of SARS-CoV-2, considering the high rates of vaccine hesitancy observed among the general public in Kuwait and the Middle East.

Persons as Effect Sizes

Traditional indices of effect size are designed to answer questions about average group differences, associations between variables, and relative risk. For many researchers, an additional, important question is, “How many people in my study behaved or responded in a manner consistent with theoretical expectation?” We show how the answer to this question can be computed and reported as a straightforward percentage for a wide variety of study designs. This percentage essentially treats persons as an effect size, and it can easily be understood by scientists, professionals, and laypersons alike. For instance, imagine that in addition to d or η2, a researcher reports that 80% of participants matched theoretical expectation. No statistical training is required to understand the basic meaning of this percentage. By analyzing recently published studies, we show how computing this percentage can reveal novel patterns within data that provide insights for extending and developing the theory under investigation.

Replication and the Establishment of Scientific Truth

The idea of replication is based on the premise that there are empirical regularities or universal laws to be replicated and verified, and the scientific method is adequate for doing it. Scientific truth, however, is not absolute but relative to time, context, and the method used. Time and context are inextricably intertwined in that time (e.g., Christmas Day vs. New Year's Day) creates different contexts for behaviors and contexts create different experiences of time, rendering psychological phenomena inherently variable. This means that internal and external conditions fluctuate and are different in a replication study vs. the original. Thus, a replication experiment is just another empirical investigation in an ongoing effort to establish scientific truth. Neither the original nor a replication is the final arbiter of whether or not something exists. Discovered patterns need not be permanent laws of human behavior proven by the pinpoint statistical verification through replication. To move forward, phenomenon replications are needed to investigate phenomena in different ways, forms, contexts, and times. Such investigations look at phenomena not just in terms the magnitude of their effects but also by their frequency, duration, and intensity in labs and real life. They will also shed light on the extent to which lab manipulations may make many phenomena subjectively conscious events and effects (e.g., causal attributions) when they are nonconsciously experienced in real life, or vice versa. As scientific knowledge in physics is temporary and incomplete, should it be any surprise that science can only provide "temporary winners" for psychological knowledge of human behavior?

The inaccuracy of sample-based confidence intervals to estimate a priori ones

Confidence intervals (CIs) constitute the most popular alternative to widely criticized null hypothesis significance tests. CIs provide more information than significance tests and lend themselves well to visual displays. Although CIs are no better than significance tests when used solely as significance tests, researchers need not limit themselves to this use of CIs. Rather, CIs can be used to estimate the precision of the data, and it is the precision argument that may set CIs in a superior position to significance tests. We tested two versions of the precision argument by performing computer simulations to test how well sample-based CIs estimate a priori CIs. One version pertains to precision of width whereas the other version pertains to precision of location. Using both versions, sample-based CIs poorly estimate a priori CIs at typical sample sizes and perform better as sample sizes increase.

Bayesian Estimation with Informative Priors is Indistinguishable from Data Falsification

Criticism of null hypothesis significance testing, confidence intervals, and frequentist statistics in general has evolved into advocacy of Bayesian analyses with informative priors for strong inference. This paper shows that Bayesian analysis with informative priors is formally equivalent to data falsification because the information carried by the prior can be expressed as the addition of fabricated observations whose statistical characteristics are determined by the parameters of the prior. This property of informative priors makes clear that only the use of non-informative, uniform priors in all types of Bayesian analyses is compatible with standards of research integrity. At the same time, though, Bayesian estimation with uniform priors yields point and interval estimates that are identical or nearly identical to those obtained with frequentist methods. At a qualitative level, frequentist and Bayesian outcomes have different interpretations but they are interchangeable when uniform priors are used. Yet, Bayesian interpretations require either the assumption that population parameters are random variables (which they are not) or an explicit acknowledgment that the posterior distribution (which is thus identical to the likelihood function except for a scale factor) only expresses the researcher's beliefs and not any information about the parameter of concern.