No relationship between researcher impact and replication effect: an analysis of five studies with 100 replications

High replicability of newly discovered social-behavioural findings is achievable

Failures to replicate evidence of new discoveries have forced scientists to ask whether this unreliability is due to suboptimal implementation of methods or whether presumptively optimal methods are not, in fact, optimal. This paper reports an investigation by four coordinated laboratories of the prospective replicability of 16 novel experimental findings using rigour-enhancing practices: confirmatory tests, large sample sizes, preregistration and methodological transparency. In contrast to past systematic replication efforts that reported replication rates averaging 50%, replication attempts here produced the expected effects with significance testing (P < 0.05) in 86% of attempts, slightly exceeding the maximum expected replicability based on observed effect sizes and sample sizes. When one lab attempted to replicate an effect discovered by another lab, the effect size in the replications was 97% that in the original study. This high replication rate justifies confidence in rigour-enhancing methods to increase the replicability of new discoveries.

Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies

Terror management theory postulates that mortality salience (MS) increases the motivation to defend one's cultural worldviews. How that motivation is expressed may depend on the social norm that is momentarily salient. Meta-analyses were conducted on studies that manipulated MS and social norm salience. Results based on 64 effect sizes for the hypothesized interaction between MS and norm salience revealed a small-to-medium effect of g = 0.34, 95% confidence interval [0.26, 0.41]. Bias-adjustment techniques suggested the presence of publication bias and/or the exploitation of researcher degrees of freedom and arrived at smaller effect size estimates for the hypothesized interaction, in several cases reducing the effect to nonsignificance (range g_corrected = -0.36 to 0.15). To increase confidence in the idea that MS and norm salience interact to influence behavior, preregistered, high-powered experiments using validated norm salience manipulations are necessary. Concomitantly, more specific theorizing is needed to identify reliable boundary conditions of the effect.

Replicability in psychological research: a reflection

Background: In recent years, psychological science has suffered a crisis of confidence that has been marked by the low rate of replicability demonstrated in collaborative projects that attempted to quantify this problem, evidencing the difficulty in making replications and the existence of a possible excess of false positives published in the scientific literature. Method: This opinion article aimed to review the panorama of the replicability crisis in psychology, as well as its possible causes. Conclusions: It began from the state of the replicability crisis, then some possible causes and their repercussions on the advancement of psychological science were highlighted, discussing various associated issues, such as individual biases on the part of researchers, the lack of incentives to replicability studies and the priority standards that journals would currently have for novel and positive studies. Finally, the existing alternatives to reverse this situation are mentioned, among them the opening to new statistical approaches, the restructuring of incentives, and the development of editorial policies that facilitate the means for replication.