Open Science Practices at the Journal of Traumatic Stress

This editorial describes new initiatives designed to promote and maintain open science practices (OSP) at the Journal of Traumatic Stress, to be enacted beginning January 2020. Following a brief description of the rationale underlying the argument for conducting and reporting research in ways that maximize transparency and replicability, this article summarizes changes in Journal submission and publication procedures that are designed to foster and highlight such practices. These include requesting an Open Science Practices Statement from authors of all accepted manuscripts, which will be published as supplementary material for each article, and providing authors with the opportunity to earn OSP badges for preregistering studies, making data available to other researchers by posting on a third party archive, and making available research materials and codes used in the study.

Functional magnetic resonance imaging (fMRI) item analysis of empathy and theory of mind

In contrast to conventional functional magnetic resonance imaging (fMRI) analysis across participants, item analysis allows generalizing the observed neural response patterns from a specific stimulus set to the entire population of stimuli. In the present study, we perform an item analysis on an fMRI paradigm (EmpaToM) that measures the neural correlates of empathy and Theory of Mind (ToM). The task includes a large stimulus set (240 emotional vs. neutral videos to probe empathic responding and 240 ToM or factual reasoning questions to probe ToM), which we tested in two large participant samples (N = 178, N = 130). Both, the empathy‐related network comprising anterior insula, anterior cingulate/dorsomedial prefrontal cortex, inferior frontal gyrus, and dorsal temporoparietal junction/supramarginal gyrus (TPJ) and the ToM related network including ventral TPJ, superior temporal gyrus, temporal poles, and anterior and posterior midline regions, were observed across participants and items. Regression analyses confirmed that these activations are predicted by the empathy or ToM condition of the stimuli, but not by low‐level features such as video length, number of words, syllables or syntactic complexity. The item analysis also allowed for the selection of the most effective items to create optimized stimulus sets that provide the most stable and reproducible results. Finally, reproducibility was shown in the replication of all analyses in the second participant sample. The data demonstrate (a) the generalizability of empathy and ToM related neural activity and (b) the reproducibility of the EmpaToM task and its applicability in intervention and clinical imaging studies.

Improving the Rigor and Replicability of Applied Psychophysiology Research: Sample Size, Standardization, Transparency, and Preregistration

Scientific research across a number of areas, including applied psychophysiology, biofeedback, and neurofeedback, is facing considerable scrutiny for poor replication rates, high numbers of false-positive findings, and insufficient scientific rigor. There are many factors underlying this replication crisis in scientific research; yet incentives for more rigorous research practices at the institutional and editorial levels lag behind the need for improvement. The author provides examples of replication and rigor difficulties in scientific research with an eye toward psychophysiological research, including researcher flexibility in data analysis, “p-hacking,” insufficient sample sizes, and lack of availability and implementation of rigorous methodological and publication guidelines. Subsequently, the author highlights examples and opportunities for improvement, including decreasing researcher flexibility, reporting sample size information, increasing sample sizes through collaboration, improving reporting standards/following established guidelines for reporting psychophysiological data, and increasing adoption of preregistration and registered reports. The author concludes that the Association for Applied Psychophysiology and Biofeedback (AAPB) can improve clinical practice and perception of public and scientific credibility by implementing rigorous and transparent research practices with a focus on replicability and clear methodological and reporting techniques and standards.

Using the landmark task to test the underlying assumptions of unilateral contraction research

The unilateral contractions procedure (i.e., squeezing a ball with one hand) supposedly enhances a wide variety of cognitive functions, from episodic recall to choking under pressure. The practicality and affordability of this procedure makes it highly appealing. But does it work? We addressed this question by testing whether intermittent and sustained unilateral contractions shifted a well-supported hemispheric asymmetry: visuospatial attention. Based on prior research, contracting the left (or right) hand should lead baseline scores on the landmark task-a visuospatial attention measure-to deviate further left (or right). We meta-analysed the results of our six experiments and showed that the unilateral contractions procedure, particularly with intermittent contractions, does not reliably shift landmark task scores measured during (Experiments 4-6) or after (Experiments 1-3 & 6) performing unilateral contractions. Although we question if and how unilateral contractions activate the contralateral hemisphere, Experiment 6 provided some support for the utility of sustained contractions.

Predicting the replicability of social science lab experiments

We measure how accurately replication of experimental results can be predicted by black-box statistical models. With data from four large-scale replication projects in experimental psychology and economics, and techniques from machine learning, we train predictive models and study which variables drive predictable replication. The models predicts binary replication with a cross-validated accuracy rate of 70% (AUC of 0.77) and estimates of relative effect sizes with a Spearman ρ of 0.38. The accuracy level is similar to market-aggregated beliefs of peer scientists [1, 2]. The predictive power is validated in a pre-registered out of sample test of the outcome of [3], where 71% (AUC of 0.73) of replications are predicted correctly and effect size correlations amount to ρ = 0.25. Basic features such as the sample and effect sizes in original papers, and whether reported effects are single-variable main effects or two-variable interactions, are predictive of successful replication. The models presented in this paper are simple tools to produce cheap, prognostic replicability metrics. These models could be useful in institutionalizing the process of evaluation of new findings and guiding resources to those direct replications that are likely to be most informative.

Acquiring group bias: Observing other people's nonverbal signals can create social group biases

Evidence of group bias based on race, ethnicity, nationality, and language emerges early in the life span. Although understanding the initial acquisition of group bias has critical theoretical and practical implications, precisely how group biases are acquired has been understudied. In two preregistered experiments, we tested the hypothesis that generalized social group biases can be acquired through exposure to positive nonverbal signals directed toward a novel adult from one group and more negative nonverbal signals directed toward a novel adult from another group. We sought to determine whether children would acquire global nonverbal signal-consistent social group biases that extended beyond their explicit social preferences, by measuring children's preferences, imitation, and behavioral intentions. Supporting our preregistered hypotheses, preschool-age participants favored small and large groups whose member received positive nonverbal signals, relative to groups whose member received more negative nonverbal signals. We also replicated prior work indicating that children will acquire individual target biases from the observation of biased nonverbal signals. Here we make the case that generalized social group biases can be rapidly and unintentionally transmitted on the basis of observational learning from nonverbal signals. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Science or pseudoscience? A distinction that matters for police officers, lawyers and judges

Scientific knowledge has been a significant contributor to the development of better practices within law enforcement agencies. However, some alleged 'experts' have been shown to have disseminated information to police officers, lawyers and judges that is neither empirically tested nor supported by scientific theory. The aim of this article is to provide organisations within the justice system with an overview of a) what science is and is not; b) what constitutes an empirically driven, theoretically founded, peer-reviewed approach; and c) how to distinguish science from pseudoscience. Using examples in relation to non-verbal communication, this article aims to demonstrate that not all information which is presented as comprehensively evaluated is methodologically reliable for use in the justice system.

An overview of issues in infant and developmental research for the creation of robust and replicable science

In the last few years, the field of psychology has been challenged with a crisis in the rigor and reproducibility of the science. The focus of these issues has primarily been in social, cognitive, and cognitive neuroscience psychology, however, the area of developmental research is not immune to these issues. This paper provides an overview of the "replication crisis" and the choices made by researchers that are often not noted in methods, thus making the replication of studies more difficult. In this review we discuss issues of researcher flexibility in the data design and selection of sample size, collection, and analysis stages of research. In each of these areas we address examples of bias and how developmental researchers can address these issues in their own research.

How Many Participants Do We Have to Include in Properly Powered Experiments? A Tutorial of Power Analysis with Reference Tables

Given that an effect size of d = .4 is a good first estimate of the smallest effect size of interest in psychological research, we already need over 50 participants for a simple comparison of two within-participants conditions if we want to run a study with 80% power. This is more than current practice. In addition, as soon as a between-groups variable or an interaction is involved, numbers of 100, 200, and even more participants are needed. As long as we do not accept these facts, we will keep on running underpowered studies with unclear results. Addressing the issue requires a change in the way research is evaluated by supervisors, examiners, reviewers, and editors. The present paper describes reference numbers needed for the designs most often used by psychologists, including single-variable between-groups and repeated-measures designs with two and three levels, two-factor designs involving two repeated-measures variables or one between-groups variable and one repeated-measures variable (split-plot design). The numbers are given for the traditional, frequentist analysis with p < .05 and Bayesian analysis with BF > 10. These numbers provide researchers with a standard to determine (and justify) the sample size of an upcoming study. The article also describes how researchers can improve the power of their study by including multiple observations per condition per participant.

Changing institutional incentives to foster sound scientific practices: One department

Replicable research and open science are of value to our field and to society at large, but most universities provide no incentives to adopt these practices. Instead, current incentive structures favor novel research, which has led to a situation in which few researchers take the time to do replications, share protocols, or share data. Obviously, several approaches to remedy this situation are possible. However, little progress can be made if becoming involved in such activities reduces a researcher's chances of rank and status advancement and other rewards. I describe in this article the way my department has modified our incentive structure to tackle this problem, including how the changes influence my research as a developmental psychologist. Finally, I offer suggestions for faculty who wish to initiate similar changes in their institutions.

Improving Psychological Science through Transparency and Openness: An Overview

The ability to independently verify and replicate observations made by other researchers is a hallmark of science. In this article, we provide an overview of recent discussions concerning replicability and best practices in mainstream psychology with an emphasis on the practical benefists to both researchers and the field as a whole. We first review challenges individual researchers face in producing research that is both publishable and reliable. We then suggest methods for producing more accurate research claims, such as transparently disclosing how results were obtained and analyzed, preregistering analysis plans, and publicly posting original data and materials. We also discuss ongoing changes at the institutional level to incentivize stronger research. These include officially recognizing open science practices at the journal level, disconnecting the publication decision from the results of a study, training students to conduct replications, and publishing replications. We conclude that these open science practices afford exciting low-cost opportunities to improve the quality of psychological science.

Altered structural and functional brain network overall organization predict human intertemporal decision-making

Intertemporal decision-making is naturally ubiquitous to us: individuals always make a decision with different consequences occurring at different moments. These choices are invariably involved in life-changing outcomes regarding marriage, education, fertility, long-term well-being, and even public policy. Previous studies have clearly uncovered the neurobiological mechanism of the intertemporal decision in the schemes of regional location or sub-network. However, it still remains unclear how to characterize intertemporal behavior with multimodal whole-brain network metrics to date. Here, we combined diffusion tensor image and resting-state functional connectivity MRI technology, in conjunction with graph-theoretical analysis, to explore the link between topological properties of integrated structural and functional whole-brain networks and intertemporal decision-making. Graph-theoretical analysis illustrated that the participants with steep discounting rates exhibited the decreased global topological organizations including small-world and rich-club regimes in both functional and structural connectivity networks, and reflected the dreadful local topological dynamics in the modularity of functional connectome. Furthermore, in the cross-modalities configuration, the same relationship was predominantly observed for the coupling of structural-functional connectivity as well. Above topological metrics are commonly indicative of the communication pattern of simultaneous global and local parallel information processing, and it thus reshapes our accounts on intertemporal decision-making from functional regional/sub-network scheme to multimodal brain overall organization.

Perceptions and Expected Immediate Reactions to Severe Storm Displays

The National Weather Service has adopted warning polygons that more specifically indicate the risk area than its previous county-wide warnings. However, these polygons are not defined in terms of numerical strike probabilities (p_s ). To better understand people's interpretations of warning polygons, 167 participants were shown 23 hypothetical scenarios in one of three information conditions-polygon-only (Condition A), polygon + tornadic storm cell (Condition B), and polygon + tornadic storm cell + flanking nontornadic storm cells (Condition C). Participants judged each polygon's p_s and reported the likelihood of taking nine different response actions. The polygon-only condition replicated the results of previous studies; p_s was highest at the polygon's centroid and declined in all directions from there. The two conditions displaying storm cells differed from the polygon-only condition only in having p_s just as high at the polygon's edge nearest the storm cell as at its centroid. Overall, p_s values were positively correlated with expectations of continuing normal activities, seeking information from social sources, seeking shelter, and evacuating by car. These results indicate that participants make more appropriate p_s judgments when polygons are presented in their natural context of radar displays than when they are presented in isolation. However, the fact that p_s judgments had moderately positive correlations with both sheltering (a generally appropriate response) and evacuation (a generally inappropriate response) suggests that experiment participants experience the same ambivalence about these two protective actions as people threatened by actual tornadoes.

Extrapyramidal motor signs in older adults with HIV disease: frequency, 1-year course, and associations with activities of daily living and quality of life

Age and HIV disease have additive effects on neural systems that support motor functioning. The current study examined the combined impact of aging and HIV on extrapyramidal motor functions, which were hypothesized to influence on activities of daily living (ADLs) and quality of life (QoL). Participants included 336 adults classified by HIV serostatus and age. A research nurse administered the Unified Parkinson's Disease Rating Scale (UPDRS) and participants completed the modified Lawton & Brody ADL and the Short Form Survey Instrument (SF-36) questionnaires as part of a larger neuropsychological research battery. A convenience subset of 172 participants completed a 14-month follow-up evaluation. At baseline, only older age was associated with mild extrapyramidal signs; however, at 14-month follow-up, independent adverse effects of both HIV status and age group were observed on a 3-level UPDRS change variable. Among older HIV+ adults, the presence of mild UPDRS motor signs was independently associated with basic and instrumental ADL dependence, as well as lower physical (ps < .05), but not mental QoL. In the modern treatment era, older HIV+ adults show higher frequency of mild extrapyramidal signs as compared to younger individuals (but not older HIV- persons) and are at higher risk of incident extrapyramidal signs relative to HIV- persons (but not younger HIV+ persons). When present in older HIV+ adults, extrapyramidal signs are of mild severity but nevertheless increase the risk of daily functioning problems and lower health-related physical QoL.

Benchmark validation of statistical models: Application to mediation analysis of imagery and memory

This article describes benchmark validation, an approach to validating a statistical model. According to benchmark validation, a valid model generates estimates and research conclusions consistent with a known substantive effect. Three types of benchmark validation-(a) benchmark value, (b) benchmark estimate, and (c) benchmark effect-are described and illustrated with examples. Benchmark validation methods are especially useful for statistical models with assumptions that are untestable or very difficult to test. Benchmark effect validation methods were applied to evaluate statistical mediation analysis in eight studies using the established effect that increasing mental imagery improves recall of words. Statistical mediation analysis led to conclusions about mediation that were consistent with established theory that increased imagery leads to increased word recall. Benchmark validation based on established substantive theory is discussed as a general way to investigate characteristics of statistical models and a complement to mathematical proof and statistical simulation. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Limited Usefulness of Capture Procedure and Capture Percentage for Evaluating Reproducibility in Psychological Science

In psychological science, there is an increasing concern regarding the reproducibility of scientific findings. For instance, Replication Project: Psychology (Open Science Collaboration, 2015) found that the proportion of successful replication in psychology was 41%. This proportion was calculated based on Cumming and Maillardet (2006) widely employed capture procedure (CPro) and capture percentage (CPer). Despite the popularity of CPro and CPer, we believe that using them may lead to an incorrect conclusion of (a) successful replication when the population effect sizes in the original and replicated studies are different; and (b) unsuccessful replication when the population effect sizes in the original and replicated studies are identical but their sample sizes are different. Our simulation results show that the performances of CPro and CPer become biased, such that researchers can easily make a wrong conclusion of successful/unsuccessful replication. Implications of these findings are considered in the conclusion.

Attention to faces and gaze-following in social anxiety: preliminary evidence from a naturalistic eye-tracking investigation

Social attentional biases are a core component of social anxiety disorder, but research has not yet determined their direction due to methodological limitations. Here we present preliminary findings from a novel, dynamic eye-tracking paradigm allowing spatial-temporal measurement of attention and gaze-following, a mechanism previously unexplored in social anxiety. 105 participants took part, with those high (N = 27) and low (N = 25) in social anxiety traits (HSA and LSA respectively) entered into the analyses. Participants watched a video of an emotionally-neutral social scene, where two actors periodically shifted their gaze towards the periphery. HSA participants looked more at the actors' faces during the initial 2s than the LSA group but there were no group differences in proportion of first fixations to the face or latency to first fixate the face, although HSA individuals' first fixations to the face were shorter. No further differences in eye movements were found, nor in gaze-following behaviour, although these null effects could potentially result from the relatively small sample. Findings suggest attention is biased towards faces in HSA individuals during initial scene inspection, but that overt gaze-following may be impervious to individual differences in social anxiety. Future research should seek to replicate these effects.

Implications of the Credibility Revolution for Productivity, Creativity, and Progress

The credibility revolution (sometimes referred to as the “replicability crisis”) in psychology has brought about many changes in the standards by which psychological science is evaluated. These changes include (a) greater emphasis on transparency and openness, (b) a move toward preregistration of research, (c) more direct-replication studies, and (d) higher standards for the quality and quantity of evidence needed to make strong scientific claims. What are the implications of these changes for productivity, creativity, and progress in psychological science? These questions can and should be studied empirically, and I present my predictions here. The productivity of individual researchers is likely to decline, although some changes (e.g., greater collaboration, data sharing) may mitigate this effect. The effects of these changes on creativity are likely to be mixed: Researchers will be less likely to pursue risky questions; more likely to use a broad range of methods, designs, and populations; and less free to define their own best practices and standards of evidence. Finally, the rate of scientific progress—the most important shared goal of scientists—is likely to increase as a result of these changes, although one’s subjective experience of making progress will likely become rarer.

The Creativity-Verification Cycle in Psychological Science: New Methods to Combat Old Idols

Over the years, researchers in psychological science have documented and investigated a host of powerful cognitive fallacies, including hindsight bias and confirmation bias. Researchers themselves may not be immune to these fallacies and may unwittingly adjust their statistical analysis to produce an outcome that is more pleasant or better in line with prior expectations. To shield researchers from the impact of cognitive fallacies, several methodologists are now advocating preregistration-that is, the creation of a detailed analysis plan before data collection or data analysis. One may argue, however, that preregistration is out of touch with academic reality, hampering creativity and impeding scientific progress. We provide a historical overview to show that the interplay between creativity and verification has shaped theories of scientific inquiry throughout the centuries; in the currently dominant theory, creativity and verification operate in succession and enhance one another's effectiveness. From this perspective, the use of preregistration to safeguard the verification stage will help rather than hinder the generation of fruitful new ideas.

The Prior Odds of Testing a True Effect in Cognitive and Social Psychology

Efforts to increase replication rates in psychology generally consist of recommended improvements to methodology, such as increasing sample sizes to increase power or using a lower alpha level. However, little attention has been paid to how the prior odds ( R) that a tested effect is true can affect the probability that a significant result will be replicable. The lower R is, the less likely a published result will be replicable even if power is high. It follows that if R is lower in one set of studies than in another, then all else being equal, published results will be less replicable in the set with lower R. We illustrate this point by presenting an analysis of data from the social-psychology and cognitive-psychology studies that were included in the Open Science Collaboration’s (2015) replication project. We found that R was lower for the social-psychology studies than for the cognitive-psychology studies, which might explain why the rate of successful replications differed between these two sets of studies. This difference in replication rates may reflect the degree to which scientists in the two fields value risky but potentially groundbreaking (i.e., low- R) research. Critically, high- R research is not inherently better or worse than low- R research for advancing knowledge. However, if they wish to achieve replication rates comparable to those of high- R fields (a judgment call), researchers in low- R fields would need to use an especially low alpha level, conduct experiments that have especially high power, or both.

Are the Motivational Effects of Autonomy-Supportive Conditions Universal? Contrasting Results Among Indians and Americans

In Western theories of motivation, autonomy is conceived as a universal motivator of human action; enhancing autonomy is expected to increase motivation panculturally. Using a novel online experimental paradigm that afforded a behavioral measure of motivation, we found that, contrary to this prevailing view, autonomy cues affect motivation differently among American and Indian corporate professionals. Autonomy-supportive instructions increased motivation among Americans but decreased motivation among Indians. The motivational Cue × Culture interaction was extraordinarily large; the populations exhibited little statistical overlap. A second study suggested that this interaction reflects culturally specific norms that are widely understood by members of the given culture. When evaluating messages to motivate workers, Indians, far more than Americans, preferred a message invoking obligations to one invoking autonomous personal choice norms. Results cast doubt on the claim, made regularly in both basic and applied psychology, that enhancing autonomy is a universally preferred method for boosting motivation.

Subjective Confidence Predicts Information Seeking in Decision Making

There is currently little direct evidence regarding the function of subjective confidence in decision making: The tight correlation between objective accuracy and subjective confidence makes it difficult to distinguish each variable's unique contribution. Here, we created conditions in a perceptual decision task that were matched in accuracy but differed in subjective evaluation of accuracy by orthogonally varying the strength versus variability of evidence. Confidence was reduced with variable (vs. weak) evidence, even across conditions matched for difficulty. Building on this dissociation, we constructed a paradigm in which participants ( N = 20) could choose to seek further information before making their decision. The data provided clear support for the hypothesis that subjective confidence predicts information seeking in decision making: Participants were more likely to sample additional information before giving a response in the condition with low confidence, despite matched accuracy. In a preregistered replication ( N = 50), these findings were replicated with increased task difficulty levels.

Taking responsibility for health in an epistemically polluted environment

Proposals for regulating or nudging healthy choices are controversial. Opponents often argue that individuals should take responsibility for their own health, rather than be paternalistically manipulated for their own good. In this paper, I argue that people can take responsibility for their own health only if they satisfy certain epistemic conditions, but we live in an epistemic environment in which these conditions are not satisfied. Satisfying the epistemic conditions for taking responsibility, I argue, requires regulation of this environment. I describe some proposals for such regulation and show that we cannot reject all regulation in the name of individual responsibility. We must either regulate individuals' healthy choices or regulate the epistemic environment.

When Action-Inaction Framing Leads to Higher Escalation of Commitment: A New Inaction-Effect Perspective on the Sunk-Cost Fallacy

Escalation of commitment to a failing course of action occurs in the presence of (a) sunk costs, (b) negative feedback that things are deviating from expectations, and (c) a decision between escalation and de-escalation. Most of the literature to date has focused on sunk costs, yet we offer a new perspective on the classic escalation-of-commitment phenomenon by focusing on the impact of negative feedback. On the basis of the inaction-effect bias, we theorized that negative feedback results in the tendency to take action, regardless of what that action may be. In four experiments, we demonstrated that people facing escalation-decision situations were indeed action oriented and that framing escalation as action and de-escalation as inaction resulted in a stronger tendency to escalate than framing de-escalation as action and escalation as inaction (mini-meta-analysis effect d = 0.37, 95% confidence interval = [0.21, 0.53]).

Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications

Bayesian parameter estimation and Bayesian hypothesis testing present attractive alternatives to classical inference using confidence intervals and p values. In part I of this series we outline ten prominent advantages of the Bayesian approach. Many of these advantages translate to concrete opportunities for pragmatic researchers. For instance, Bayesian hypothesis testing allows researchers to quantify evidence and monitor its progression as data come in, without needing to know the intention with which the data were collected. We end by countering several objections to Bayesian hypothesis testing. Part II of this series discusses JASP, a free and open source software program that makes it easy to conduct Bayesian estimation and testing for a range of popular statistical scenarios (Wagenmakers et al. this issue).

Zur Methodologie von Replikationsstudien

Zusammenfassung. Replikationsstudien sind in den empirischen Wissenschaften mit unterschiedlichen Zielen verbunden, abhängig davon, ob wir uns im Kontext der Theorieentwicklung oder im Kontext der Theorieüberprüfung bewegen (Entdeckungs- vs. Begründungszusammenhang sensu Reichenbach, 1938 ). Konzeptuelle Replikationsstudien zielen auf Generalisierung ab und können im Entdeckungszusammenhang nützlich sein. Direkte Replikationsstudien zielen demgegenüber auf den Nachweis der Replizierbarkeit eines bestimmten Forschungsergebnisses unter unabhängigen Bedingungen ab und sind im Begründungszusammenhang unverzichtbar. Ohne die Annahme der direkten Replizierbarkeit wird man sich kaum auf allgemein akzeptierte empirische Tatbestände einigen können, die eine notwendige Voraussetzung für Theorieüberprüfungen in den empirischen Wissenschaften sind. Vor diesem Hintergrund werden Standards für Replikationsstudien vorgeschlagen und begründet. Eine Besonderheit in der Psychologie besteht darin, dass das Replikandum in aller Regel eine statistische Hypothese ist, über die lediglich probabilistisch entschieden werden kann. Dies wirft Folgeprobleme in Bezug auf die Formulierung der Replizierbarkeitshypothese, die Kontrolle statistischer Fehlerwahrscheinlichkeiten bei der Entscheidung über die Replizierbarkeitshypothese, die Bestimmung der zu entdeckenden Effektgröße bei Verzerrung vorliegender Ergebnisse durch Publication Bias, die Festlegung des Stichprobenumfangs und die korrekte Interpretation der Replikationsquote auf, für die Lösungsvorschläge unterbreitet und diskutiert werden.

Testing Significance Testing

The practice of Significance Testing (ST) remains widespread in psychological science despite continual criticism of its flaws and abuses. Using simulation experiments, we address four concerns about ST and for two of these we compare ST’s performance with prominent alternatives. We find the following: First, the p values delivered by ST predict the posterior probability of the tested hypothesis well under many research conditions. Second, low p values support inductive inferences because they are most likely to occur when the tested hypothesis is false. Third, p values track likelihood ratios without raising the uncertainties of relative inference. Fourth, p values predict the replicability of research findings better than confidence intervals do. Given these results, we conclude that p values may be used judiciously as a heuristic tool for inductive inference. Yet, p values cannot bear the full burden of inference. We encourage researchers to be flexible in their selection and use of statistical methods.

Replication and extension of long-term implicit memory: Perceptual priming but conceptual cessation

We endeavored to replicate Mitchell's (2006) finding of 17-year implicit memory priming. Subjects saw word and picture stimuli in 1999-2000 (M age = 18.9) and were retested after 11-14 years (M = 13.2; M age = 32.1). Via the internet, they completed four implicit memory tasks: picture fragment identification, word fragment completion, word stem completion, and category exemplar generation. Relative to control subjects (matched on stimuli, age, and education), longitudinal subjects revealed priming on picture and word fragment identification (perceptual tasks), but no priming on word stem completion or category exemplar generation (conceptual tasks). Four longitudinal subjects who failed to recall participating in the prior laboratory session had priming similar to the 10 subjects who did remember. Thus, we replicated the longevity of perceptual priming for pictures, and extended this to word fragment priming as well.

On Nomological Validity and Auxiliary Assumptions: The Importance of Simultaneously Testing Effects in Social Cognitive Theories Applied to Health Behavior and Some Guidelines

Tests of social cognitive theories provide informative data on the factors that relate to health behavior, and the processes and mechanisms involved. In the present article, we contend that tests of social cognitive theories should adhere to the principles of nomological validity, defined as the degree to which predictions in a formal theoretical network are confirmed. We highlight the importance of nomological validity tests to ensure theory predictions can be disconfirmed through observation. We argue that researchers should be explicit on the conditions that lead to theory disconfirmation, and identify any auxiliary assumptions on which theory effects may be conditional. We contend that few researchers formally test the nomological validity of theories, or outline conditions that lead to model rejection and the auxiliary assumptions that may explain findings that run counter to hypotheses, raising potential for ‘falsification evasion.’ We present a brief analysis of studies (k = 122) testing four key social cognitive theories in health behavior to illustrate deficiencies in reporting theory tests and evaluations of nomological validity. Our analysis revealed that few articles report explicit statements suggesting that their findings support or reject the hypotheses of the theories tested, even when findings point to rejection. We illustrate the importance of explicit a priori specification of fundamental theory hypotheses and associated auxiliary assumptions, and identification of the conditions which would lead to rejection of theory predictions. We also demonstrate the value of confirmatory analytic techniques, meta-analytic structural equation modeling, and Bayesian analyses in providing robust converging evidence for nomological validity. We provide a set of guidelines for researchers on how to adopt and apply the nomological validity approach to testing health behavior models.

Do delayed judgements of learning reduce metamemory illusions? A meta-analysis

Past research has shown that the perceptual characteristics of studied items (e.g., font size) lead to a metamemory illusion, and that delayed judgements of learning (JOLs) are better predictors of memory performance than immediate JOLs. Here, we tested whether delayed JOLs could reduce or eliminate the effect of perceptual characteristics on JOLs and restudy decisions. We adopted a meta-analytic approach and analysed the results of 28 experiments in which participants' studied items were presented in either large or small font. JOLs and, sometimes, restudy decisions were collected either immediately or after a delay. Finally, participants completed a memory test. The results of the meta-analyses confirmed the effect of the font size on JOLs and restudy decisions. The delayed procedures reliably reduced the effect of perceptual characteristics on JOLs, but the effect was still significant after a delay. For restudy decisions, delayed procedures only reduced numerically the effect. Surprisingly, the meta-analysis also showed a very subtle memory advantage for items presented in large font over small font, although no individual study showed a significant difference and the overall effect size was small. One plausible explanation is that after a delay, information about font size is not available for some items, causing a reduction in the effect. Moreover, our results suggest that the dissociation between memory and metamemory reported previously may not be dissociation at all, but a mistmatched effect of font size on memory and metamemory.

Psychometric Characteristics of a New Scale for Measuring Self-efficacy in the Regulation of Gambling Behavior

Since its introduction in 1977, self-efficacy has proven to be a fundamental predictor of positive adjustment and achievement in many domains. In problem gambling studies, self-efficacy has been defined mainly as an individual's ability to avoid gambling in risky situations. The interest in this construct developed mainly with regard to treatment approaches, where abstinence from gambling is required. Very little is known, however, regarding self-efficacy as a protective factor for problem gambling. This study aims to fill this gap, proposing a new self-efficacy scale which measures not only the ability to restrain oneself from gambling but also the ability to self-regulate one's gambling behavior. Two studies were conducted in which the data from two Italian prevalence surveys on problem gambling were considered. A total of about 6,000 participants were involved. In the first study, the psychometric characteristics of this new self-efficacy scale were investigated through exploratory and confirmatory factor analyses. The results indicated the presence of two different factors: self-efficacy in self-regulating gambling behavior and self-efficacy in avoiding risky gambling behavior. The second study confirmed the replicability of the two-factor solution and displayed high correlations among these two self-efficacy dimensions and different measures of gambling activities as well as other psychological variables related to gambling (gambling beliefs, gambling motivation, risk propensity, and impulsiveness). The results of logistic regression analyses showed the particular importance of self-regulating gaming behavior in explaining problem gambling as measured by Problem Gambling Severity Index and South Oaks Gambling Screen, thus proving the role of self-efficacy as a pivotal protective factor for problem gambling.

Show Me the Money: A Systematic Exploration of Manipulations, Moderators, and Mechanisms of Priming Effects

A major challenge for accumulating knowledge in psychology is the variation in methods and participant populations across studies in a single domain. We offer a systematic approach to addressing this challenge and implement it in the domain of money priming. In three preregistered experiments ( N = 4,649), participants were exposed to one of a number of money manipulations before completing self-report measures of money activation (Study 1); engaging in a behavioral-persistence task (Study 3); completing self-report measures of subjective wealth, self-sufficiency, and communion-agency (Studies 1-3); and completing demographic questions (Studies 1-3). Four of the five manipulations we tested activated the concept of money, but, contrary to what we expected based on the preponderance of the published literature, no manipulation consistently affected any dependent measure. Moderation by sociodemographic characteristics was sparse and inconsistent across studies. We discuss implications for theories of money priming and explain how our approach can complement recent efforts to build a reproducible, cumulative psychological science.

Going in Many Right Directions, All at Once

When asked whether the field is going in the right direction, my answer is "yes, mostly." Policymakers' respect and the public's interest put us in the spotlight for our intrinsic strengths-but they also draw attention to our weaknesses as reflected by the replication crisis, about which reasonable opinions differ. Among other concerns, civility and mutual tolerance have sometimes been an issue in these debates. As an example of a constructive debate, our lab's recent experience with mutually respectful engagement has advanced solving one scientific puzzle. Principles facilitating this adversarial collaboration include using our respective tribes as secure bases for exploration, sharing agreed-upon rigorous standards, and establishing mutual trust. Finally, my own career path has perhaps oriented me to rely on perseverance, flexibility, tolerance, and optimism for the field.

Introduction to the special issue on recentering science: Replication, robustness, and reproducibility in psychophysiology

In recent years, the psychological and behavioral sciences have increased efforts to strengthen methodological practices and publication standards, with the ultimate goal of enhancing the value and reproducibility of published reports. These issues are especially important in the multidisciplinary field of psychophysiology, which yields rich and complex data sets with a large number of observations. In addition, the technological tools and analysis methods available in the field of psychophysiology are continually evolving, widening the array of techniques and approaches available to researchers. This special issue presents articles detailing rigorous and systematic evaluations of tasks, measures, materials, analysis approaches, and statistical practices in a variety of subdisciplines of psychophysiology. These articles highlight challenges in conducting and interpreting psychophysiological research and provide data-driven, evidence-based recommendations for overcoming those challenges to produce robust, reproducible results in the field of psychophysiology.

Are Psychology Journals Anti-replication? A Snapshot of Editorial Practices

Recent research in psychology has highlighted a number of replication problems in the discipline, with publication bias - the preference for publishing original and positive results, and a resistance to publishing negative results and replications- identified as one reason for replication failure. However, little empirical research exists to demonstrate that journals explicitly refuse to publish replications. We reviewed the instructions to authors and the published aims of 1151 psychology journals and examined whether they indicated that replications were permitted and accepted. We also examined whether journal practices differed across branches of the discipline, and whether editorial practices differed between low and high impact journals. Thirty three journals (3%) stated in their aims or instructions to authors that they accepted replications. There was no difference between high and low impact journals. The implications of these findings for psychology are discussed.