Publication bias and low power in field studies on goal priming

Developmental psychologists should adopt citizen science to improve generalization and reproducibility

Widespread failures of replication and generalization are, ironically, a scientific triumph, in that they confirm the fundamental metascientific theory that underlies our field. Generalizable and replicable findings require testing large numbers of subjects from a wide range of demographics with a large, randomly-sampled stimulus set, and using a variety of experimental parameters. Because few studies accomplish any of this, meta-scientists predict that findings will frequently fail to replicate or generalize. We argue that to be more robust and replicable, developmental psychology needs to find a mechanism for collecting data at greater scale and from more diverse populations. Luckily, this mechanism already exists: Citizen science, in which large numbers of uncompensated volunteers provide data. While best-known for its contributions to astronomy and ecology, citizen science has also produced major findings in neuroscience and psychology, and increasingly in developmental psychology. We provide examples, address practical challenges, discuss limitations, and compare to other methods of obtaining large datasets. Ultimately, we argue that the range of studies where it makes sense *not* to use citizen science is steadily dwindling.

Influence of Economic Scarcity on Race Perception

Racial socioeconomic gaps are widened in periods of economic recession. Besides social and institutional factors, black people also struggle with many psychological factors. The literature reports racial-biased complex behaviors and high-level processes that are influenced by economic scarcity. A previous study found a bias at the perceptual level: an experimental manipulation of scarcity (a subliminal priming paradigm) lowered the black-white race categorization threshold. Here we present a conceptual replication in a higher ecological setup. In our main analysis we compared the categorization threshold of participants that received the Brazilian government's emergency economic aid in the context of the COVID-19 pandemic (n = 136) and participants that did not receive the economic aid (n = 135) in an online psychophysical task that presented faces in a black-white race continuum. Additionally, we analyzed the economic impact of COVID-19 on household income, and in cases of family unemployment. Our results do not support the claim that perception of race is influenced by economic scarcity. Interestingly, we found that when people differ greatly in terms of racial prejudice, they encode visual information related to race differently. People with higher scores on a prejudice scale needed more phenotypic traits of the black race to categorize a face as black. We discuss the results in terms of differences in method and sample.

An updated meta-analysis of the primed goal-organizational behaviour relationship

Environmental cues (e.g. achievement-related words and pictures) can prime/activate, in the absence of awareness, a mental representation of importance stored in memory. Chen et al.'s 2021 Applied Psychology: An International Review 70, 216-253. (doi:10.1111/apps.12239) meta-analysis revealed a moderate, significant overall effect for the goal priming-organizational behaviour relationship, with three moderators identified: context-specific versus a general prime, prime modality (i.e. visual versus linguistic) and experimental setting (field versus laboratory). An independent researcher found that their finding was negligibly affected by a publication bias. Shanks & Vadillo (2021), Royal Society Open Science 8, 210544. (doi:10.1098/rsos.210544) (field: k = 13, N = 683, d = 0.64), questioned Chen et al.'s conclusion regarding the effect size found in field studies (field: k = 8, N = 357, d = 0.68). In this paper, we discussed Shanks & Vadillo's selection of additional field experiments that led to their conclusion of a publication bias. We updated Chen et al.'s meta-analysis to include relevant studies conducted since that study's publication. The present meta-analysis reproduced the original findings in Chen et al. (field: k = 11, N = 534, d = 0.67). The updated findings are consistent with: (i) laboratory findings, (ii) the findings obtained in field experiments on consciously set goals and (iii) goal setting theory (Latham & Locke, 2018 In Handbook of industrial, work & organizational Psychology, vol. 1 (eds D Ones, N Anderson, C Viswesvaran, H Sinangil), pp. 103-124).