The Psychology of Construal in the Design of Field Experiments a aRevised version, prepared following the NBER Conference on Economics of Field Experiments, organized by Esther Duflo & Abhijit Banerjee. Thank you to Robin Gomila, Sachin Banker, Peter Aronow, and Ruth Ditlmann for helpful comments. Address correspondence to epaluck@princeton.edu and eshafir@princeton.edu. Handbook of Field Experiments. Elsevier; 2017. pp. 245-68. [DOI: 10.1016/bs.hefe.2016.12.001]

Quantifying idiosyncratic and shared contributions to judgment

Identifying relative idiosyncratic and shared contributions to judgments is a fundamental challenge to the study of human behavior, yet there is no established method for estimating these contributions. Using edge cases of stimuli varying in intrarater reliability and interrater agreement-faces (high on both), objects (high on the former, low on the latter), and complex patterns (low on both)-we showed that variance component analyses (VCAs) accurately captured the psychometric properties of the data (Study 1). Simulations showed that the VCA generalizes to any arbitrary continuous rating and that both sample and stimulus set size affect estimate precision (Study 2). Generally, a minimum of 60 raters and 30 stimuli provided reasonable estimates within our simulations. Furthermore, VCA estimates stabilized given more than two repeated measures, consistent with the finding that both intrarater reliability and interrater agreement increased nonlinearly with repeated measures (Study 3). The VCA provides a rigorous examination of where variance lies in data, can be implemented using mixed models with crossed random effects, and is general enough to be useful in any judgment domain in which agreement and disagreement are important to quantify and in which multiple raters independently rate multiple stimuli.

Difference Matters: Teaching Students a Contextual Theory of Difference Can Help Them Succeed

Today’s increasingly diverse and divided world requires the ability to understand and navigate across social-group differences. We propose that interventions that teach students about these differences can not only improve all students’ intergroup skills but also help disadvantaged students succeed in school. Drawing on interdisciplinary research, this article theorizes that teaching students a contextual understanding of difference can accomplish both of these important goals. Understanding difference as contextual means recognizing that social-group differences come from participating in and adapting to diverse sociocultural contexts. This article begins by reviewing research that highlights two distinct understandings of social-group differences—as contextual or essential—and demonstrates their consequences for intergroup outcomes. We then review research on multicultural and social justice education that highlights the potential benefits of educating students about social-group differences. We propose that these educational approaches are associated with intergroup and academic benefits for one key reason: They teach students a contextual theory of difference. Finally, to illustrate and provide causal evidence for our theory of how a contextual understanding of difference affords these benefits, this article provides an overview of the first social psychological intervention to teach students a contextual understanding of difference: difference-education.

A pragmatist philosophy of psychological science and its implications for replication

A pragmatist philosophy of psychological science offers to the direct replication debate concrete recommendations and novel benefits that are not discussed in Zwaan et al. This philosophy guides our work as field experimentalists interested in behavioral measurement. Furthermore, all psychologists can relate to its ultimate aim set out by William James: to study mental processes that provide explanations for why people behave as they do in the world.