psiTurk: An open-source framework for conducting replicable behavioral experiments online

Cue relevance drives early quitting in visual search

Irrelevant salient distractors can trigger early quitting in visual search, causing observers to miss targets they might otherwise find. Here, we asked whether task-relevant salient cues can produce a similar early quitting effect on the subset of trials where those cues fail to highlight the target. We presented participants with a difficult visual search task and used two cueing conditions. In the high-predictive condition, a salient cue in the form of a red circle highlighted the target most of the time a target was present. In the low-predictive condition, the cue was far less accurate and did not reliably predict the target (i.e., the cue was often a false positive). These were contrasted against a control condition in which no cues were presented. In the high-predictive condition, we found clear evidence of early quitting on trials where the cue was a false positive, as evidenced by both increased miss errors and shorter response times on target absent trials. No such effects were observed with low-predictive cues. Together, these results suggest that salient cues which are false positives can trigger early quitting, though perhaps only when the cues have a high-predictive value. These results have implications for real-world searches, such as medical image screening, where salient cues (referred to as computer-aided detection or CAD) may be used to highlight potentially relevant areas of images but are sometimes inaccurate.

Task-irrelevant semantic relationship between objects and scene influence attentional allocation

Recent behavioral evidence suggests that the semantic relationships between isolated objects can influence attentional allocation, with highly semantically related objects showing an increase in processing efficiency. This semantic influence is present even when it is task-irrelevant (i.e., when semantic information is not central to the task). However, given that objects exist within larger contexts, i.e., scenes, it is critical to understand whether the semantic relationship between a scene and its objects continuously influence attention. Here, we investigated the influence of task-irrelevant scene semantic properties on attentional allocation and the degree to which semantic relationships between scenes and objects interact. Results suggest that task-irrelevant associations between scenes and objects continuously influence attention and that this influence is directly predicted by the perceived strength of semantic associations.

Control adjustment costs limit goal flexibility: Empirical evidence and a computational account

A cornerstone of human intelligence is the ability to flexibly adjust our cognition and behavior as our goals change. For instance, achieving some goals requires efficiency, while others require caution. Adapting to these changing goals require corresponding adjustments in cognitive control (e.g., levels of attention, response thresholds). However, adjusting our control to meet new goals comes at a cost: we are better at achieving a goal in isolation than when transitioning between goals. The source of these control adjustment costs remains poorly understood, and the bulk of our understanding of such costs comes from settings in which participants transition between discrete task sets, rather than performance goals. Across four experiments, we show that adjustments in continuous control states incur a performance cost, and that a dynamical systems model can explain the source of these costs. Participants performed a single cognitively demanding task under varying performance goals (e.g., to be fast or to be accurate). We modeled control allocation to include a dynamic process of adjusting from one's current control state to a target state for a given performance goal. By incorporating inertia into this adjustment process, our model accounts for our empirical findings that people under-shoot their target control state more (i.e., exhibit larger adjustment costs) when (a) goals switch rather than remain fixed over a block (Study 1); (b) target control states are more distant from one another (Study 2); (c) less time is given to adjust to the new goal (Study 3); and (d) when anticipating having to switch goals more frequently (Study 4). Our findings characterize the costs of adjusting control to meet changing goals, and show that these costs can emerge directly from cognitive control dynamics. In so doing, they shed new light on the sources of and constraints on flexibility in human goal-directed behavior.

Self-prioritization in working memory gating

Working memory (WM) involves a dynamic interplay between temporary maintenance and updating of goal-relevant information. The balance between maintenance and updating is regulated by an input-gating mechanism that determines which information should enter WM (gate opening) and which should be kept out (gate closing). We investigated whether updating and gate opening/closing are differentially sensitive to the kind of information to be encoded and maintained in WM. Specifically, since the social salience of a stimulus is known to affect cognitive performance, we investigated if self-relevant information differentially impacts maintenance, updating, or gate opening/closing. Participants first learned to associate two neutral shapes with two social labels (i.e., "you" vs. "stranger"), respectively. Subsequently they performed the reference-back paradigm, a well-established WM task that disentangles WM updating, gate opening, and gate closing. Crucially, the shapes previously associated with the self or a stranger served as target stimuli in the reference-back task. We replicated the typical finding of a repetition benefit when consecutive trials require opening the gate to WM. In Study 1 (N = 45) this advantage disappeared when self-associated stimuli were recently gated into WM and immediately needed to be replaced by stranger-associated stimuli. However, this was not replicated in a larger sample (Study 2; N = 90), where a repetition benefit always occurred on consecutive gate-opening trials. Overall, our results do not provide evidence that the self-relevance of stimuli modulates component processes of WM. We discuss possible reasons for this null finding, including the importance of continuous reinstatement and task-relevance of the shape-label associations.

DataPipe: Born-open data collection for online experiments

DataPipe ( https://pipe.jspsych.org ) is a tool that allows researchers to save data from a behavioral experiment directly to the Open Science Framework. Researchers can configure data storage options for an experiment on the DataPipe website and then use the DataPipe API to send data to the Open Science Framework from any Internet-connected experiment. DataPipe is free to use and open-source. This paper describes the design of DataPipe and how it can help researchers adopt the practice of born-open data collection.

Improving Memory Search Through Model-Based Cue Selection

We often use cues from our environment when we get stuck searching our memories, but prior research has failed to show benefits of cuing with other, randomly selected list items during memory search. What accounts for this discrepancy? We proposed that cues' content critically determines their effectiveness and sought to select the right cues by building a computational model of how cues affect memory search. Participants (N = 195 young adults from the United States) recalled significantly more items when receiving our model's best (vs. worst) cue. Our model provides an account of why some cues better aid recall: Effective cues activate contexts most similar to the remaining items' contexts, facilitating recall in an unsearched area of memory. We discuss our contributions in relation to prominent theories about the effect of external cues.

FAB: A "Dummy's" program for self-paced forward and backward reading

The self-paced reading paradigm has been popular and widely used in psycholinguistic research for several decades. The tool described in this paper, FAB (Forward and Backward reading), is a tool created to hopefully and maximally reduce the coding demands and simplify the operation costs for experimental researchers and clinical researchers who are doing experimental work, in their designing, coding, implementing, and analyzing self-paced reading tasks. Its basis in web languages (HTML, JavaScript) also promotes experimental implementation and material sharing in our era of open science. In addition, FAB has a unique forward-and-backward mode that can track regressive-like behaviors that are usually only recordable using eye-tracking or mouse-tracking equipment. In this paper, the specific application and usage of FAB is demonstrated in one laboratory and two online validation experiments. We hope this free and open-sourced tool can benefit research in a diverse range of contexts where self-paced reading is desirable.

Are acoustics enough? Semantic effects on auditory salience in natural scenes

Auditory salience is a fundamental property of a sound that allows it to grab a listener's attention regardless of their attentional state or behavioral goals. While previous research has shed light on acoustic factors influencing auditory salience, the semantic dimensions of this phenomenon have remained relatively unexplored owing both to the complexity of measuring salience in audition as well as limited focus on complex natural scenes. In this study, we examine the relationship between acoustic, contextual, and semantic attributes and their impact on the auditory salience of natural audio scenes using a dichotic listening paradigm. The experiments present acoustic scenes in forward and backward directions; the latter allows to diminish semantic effects, providing a counterpoint to the effects observed in forward scenes. The behavioral data collected from a crowd-sourced platform reveal a striking convergence in temporal salience maps for certain sound events, while marked disparities emerge in others. Our main hypothesis posits that differences in the perceptual salience of events are predominantly driven by semantic and contextual cues, particularly evident in those cases displaying substantial disparities between forward and backward presentations. Conversely, events exhibiting a high degree of alignment can largely be attributed to low-level acoustic attributes. To evaluate this hypothesis, we employ analytical techniques that combine rich low-level mappings from acoustic profiles with high-level embeddings extracted from a deep neural network. This integrated approach captures both acoustic and semantic attributes of acoustic scenes along with their temporal trajectories. The results demonstrate that perceptual salience is a careful interplay between low-level and high-level attributes that shapes which moments stand out in a natural soundscape. Furthermore, our findings underscore the important role of longer-term context as a critical component of auditory salience, enabling us to discern and adapt to temporal regularities within an acoustic scene. The experimental and model-based validation of semantic factors of salience paves the way for a complete understanding of auditory salience. Ultimately, the empirical and computational analyses have implications for developing large-scale models for auditory salience and audio analytics.

EasyEyes - A new method for accurate fixation in online vision testing

Online methods allow testing of larger, more diverse populations, with much less effort than in-lab testing. However, many psychophysical measurements, including visual crowding, require accurate eye fixation, which is classically achieved by testing only experienced observers who have learned to fixate reliably, or by using a gaze tracker to restrict testing to moments when fixation is accurate. Alas, both approaches are impractical online as online observers tend to be inexperienced, and online gaze tracking, using the built-in webcam, has a low precision (±4 deg). EasyEyes open-source software reliably measures peripheral thresholds online with accurate fixation achieved in a novel way, without gaze tracking. It tells observers to use the cursor to track a moving crosshair. At a random time during successful tracking, a brief target is presented in the periphery. The observer responds by identifying the target. To evaluate EasyEyes fixation accuracy and thresholds, we tested 12 naive observers in three ways in a counterbalanced order: first, in the laboratory, using gaze-contingent stimulus presentation; second, in the laboratory, using EasyEyes while independently monitoring gaze using EyeLink 1000; third, online at home, using EasyEyes. We find that crowding thresholds are consistent and individual differences are conserved. The small root mean square (RMS) fixation error (0.6 deg) during target presentation eliminates the need for gaze tracking. Thus, this method enables fixation-dependent measurements online, for easy testing of larger and more diverse populations.

Human-like systematic generalization through a meta-learning neural network

The power of human language and thought arises from systematic compositionality-the algebraic ability to understand and produce novel combinations from known components. Fodor and Pylyshyn1 famously argued that artificial neural networks lack this capacity and are therefore not viable models of the mind. Neural networks have advanced considerably in the years since, yet the systematicity challenge persists. Here we successfully address Fodor and Pylyshyn's challenge by providing evidence that neural networks can achieve human-like systematicity when optimized for their compositional skills. To do so, we introduce the meta-learning for compositionality (MLC) approach for guiding training through a dynamic stream of compositional tasks. To compare humans and machines, we conducted human behavioural experiments using an instruction learning paradigm. After considering seven different models, we found that, in contrast to perfectly systematic but rigid probabilistic symbolic models, and perfectly flexible but unsystematic neural networks, only MLC achieves both the systematicity and flexibility needed for human-like generalization. MLC also advances the compositional skills of machine learning systems in several systematic generalization benchmarks. Our results show how a standard neural network architecture, optimized for its compositional skills, can mimic human systematic generalization in a head-to-head comparison.

Rapid runtime learning by curating small datasets of high-quality items obtained from memory

We propose the "runtime learning" hypothesis which states that people quickly learn to perform unfamiliar tasks as the tasks arise by using task-relevant instances of concepts stored in memory during mental training. To make learning rapid, the hypothesis claims that only a few class instances are used, but these instances are especially valuable for training. The paper motivates the hypothesis by describing related ideas from the cognitive science and machine learning literatures. Using computer simulation, we show that deep neural networks (DNNs) can learn effectively from small, curated training sets, and that valuable training items tend to lie toward the centers of data item clusters in an abstract feature space. In a series of three behavioral experiments, we show that people can also learn effectively from small, curated training sets. Critically, we find that participant reaction times and fitted drift rates are best accounted for by the confidences of DNNs trained on small datasets of highly valuable items. We conclude that the runtime learning hypothesis is a novel conjecture about the relationship between learning and memory with the potential for explaining a wide variety of cognitive phenomena.

Introducing the Fear Learning and Anxiety Response (FLARe) app and web portal for the remote delivery of fear conditioning experiments

Experimental paradigms measuring key psychological constructs can enhance our understanding of mechanisms underlying human psychological well-being and mental health. Delivering such paradigms remotely affords opportunities to reach larger, more representative samples than is typically possible with in-person research. The efficiency gained from remote delivery makes it easier to test replication of previously established effects in well-powered samples. There are several challenges to the successful development and delivery of remote experimental paradigms, including use of an appropriate delivery platform, identifying feasible outcome measures, and metrics of participant compliance. In this paper, we present FLARe (Fear Learning and Anxiety Response), open-source software in the form of a smartphone app and web portal for the creation and delivery of remote fear conditioning experiments. We describe the benefits and challenges associated with the creation of a remote delivery platform for fear conditioning, before presenting in detail the resultant software suite, and one instance of deploying this using the FLARe Research infrastructure. We provide examples of the application of FLARe to several research questions which illustrate the benefits of the remote approach to experiment delivery. The FLARe smartphone app and web portal are available for use by other researchers and have been designed to be user-friendly and intuitive. We hope that FLARe will be a useful tool for those interested in conducting well-powered fear conditioning studies to inform our understanding of the development and treatment of anxiety disorders.

A computational process-tracing method for measuring people's planning strategies and how they change over time

One of the most unique and impressive feats of the human mind is its ability to discover and continuously refine its own cognitive strategies. Elucidating the underlying learning and adaptation mechanisms is very difficult because changes in cognitive strategies are not directly observable. One important domain in which strategies and mechanisms are studied is planning. To enable researchers to uncover how people learn how to plan, we offer a tutorial introduction to a recently developed process-tracing paradigm along with a new computational method for measuring the nature and development of a person's planning strategies from the resulting process-tracing data. Our method allows researchers to reveal experience-driven changes in people's choice of individual planning operations, planning strategies, strategy types, and the relative contributions of different decision systems. We validate our method on simulated and empirical data. On simulated data, its inferences about the strategies and the relative influence of different decision systems are accurate. When evaluated on human data generated using our process-tracing paradigm, our computational method correctly detects the plasticity-enhancing effect of feedback and the effect of the structure of the environment on people's planning strategies. Together, these methods can be used to investigate the mechanisms of cognitive plasticity and to elucidate how people acquire complex cognitive skills such as planning and problem-solving. Importantly, our methods can also be used to measure individual differences in cognitive plasticity and examine how different types (pedagogical) interventions affect the acquisition of cognitive skills.

The visible gorilla: Unexpected fast-not physically salient-Objects are noticeable

It is widely believed that observers can fail to notice clearly visible unattended objects, even if they are moving. Here, we created parametric tasks to test this belief and report the results of three high-powered experiments (total n = 4,493) indicating that this effect is strongly modulated by the speed of the unattended object. Specifically, fast-but not slow-objects are readily noticeable, whether they are attended or not. These results suggest that fast motion serves as a potent exogenous cue that overrides task-focused attention, showing that fast speeds, not long exposure duration or physical salience, strongly diminish inattentional blindness effects.

Performance-linked visual feedback slows response times during a sustained attention task

In the present study, we tested a visual feedback triggering system based on real-time tracking of response time (RT) in a sustained attention task. In our task, at certain points, brief visual feedback epochs were presented without interrupting the task itself. When these feedback epochs were performance-linked-meaning that they were triggered because participants were responding more quickly than usual-RTs were slowed after the presentation of feedback. However, visual feedback epochs displayed at predetermined times that were independent of participants' performance did not slow RTs. Results from a second experiment support the idea that this is not simply a return to baseline that would have occurred had the feedback not been presented, but instead suggest that the feedback itself was effective in altering participants' responses. In a third experiment, we replicated this result across with both written word feedback and visual symbolic feedback, as well as in cases where the participant was explicitly told that the feedback was linked to their performance. All together, these data provide insight into potential mechanisms for detecting and disrupting lapses in sustained attention without interrupting a continuous task.

Evidence That Event Boundaries Are Access Points for Memory Retrieval

When recalling memories, we often scan information-rich continuous episodes, for example, to find our keys. How does our brain access and search through those memories? We suggest that high-level structure, marked by event boundaries, guides us through this process: In our computational model, memory scanning is sped up by skipping ahead to the next event boundary upon reaching a decision threshold. In adult Mechanical Turk workers from the United States, we used a movie (normed for event boundaries; Study 1, N = 203) to prompt memory scanning of movie segments for answers (Study 2, N = 298) and mental simulation (Study 3, N = 100) of these segments. Confirming model predictions, we found that memory-scanning times varied as a function of the number of event boundaries within a segment and the distance of the search target to the previous boundary (the key diagnostic parameter). Mental simulation times were also described by a skipping process with a higher skipping threshold than memory scanning. These findings identify event boundaries as access points to memory.

Learning when effort matters: neural dynamics underlying updating and adaptation to changes in performance efficacy

To determine how much cognitive control to invest in a task, people need to consider whether exerting control matters for obtaining rewards. In particular, they need to account for the efficacy of their performance-the degree to which rewards are determined by performance or by independent factors. Yet it remains unclear how people learn about their performance efficacy in an environment. Here we combined computational modeling with measures of task performance and EEG, to provide a mechanistic account of how people (i) learn and update efficacy expectations in a changing environment and (ii) proactively adjust control allocation based on current efficacy expectations. Across 2 studies, subjects performed an incentivized cognitive control task while their performance efficacy (the likelihood that rewards are performance-contingent or random) varied over time. We show that people update their efficacy beliefs based on prediction errors-leveraging similar neural and computational substrates as those that underpin reward learning-and adjust how much control they allocate according to these beliefs. Using computational modeling, we show that these control adjustments reflect changes in information processing, rather than the speed-accuracy tradeoff. These findings demonstrate the neurocomputational mechanism through which people learn how worthwhile their cognitive control is.

What would have happened? Counterfactuals, hypotheticals and causal judgements

How do people make causal judgements? In this paper, I show that counterfactual simulations are necessary for explaining causal judgements about events, and that hypotheticals do not suffice. In two experiments, participants viewed video clips of dynamic interactions between billiard balls. In Experiment 1, participants either made hypothetical judgements about whether ball B would go through the gate if ball A were not present in the scene, or counterfactual judgements about whether ball B would have gone through the gate if ball A had not been present. Because the clips featured a block in front of the gate that sometimes moved and sometimes stayed put, hypothetical and counterfactual judgements came apart. A computational model that evaluates hypotheticals and counterfactuals by running noisy physical simulations accurately captured participants' judgements. In Experiment 2, participants judged whether ball A caused ball B to go through the gate. The results showed a tight fit between counterfactual and causal judgements, whereas hypotheticals did not predict causal judgements. I discuss the implications of this work for theories of causality, and for studying the development of counterfactual thinking in children. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.

Recovering Individual Emotional States from Sparse Ratings Using Collaborative Filtering

A fundamental challenge in emotion research is measuring feeling states with high granularity and temporal precision without disrupting the emotion generation process. Here we introduce and validate a new approach in which responses are sparsely sampled and the missing data are recovered using a computational technique known as collaborative filtering (CF). This approach leverages structured covariation across individual experiences and is available in Neighbors, an open-source Python toolbox. We validate our approach across three different experimental contexts by recovering dense individual ratings using only a small subset of the original data. In dataset 1, participants (n=316) separately rated 112 emotional images on 6 different discrete emotions. In dataset 2, participants (n=203) watched 8 short emotionally engaging autobiographical stories while simultaneously providing moment-by-moment ratings of the intensity of their affective experience. In dataset 3, participants (n=60) with distinct social preferences made 76 decisions about how much money to return in a hidden multiplier trust game. Across all experimental contexts, CF was able to accurately recover missing data and importantly outperformed mean and multivariate imputation, particularly in contexts with greater individual variability. This approach will enable new avenues for affective science research by allowing researchers to acquire high dimensional ratings from emotional experiences with minimal disruption to the emotion-generation process.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-022-00161-2.

Flexible top-down control in the interaction between working memory and perception

Successful goal-directed behavior often requires continuous sensory processing while simultaneously maintaining task-related information in working memory (WM). Although WM and perception are known to interact, little is known about how their interactions are controlled. Here, we tested the hypothesis that WM perception interactions engage two distinct modes of control – proactive and reactive – in a manner similar to classic conflict-adaptation tasks (e.g. Stroop, flanker, and Simon). Participants performed a delayed recall-of-orientation WM task, plus a standalone visual discrimination-of-orientation task the occurred during the delay period, and with the congruity in orientation between the tasks manipulated. Proactive control was seen in the sensitivity of task performance to the previous trial's congruity (i.e. a Gratton effect). Reactive control was observed in a repulsive serial-dependence produced by incongruent discriminanda. Quantitatively, these effects were explained by parameters from a reinforcement learning-based model that tracks trial-to-trial fluctuations in control demand: reactive control by a phasic control prediction error (control PE), and proactive control by a tonic level of predicted conflict updated each trial by the control PE. Thus, WM-perception interactions may be controlled by the same mechanisms that govern conflict in other domains of cognition, such as response selection.

Fitness tracking reveals task-specific associations between memory, mental health, and physical activity

Physical activity can benefit both physical and mental well-being. Different forms of exercise (e.g., aerobic versus anaerobic; running versus walking, swimming, or yoga; high-intensity interval training versus endurance workouts; etc.) impact physical fitness in different ways. For example, running may substantially impact leg and heart strength but only moderately impact arm strength. We hypothesized that the mental benefits of physical activity might be similarly differentiated. We focused specifically on how different intensities of physical activity might relate to different aspects of memory and mental health. To test our hypothesis, we collected (in aggregate) roughly a century’s worth of fitness data. We then asked participants to fill out surveys asking them to self-report on different aspects of their mental health. We also asked participants to engage in a battery of memory tasks that tested their short and long term episodic, semantic, and spatial memory performance. We found that participants with similar physical activity habits and fitness profiles tended to also exhibit similar mental health and task performance profiles. These effects were task-specific in that different physical activity patterns or fitness characteristics varied with different aspects of memory, on different tasks. Taken together, these findings provide foundational work for designing physical activity interventions that target specific components of cognitive performance and mental health by leveraging low-cost fitness tracking devices.

Narrative thinking lingers in spontaneous thought

Some experiences linger in mind, spontaneously returning to our thoughts for minutes after their conclusion. Other experiences fall out of mind immediately. It remains unclear why. We hypothesize that an input is more likely to persist in our thoughts when it has been deeply processed: when we have extracted its situational meaning rather than its physical properties or low-level semantics. Here, participants read sequences of words with different levels of coherence (word-, sentence-, or narrative-level). We probe participants’ spontaneous thoughts via free word association, before and after reading. By measuring lingering subjectively (via self-report) and objectively (via changes in free association content), we find that information lingers when it is coherent at the narrative level. Furthermore, and an individual’s feeling of transportation into reading material predicts lingering better than the material’s objective coherence. Thus, our thoughts in the present moment echo prior experiences that have been incorporated into deeper, narrative forms of thinking.

Some experiences linger in our minds, while others quickly fade. Here, the authors show that the extent to which our recent experiences linger into subsequent thought increases as a function of processing depth.

The Potential of Speech as the Calibration Sound for Level Calibration of Non-Laboratory Listening Test Setups

The pandemic of COVID-19 and the resulting countermeasures have made it difficult or impossible to perform listening tests in controlled laboratory environments. This paper examines the possibility of using speech for level calibration of sound reproduction systems used in listening tests performed in non-laboratory conditions, i.e., when such tests are distributed through the means of electronic communication and performed in a home environment. Moreover, a larger pool of potential test subjects can be reached in this manner. The perception of what the “normal” level of reproduced speech should be was examined through a listening experiment by letting the listeners set the level of reproduced speech samples as they saw fit, depending on the used sound reproduction system, the (non)existence of visual stimulus, and the voice of the speaker. The results show that the perception of normal speech level is highly individual when it comes to setting that level by listening to reproduced speech. The interindividual differences between the subjects are considerably larger than the impact of the three main effects. The understanding of what the “normal” level of read speech should be was examined experimentally as well by asking the subjects to read a paragraph of text, depending on the visual stimulus. The results show that the “normal” level of read speech is reasonably consistent and averages at 55 dBA at a normal conversational distance of 1 m, in a room with room acoustics conditions typical for home environment and low background noise, and with the visual stimulus that mimics the interlocutor put within the personal space of the reader. A preliminary proposal is given of a level calibration method for non-laboratory listening experiments based on these results, and some of its aspects that require further research are discussed.

Enabling Morally Sensitive Robotic Clarification Requests

The design of current natural language-oriented robot architectures enables certain architectural components to circumvent moral reasoning capabilities. One example of this is reflexive generation of clarification requests as soon as referential ambiguity is detected in a human utterance. As shown in previous research, this can lead robots to (1) miscommunicate their moral dispositions and (2) weaken human perception or application of moral norms within their current context. We present a solution to these problems by performing moral reasoning on each potential disambiguation of an ambiguous human utterance and responding accordingly, rather than immediately and naively requesting clarification. We implement our solution in the Distributed Integrated Cognition Affect and Reflection robot architecture, which, to our knowledge, is the only current robot architecture with both moral reasoning and clarification request generation capabilities. We then evaluate our method with a human subjects experiment, the results of which indicate that our approach successfully ameliorates the two identified concerns.

Simulating behavior to help researchers build experiments

Testing that an experiment works as intended is critical for identifying design problems and catching technical errors that could invalidate the results. Testing is also time-consuming because of the need to manually run the experiment. This makes testing the experiment costly for researchers, and therefore testing is less comprehensive than in other kinds of software development where tools to automate and speed up the testing process are widely used. In this paper, we describe an approach that substantially reduces the time required to test behavioral experiments: automated simulation of participant behavior. We describe how software that is used to build experiments can use information contained in the experiment's code to automatically generate plausible participant behavior. We demonstrate this through an implementation using jsPsych. We then describe four potential scenarios where automated simulation of participant behavior can improve the way researchers build experiments. Each scenario includes a demo and accompanying code. The full set of examples can be found at https://jspsych.github.io/simulation-examples/ .

The Paradox of Time in Dynamic Causal Systems

Recent work has shown that people use temporal information including order, delay, and variability to infer causality between events. In this study, we build on this work by investigating the role of time in dynamic systems, where causes take continuous values and also continually influence their effects. Recent studies of learning in these systems explored short interactions in a setting with rapidly evolving dynamics and modeled people as relying on simpler, resource-limited strategies to grapple with the stream of information. A natural question that arises from such an account is whether interacting with systems that unfold more slowly might reduce the systematic errors that result from these strategies. Paradoxically, we find that slowing the task indeed reduced the frequency of one type of error, albeit at the cost of increasing the overall error rate. To explain these results we posit that human learners analyze continuous dynamics into discrete events and use the observed relationships between events to draw conclusions about causal structure. We formalize this intuition in terms of a novel Causal Event Abstraction model and show that this model indeed captures the observed pattern of errors. We comment on the implications these results have for causal cognition.

Semantic Memory and Creativity: The Costs and Benefits of Semantic Memory Structure in Generating Original Ideas

Despite its theoretical importance, little is known about how semantic memory structure facilitates and constrains creative idea production. We examine whether the semantic richness of a concept has both benefits and costs to creative idea production. Specifically, we tested whether cue set-size-an index of semantic richness reflecting the average number of elements associated with a given concept-impacts the quantity (fluency) and quality (originality) of responses generated during the alternate uses task (AUT). Across four studies, we show that low-association, sparse, AUT cues benefit originality at the cost of fluency compared to high-association, rich, AUT cues. Furthermore, we found an interaction with individual differences in fluid intelligence in the low-association AUT cues, suggesting that constraints of sparse semantic knowledge can be overcome with top-down intervention. The findings indicate that semantic richness differentially impacts the quality and quantity of generated ideas, and that cognitive control processes can facilitate idea production when conceptual knowledge is limited.

People construct simplified mental representations to plan

One of the most striking features of human cognition is the ability to plan. Two aspects of human planning stand out-its efficiency and flexibility. Efficiency is especially impressive because plans must often be made in complex environments, and yet people successfully plan solutions to many everyday problems despite having limited cognitive resources^1-3. Standard accounts in psychology, economics and artificial intelligence have suggested that human planning succeeds because people have a complete representation of a task and then use heuristics to plan future actions in that representation^4-11. However, this approach generally assumes that task representations are fixed. Here we propose that task representations can be controlled and that such control provides opportunities to quickly simplify problems and more easily reason about them. We propose a computational account of this simplification process and, in a series of preregistered behavioural experiments, show that it is subject to online cognitive control^12-14 and that people optimally balance the complexity of a task representation and its utility for planning and acting. These results demonstrate how strategically perceiving and conceiving problems facilitates the effective use of limited cognitive resources.

Conceptual knowledge shapes visual working memory for complex visual information

Human visual working memory (VWM) is a memory store people use to maintain the visual features of objects and scenes. Although it is obvious that bottom-up information influences VWM, the extent to which top-down conceptual information influences VWM is largely unknown. We report an experiment in which groups of participants were trained in one of two different categories of geologic faults (left/right lateral, or normal/reverse faults), or received no category training. Following training, participants performed a visual change detection task in which category knowledge was irrelevant to the task. Participants were more likely to detect a change in geologic scenes when the changes crossed a trained categorical distinction (e.g., the left/right lateral fault boundary), compared to within-category changes. In addition, participants trained to distinguish left/right lateral faults were more likely to detect changes when the scenes were mirror images along the left/right dimension. Similarly, participants trained to distinguish normal/reverse faults were more likely to detect changes when scenes were mirror images along the normal/reverse dimension. Our results provide direct empirical evidence that conceptual knowledge influences VWM performance for complex visual information. An implication of our results is that cognitive scientists may need to reconceptualize VWM so that it is closer to "conceptual short-term memory".

Face neurons encode nonsemantic features

The primate inferior temporal cortex contains neurons that respond more strongly to faces than to other objects. Termed “face neurons,” these neurons are thought to be selective for faces as a semantic category. However, face neurons also partly respond to clocks, fruits, and single eyes, raising the question of whether face neurons are better described as selective for visual features related to faces but dissociable from them. We used a recently described algorithm, XDream, to evolve stimuli that strongly activated face neurons. XDream leverages a generative neural network that is not limited to realistic objects. Human participants assessed images evolved for face neurons and for nonface neurons and natural images depicting faces, cars, fruits, etc. Evolved images were consistently judged to be distinct from real faces. Images evolved for face neurons were rated as slightly more similar to faces than images evolved for nonface neurons. There was a correlation among natural images between face neuron activity and subjective “faceness” ratings, but this relationship did not hold for face neuron–evolved images, which triggered high activity but were rated low in faceness. Our results suggest that so-called face neurons are better described as tuned to visual features rather than semantic categories.

SweetPea: A standard language for factorial experimental design

Experimental design is a key ingredient of reproducible empirical research. Yet, given the increasing complexity of experimental designs, researchers often struggle to implement ones that allow them to measure their variables of interest without confounds. SweetPea ( https://sweetpea-org.github.io/ ) is an open-source declarative language in Python, in which researchers can describe their desired experiment as a set of factors and constraints. The language leverages advances in areas of computer science to sample experiment sequences in an unbiased way. In this article, we provide an overview of SweetPea's capabilities, and demonstrate its application to the design of psychological experiments. Finally, we discuss current limitations of SweetPea, as well as potential applications to other domains of empirical research, such as neuroscience and machine learning.

Leveraging artificial intelligence to improve people's planning strategies

SignificanceMany bad decisions and their devastating consequences could be avoided if people used optimal decision strategies. Here, we introduce a principled computational approach to improving human decision making. The basic idea is to give people feedback on how they reach their decisions. We develop a method that leverages artificial intelligence to generate this feedback in such a way that people quickly discover the best possible decision strategies. Our empirical findings suggest that a principled computational approach leads to improvements in decision-making competence that transfer to more difficult decisions in more complex environments. In the long run, this line of work might lead to apps that teach people clever strategies for decision making, reasoning, goal setting, planning, and goal achievement.

Online testing in developmental science: A guide to design and implementation

At present, most developmental psychology experiments use participants from a mere subsection of the world's population. Moreover, like other fields of psychology, many studies in developmental psychology suffer from low statistical power due to small samples and limited observations. Online testing holds promise as a way to achieve more representative and robust, better powered experiments. As participants do not have to visit in person, it is easier to access populations living further away from a developmental lab, enabling testing of more diverse populations (e.g., urban vs rural areas, various different nationalities or geographies), both within and beyond the researcher's home country. Furthermore, due to the codified nature of browser-based online testing, it is possible for multiple labs to carry out the exact same study, allowing for better replications. Because of these advantages, developmental researchers have started to move experiments online so that caregivers and their children can participate from their home environments. However, the transition from traditional lab testing to remote online testing brings many challenges. Laboratory studies of infant and child development are typically conducted under highly standardized conditions to control factors, such as distractors, distance to the screen, movement, and lighting, and often rely on specialized equipment for measuring behavior. In this chapter, we provide a guide for researchers considering online testing of a developmental population. The different sections comprise an overview of the decision-making processes and the state-of-the-art advances associated with, as well as tangible recommendations for, online data collection.

Latent motives guide structure learning during adaptive social choice

Predicting the behaviour of others is an essential part of social cognition. Despite its ubiquity, social prediction poses a poorly understood generalization problem: we cannot assume that others will repeat past behaviour in new settings or that their future actions are entirely unrelated to the past. We demonstrate that humans solve this challenge using a structure learning mechanism that uncovers other people's latent, unobservable motives, such as greed and risk aversion. In four studies, participants (N = 501) predicted other players' decisions across four economic games, each with different social tensions (for example, Prisoner's Dilemma and Stag Hunt). Participants achieved accurate social prediction by learning the stable motivational structure underlying a player's changing actions across games. This motive-based abstraction enabled participants to attend to information diagnostic of the player's next move and disregard irrelevant contextual cues. Participants who successfully learned another's motives were more strategic in a subsequent competitive interaction with that player in entirely new contexts, reflecting that social structure learning supports adaptive social behaviour.

Correction Without Consciousness in Complex Tasks: Evidence from Typing

It has been demonstrated that with practice, complex tasks can become independent of conscious control, but even in those cases, repairing errors is thought to remain dependent on conscious control. This paper reports two studies probing conscious awareness over repairs in nearly 15,000 typing errors collected from 145 participants in a single-word typing-to-dictation task. We provide evidence for subconscious repairs by ruling out alternative accounts, and report two sets of analyses showing that a) such repairs are not confined to a specific stage of processing and b) that they are sensitive to the final outcome of repair. A third set of analyses provides a detailed comparison of the timeline of trials with conscious and subconscious repairs, revealing that the difference is confined to the repair process itself. We propose an account of repair processing that accommodates these empirical findings.

Bringing Home Baby Euclid: Testing Infants' Basic Shape Discrimination Online

Online developmental psychology studies are still in their infancy, but their role is newly urgent in the light of the COVID-19 pandemic and the suspension of in-person research. Are online studies with infants a suitable stand-in for laboratory-based studies? Across two unmonitored online experiments using a change-detection looking-time paradigm with 96 7-month-old infants, we found that infants did not exhibit measurable sensitivities to the basic shape information that distinguishes between 2D geometric forms, as had been observed in previous laboratory experiments. Moreover, while infants were distracted in our online experiments, such distraction was nevertheless not a reliable predictor of their ability to discriminate shape information. Our findings suggest that the change-detection paradigm may not elicit infants' shape discrimination abilities when stimuli are presented on small, personal computer screens because infants may not perceive two discrete events with only one event displaying uniquely changing information that draws their attention. Some developmental paradigms used with infants, even those that seem well-suited to the constraints and goals of online data collection, may thus not yield results consistent with the laboratory results that rely on highly controlled settings and specialized equipment, such as large screens. As developmental researchers continue to adapt laboratory-based methods to online contexts, testing those methods online is a necessary first step in creating robust tools and expanding the space of inquiry for developmental science conducted online.

Dissociable influences of reward and punishment on adaptive cognitive control

To invest effort into any cognitive task, people must be sufficiently motivated. Whereas prior research has focused primarily on how the cognitive control required to complete these tasks is motivated by the potential rewards for success, it is also known that control investment can be equally motivated by the potential negative consequence for failure. Previous theoretical and experimental work has yet to examine how positive and negative incentives differentially influence the manner and intensity with which people allocate control. Here, we develop and test a normative model of control allocation under conditions of varying positive and negative performance incentives. Our model predicts, and our empirical findings confirm, that rewards for success and punishment for failure should differentially influence adjustments to the evidence accumulation rate versus response threshold, respectively. This dissociation further enabled us to infer how motivated a given person was by the consequences of success versus failure.

Auditory salience using natural scenes: An online study

Salience is the quality of a sensory signal that attracts involuntary attention in humans. While it primarily reflects conspicuous physical attributes of a scene, our understanding of processes underlying what makes a certain object or event salient remains limited. In the vision literature, experimental results, theoretical accounts, and large amounts of eye-tracking data using rich stimuli have shed light on some of the underpinnings of visual salience in the brain. In contrast, studies of auditory salience have lagged behind due to limitations in both experimental designs and stimulus datasets used to probe the question of salience in complex everyday soundscapes. In this work, we deploy an online platform to study salience using a dichotic listening paradigm with natural auditory stimuli. The study validates crowd-sourcing as a reliable platform to collect behavioral responses to auditory salience by comparing experimental outcomes to findings acquired in a controlled laboratory setting. A model-based analysis demonstrates the benefits of extending behavioral measures of salience to broader selection of auditory scenes and larger pools of subjects. Overall, this effort extends our current knowledge of auditory salience in everyday soundscapes and highlights the limitations of low-level acoustic attributes in capturing the richness of natural soundscapes.

Isolated Sandbox Environment Architecture for Running Cognitive Psychological Experiments in Web Platforms

Web surveys are an integral part of the feedback of Internet services, a research tool for respondents, including in the field of health and psychology. Web technologies allow conducting research on large samples. For mental health, an important metric is reaction time in cognitive tests and in answering questions. The use of mobile devices such as smartphones and tablets has increased markedly in web surveys, so the impact of device types and operating systems needs to be investigated. This article proposes an architectural solution aimed at reducing the effect of device variability on the results of cognitive psychological experiments. An experiment was carried out to formulate the requirements for software and hardware. Three groups of 1000 respondents were considered, corresponding to three types of computers and operating systems: Mobile Device, Legacy PC, and Modern PC. The results obtained showed a slight bias in the estimates for each group. It is noticed that the error for a group of devices differs both upward and downward for various tasks in a psychological experiment. Thus, for cognitive tests, in which the reaction time is critical, an architectural solution was synthesized for conducting psychological research in a web browser. The proposed architectural solution considers the characteristics of the device used by participants to undergo research in the web platform and allows to restrict access from devices that do not meet the specified criteria.

Moral dynamics: Grounding moral judgment in intuitive physics and intuitive psychology

When holding others morally responsible, we care about what they did, and what they thought. Traditionally, research in moral psychology has relied on vignette studies, in which a protagonist's actions and thoughts are explicitly communicated. While this research has revealed what variables are important for moral judgment, such as actions and intentions, it is limited in providing a more detailed understanding of exactly how these variables affect moral judgment. Using dynamic visual stimuli that allow for a more fine-grained experimental control, recent studies have proposed a direct mapping from visual features to moral judgments. We embrace the use of visual stimuli in moral psychology, but question the plausibility of a feature-based theory of moral judgment. We propose that the connection from visual features to moral judgments is mediated by an inference about what the observed action reveals about the agent's mental states, and what causal role the agent's action played in bringing about the outcome. We present a computational model that formalizes moral judgments of agents in visual scenes as computations over an intuitive theory of physics combined with an intuitive theory of mind. We test the model's quantitative predictions in three experiments across a wide variety of dynamic interactions.

Semantic influence on visual working memory of object identity and location

Does semantic information-in particular, regularities in category membership across objects-influence visual working memory (VWM) processing? We predict that the answer is "yes". Four experiments evaluating this prediction are reported. Experimental stimuli were images of real-world objects arranged in either one or two spatial clusters. On coherent trials, all objects belonging to a cluster also belonged to the same category. On incoherent trials, at least one cluster contained objects from different categories. Experiments using a change-detection paradigm (Experiments 1-3) and an experiment in which participants recalled the locations of objects in a scene (Experiment 4) yielded the same result: participants showed better memory performance on coherent trials than on incoherent trials. Taken as a whole, these experiments provide the best (perhaps only) data to date demonstrating that statistical regularities in semantic category membership improve VWM performance. Because a conventional perspective in cognitive science regards VWM as being sensitive solely to bottom-up visual properties of objects (e.g., shape, color, orientation), our results indicate that cognitive science may need to modify its conceptualization of VWM so that it is closer to "conceptual short-term memory", a short-term memory store representing current stimuli and their associated concepts (Potter, 1993, 2012).

Predicting responsibility judgments from dispositional inferences and causal attributions

The question of how people hold others responsible has motivated decades of theorizing and empirical work. In this paper, we develop and test a computational model that bridges the gap between broad but qualitative framework theories, and quantitative but narrow models. In our model, responsibility judgments are the result of two cognitive processes: a dispositional inference about a person's character from their action, and a causal attribution about the person's role in bringing about the outcome. We test the model in a group setting in which political committee members vote on whether or not a policy should be passed. We assessed participants' dispositional inferences and causal attributions by asking how surprising and important a committee member's vote was. Participants' answers to these questions in Experiment 1 accurately predicted responsibility judgments in Experiment 2. In Experiments 3 and 4, we show that the model also predicts moral responsibility judgments, and that importance matters more for responsibility, while surprise matters more for judgments of wrongfulness.

Honeycomb: a template for reproducible psychophysiological tasks for clinic, laboratory, and home use

Objective:

To improve the ability of psychiatry researchers to build, deploy, maintain, reproduce, and share their own psychophysiological tasks. Psychophysiological tasks are a useful tool for studying human behavior driven by mental processes such as cognitive control, reward evaluation, and learning. Neural mechanisms during behavioral tasks are often studied via simultaneous electrophysiological recordings. Popular online platforms such as Amazon Mechanical Turk (MTurk) and Prolific enable deployment of tasks to numerous participants simultaneously. However, there is currently no task-creation framework available for flexibly deploying tasks both online and during simultaneous electrophysiology.

Methods:

We developed a task creation template, termed Honeycomb, that standardizes best practices for building jsPsych-based tasks. Honeycomb offers continuous deployment configurations for seamless transition between use in research settings and at home. Further, we have curated a public library, termed BeeHive, of ready-to-use tasks.

Results:

We demonstrate the benefits of using Honeycomb tasks with a participant in an ongoing study of deep brain stimulation for obsessive compulsive disorder, who completed repeated tasks both in the clinic and at home.

Conclusion:

Honeycomb enables researchers to deploy tasks online, in clinic, and at home in more ecologically valid environments and during concurrent electrophysiology.

Expectations about future learning influence moment-to-moment feelings of suspense

Suspense is a cognitive and affective state that is often experienced in the anticipation of information and contributes to the enjoyment and consumption of entertainment such as movies or sports. Ely et al. proposed a formal definition of suspense which relies upon predictions about future belief updates. In order to empirically evaluate this theory, we designed a task based on the casino card game Blackjack where a variety of suspense dynamics can be experimentally induced. Our behavioural data confirmed the explanatory power of this theory. We further compared this formulation with other heuristic models inspired by studies in other domains such as narratives and found that most heuristic models cannot well account for the specific temporal dynamics of suspense across wide range of game variants. We additionally propose a way to test whether experiencing greater levels of suspense motivates more game-playing. In summary, this work is an initial attempt to link formal models of information and uncertainty with affective cognitive states and motivation.

Much Ado About Nothing: The Mental Representation of Omissive Relations

When the absence of an event causes some outcome, it is an instance of omissive causation. For instance, not eating lunch may cause you to be hungry. Recent psychological proposals concur that the mind represents causal relations, including omissive causal relations, through mental simulation, but they disagree on the form of that simulation. One theory states that people represent omissive causes as force vectors; another states that omissions are representations of contrasting counterfactual simulations; a third argues that people think about omissions by representing sets of iconic possibilities – mental models – in a piecemeal fashion. In this paper, we tease apart the empirical predictions of the three theories and describe experiments that run counter to two of them. Experiments 1 and 2 show that reasoners can infer temporal relations from omissive causes – a pattern that contravenes the force theory. Experiment 3 asked participants to list the possibilities consistent with an omissive cause – it found that they tended to list particular privileged possibilities first, most often, and faster than alternative possibilities. The pattern is consistent with the model theory, but inconsistent with the contrast hypothesis. We marshal the evidence and explain why it helps to solve a long-standing debate about how the mind represents omissions.

Now You Hear Me, Later You Don't: The Immediacy of Linguistic Computation and the Representation of Speech

What happens to an acoustic signal after it enters the mind of a listener? Previous work has demonstrated that listeners maintain intermediate representations over time. However, the internal structure of such representations-be they the acoustic-phonetic signal or more general information about the probability of possible categories-remains underspecified. We present two experiments using a novel speaker-adaptation paradigm aimed at uncovering the format of speech representations. We exposed adult listeners (N = 297) to a speaker whose utterances contained acoustically ambiguous information concerning phones (and thus words), and we manipulated the temporal availability of disambiguating cues via visually presented text (presented before or after each utterance). Results from a traditional phoneme-categorization task showed that listeners adapted to a modified acoustic distribution when disambiguating text was provided before but not after the audio. These results support the position that speech representations consist of activation over categories and are inconsistent with direct maintenance of the acoustic-phonetic signal.

Getting to the heart of it: Multi-method exploration of nonconscious prioritization processes

Understanding the determinants of consciousness is crucial for theories that see it as functionally adaptive, and for explaining how consciousness affects higher-level cognition. The invention of continuous flash suppression (CFS), a long-duration suppression technique, resulted in a proliferation of research into the process of prioritization for consciousness. We developed a new technique, repeated masked suppression (RMS), that facilitates the measurement of long suppression times, but relies on different visual principles. RMS enables a theoretical leap: It allows scientists to examine the central process of prioritization across different suppression methods. In five experiments (n = 282) we collected chronometric RMS and CFS data, finding that the previously reported face inversion effect and the face priority-dimension generalize beyond CFS. Our results validate the use of multi-method designs in the study of prioritization for consciousness. Furthermore, we show how RMS could be used online to reach diverse samples, previously beyond the reach of consciousness science.

Advancing Developmental Science via Unmoderated Remote Research with Children

This article introduces an accessible approach to implementing unmoderated remote research in developmental science-research in which children and families participate in studies remotely and independently, without directly interacting with researchers. Unmoderated remote research has the potential to strengthen developmental science by: (1) facilitating the implementation of studies that are easily replicable, (2) allowing for new approaches to longitudinal studies and studies of parent-child interaction, and (3) including families from more diverse backgrounds and children growing up in more diverse environments in research. We describe an approach we have used to design and implement unmoderated remote research that is accessible to researchers with limited programming expertise, and we describe the resources we have made available on a new website (discoveriesonline.org) to help researchers get started with implementing this approach. We discuss the potential of this method for developmental science and highlight some challenges still to be overcome to harness the power of unmoderated remote research for advancing the field.