The effects of induced positive and negative affect on Pavlovian-instrumental interactions

Artificial neural networks for model identification and parameter estimation in computational cognitive models

Computational cognitive models have been used extensively to formalize cognitive processes. Model parameters offer a simple way to quantify individual differences in how humans process information. Similarly, model comparison allows researchers to identify which theories, embedded in different models, provide the best accounts of the data. Cognitive modeling uses statistical tools to quantitatively relate models to data that often rely on computing/estimating the likelihood of the data under the model. However, this likelihood is computationally intractable for a substantial number of models. These relevant models may embody reasonable theories of cognition, but are often under-explored due to the limited range of tools available to relate them to data. We contribute to filling this gap in a simple way using artificial neural networks (ANNs) to map data directly onto model identity and parameters, bypassing the likelihood estimation. We test our instantiation of an ANN as a cognitive model fitting tool on classes of cognitive models with strong inter-trial dependencies (such as reinforcement learning models), which offer unique challenges to most methods. We show that we can adequately perform both parameter estimation and model identification using our ANN approach, including for models that cannot be fit using traditional likelihood-based methods. We further discuss our work in the context of the ongoing research leveraging simulation-based approaches to parameter estimation and model identification, and how these approaches broaden the class of cognitive models researchers can quantitatively investigate.

Artificial neural networks for model identification and parameter estimation in computational cognitive models

Computational cognitive models have been used extensively to formalize cognitive processes. Model parameters offer a simple way to quantify individual differences in how humans process information. Similarly, model comparison allows researchers to identify which theories, embedded in different models, provide the best accounts of the data. Cognitive modeling uses statistical tools to quantitatively relate models to data that often rely on computing/estimating the likelihood of the data under the model. However, this likelihood is computationally intractable for a substantial number of models. These relevant models may embody reasonable theories of cognition, but are often under-explored due to the limited range of tools available to relate them to data. We contribute to filling this gap in a simple way using artificial neural networks (ANNs) to map data directly onto model identity and parameters, bypassing the likelihood estimation. We test our instantiation of an ANN as a cognitive model fitting tool on classes of cognitive models with strong inter-trial dependencies (such as reinforcement learning models), which offer unique challenges to most methods. We show that we can adequately perform both parameter estimation and model identification using our ANN approach, including for models that cannot be fit using traditional likelihood-based methods. We further discuss our work in the context of the ongoing research leveraging simulation-based approaches to parameter estimation and model identification, and how these approaches broaden the class of cognitive models researchers can quantitatively investigate.

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.

Mouse Behavior on the Trial-Unique Nonmatching-to-Location (TUNL) Touchscreen Task Reflects a Mixture of Distinct Working Memory Codes and Response Biases

The trial-unique nonmatching to location (TUNL) touchscreen task shows promise as a translational assay of working memory (WM) deficits in rodent models of autism, ADHD, and schizophrenia. However, the low-level neurocognitive processes that drive behavior in the TUNL task have not been fully elucidated. In particular, it is commonly assumed that the TUNL task predominantly measures spatial WM dependent on hippocampal pattern separation, but this proposition has not previously been tested. In this project, we tested this question using computational modeling of behavior from male and female mice performing the TUNL task (N = 163 across three datasets; 158,843 trials). Using this approach, we empirically tested whether TUNL behavior solely measured retrospective WM, or whether it was possible to deconstruct behavior into additional neurocognitive subprocesses. Overall, contrary to common assumptions, modeling analyses revealed that behavior on the TUNL task did not primarily reflect retrospective spatial WM. Instead, behavior was best explained as a mixture of response strategies, including both retrospective WM (remembering the spatial location of a previous stimulus) and prospective WM (remembering an anticipated future behavioral response) as well as animal-specific response biases. These results suggest that retrospective spatial WM is just one of a number of cognitive subprocesses that contribute to choice behavior on the TUNL task. We suggest that findings can be understood within a resource-rational framework, and use computational model simulations to propose several task-design principles that we predict will maximize spatial WM and minimize alternative behavioral strategies in the TUNL task.SIGNIFICANCE STATEMENT Touchscreen tasks represent a paradigm shift for assessment of cognition in nonhuman animals by automating large-scale behavioral data collection. Their main relevance, however, depends on the assumption of functional equivalence to cognitive domains in humans. The trial-unique, delayed nonmatching to location (TUNL) touchscreen task has revolutionized the study of rodent spatial working memory. However, its assumption of functional equivalence to human spatial working memory is untested. We leveraged previously untapped single-trial TUNL data to uncover a novel set of hierarchically ordered cognitive processes that underlie mouse behavior on this task. The strategies used demonstrate multiple cognitive approaches to a single behavioral outcome and the requirement for more precise task design and sophisticated data analysis in interpreting rodent spatial working memory.