Stochastic satisficing account of confidence in uncertain value-based decisions

Temporal dynamics of the semantic versus affective representations of valence during reversal learning

Valence, the representation of a stimulus in terms of good or bad, plays a central role in models of affect, value-based learning theories, and value-based decision-making models. Previous work used Unconditioned Stimulus (US) to support a theoretical division between two different types of valence representations for a stimulus: the semantic representation of valence, i.e., stored accumulated knowledge about the value of the stimulus, and the affective representation of valence, i.e., the valence of the affective response to this stimulus. The current work extended past research by using a neutral Conditioned Stimulus (CS) in the context of reversal learning, a type of associative learning. The impact of expected uncertainty (the variability of rewards) and unexpected uncertainty (reversal) on the evolving temporal dynamics of the two types of valence representations of the CS was tested in two experiments. Results show that in an environment presenting the two types of uncertainty, the adaptation process (learning rate) of the choices and of the semantic valence representation is slower than the adaptation of the affective valence representation. In contrast, in environments with only unexpected uncertainty (i.e., fixed rewards), there is no difference in the temporal dynamics of the two types of valence representations. Implications for models of affect, value-based learning theories, and value-based decision-making models are discussed.

Using deep learning to predict human decisions and using cognitive models to explain deep learning models

Deep neural networks (DNNs) models have the potential to provide new insights in the study of cognitive processes, such as human decision making, due to their high capacity and data-driven design. While these models may be able to go beyond theory-driven models in predicting human behaviour, their opaque nature limits their ability to explain how an operation is carried out, undermining their usefulness as a scientific tool. Here we suggest the use of a DNN model as an exploratory tool to identify predictable and consistent human behaviour, and using explicit, theory-driven models, to characterise the high-capacity model. To demonstrate our approach, we trained an exploratory DNN model to predict human decisions in a four-armed bandit task. We found that this model was more accurate than two explicit models, a reward-oriented model geared towards choosing the most rewarding option, and a reward-oblivious model that was trained to predict human decisions without information about rewards. Using experimental simulations, we were able to characterise the exploratory model using the explicit models. We found that the exploratory model converged with the reward-oriented model’s predictions when one option was clearly better than the others, but that it predicted pattern-based explorations akin to the reward-oblivious model’s predictions. These results suggest that predictable decision patterns that are not solely reward-oriented may contribute to human decisions. Importantly, we demonstrate how theory-driven cognitive models can be used to characterise the operation of DNNs, making DNNs a useful explanatory tool in scientific investigation.

Adaptive Empathy: A Model for Learning Empathic Responses in Response to Feedback

Empathy is usually deployed in social interactions. Nevertheless, common measures and examinations of empathy study this construct in isolation from the person in distress. In this article we seek to extend the field of examination to include both empathizer and target to determine whether and how empathic responses are affected by feedback and learned through interaction. Building on computational approaches in feedback-based adaptations (e.g., no feedback, model-free and model-based learning), we propose a framework for understanding how empathic responses are learned on the basis of feedback. In this framework, adaptive empathy, defined as the ability to adapt one's empathic responses, is a central aspect of empathic skills and can provide a new dimension to the evaluation and investigation of empathy. By extending existing neural models of empathy, we suggest that adaptive empathy may be mediated by interactions between the neural circuits associated with valuation, shared distress, observation-execution, and mentalizing. Finally, we propose that adaptive empathy should be considered a prominent facet of empathic capabilities with the potential to explain empathic behavior in health and psychopathology.

Trusting and learning from others: immediate and long-term effects of learning from observation and advice

Social learning underpins our species's extraordinary success. Learning through observation has been investigated in several species, but learning from advice-where information is intentionally broadcast-is less understood. We used a pre-registered, online experiment (n = 1492) combined with computational modelling to examine learning through observation and advice. Participants were more likely to immediately follow advice than to copy an observed choice, but this was dependent upon trust in the adviser: highly paranoid participants were less likely to follow advice in the short term. Reinforcement learning modelling revealed two distinct patterns regarding the long-term effects of social information: some individuals relied fully on social information, whereas others reverted to trial-and-error learning. This variation may affect the prevalence and fidelity of socially transmitted information. Our results highlight the privileged status of advice relative to observation and how the assimilation of intentionally broadcast information is affected by trust in others.

Adaptive Empathy: Empathic Response Selection as a Dynamic, Feedback-Based Learning Process

Empathy allows us to respond to the emotional state of another person. Considering that an empathic interaction may last beyond the initial response, learning mechanisms may be involved in dynamic adaptation of the reaction to the changing emotional state of the other person. However, traditionally, empathy is assessed through sets of isolated reactions to another's distress. Here we address this gap by focusing on adaptive empathy, defined as the ability to learn and adjust one's empathic responses based on feedback. For this purpose, we designed a novel paradigm of associative learning in which participants chose one of two empathic strategies (reappraisal or distraction) to attenuate the distress of a target person, where one strategy had a higher probability of relieving distress. After each choice, participants received feedback about the success of their chosen strategy in relieving the target person's distress, which they could use to inform their future decisions. The results show that the participants made more accurate choices in the adaptive empathy condition than in a non-social control condition, pointing to an advantage for learning from social feedback. We found a correlation between adaptive empathy and a trait measure of cognitive empathy. These findings indicate that the ability to learn about the effectiveness of empathic responses may benefit from incorporating mentalizing abilities. Our findings provide a lab-based model for studying adaptive empathy and point to the potential contribution of learning theory to enhancing our understanding of the dynamic nature of empathy.

Perceived patterns in decisions from experience and their influence on choice variability and policy diversification: A response to Ashby, Konstantinidis, & Yechiam, 2017

Searching for and acting upon perceived patterns of regularity is a fundamental learning process critical for adapting to changes in the environment. Yet in more artificial, static settings, in which patterns do not exist, this mechanism could interfere with choice maximization and manifest as unexplained choice variability in later trials. Recently however, Ashby et al. (2017) found that choice variability in later trials of a repeated choice setting is correlated with levels of diversification in policy tasks, in which patterns can never be exploited. They concluded that in repeated choice tasks, choice-variability in later trials is unlikely the result of following perceived patterns. Here, we demonstrate that correlations between choice variability and policy diversification can actually be the result of pattern seeking, rather than serving as evidence against it. We review evidence for the robustness of pattern seeking mechanisms in repeated choices and explain how such mechanisms could in fact create the results observed by Ashby et al. To examine our interpretation for their results, we conducted a sequential dependencies analysis of their data and find evidence that many participants behaved as if they believed trials are inter-dependent, even though they were explicitly instructed that the environment is stationary. The results of a new experiment in which sequential patterns are directly manipulated support our interpretation: Experiencing patterns affected both choice variability in later trials and policy diversification. Finally, we argue that decisions from experience tasks are a valid tool to examine the generation of preferences via fundamental learning processes.