Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex

Neurocomputational mechanisms involved in adaptation to fluctuating intentions of others

Humans frequently interact with agents whose intentions can fluctuate between competition and cooperation over time. It is unclear how the brain adapts to fluctuating intentions of others when the nature of the interactions (to cooperate or compete) is not explicitly and truthfully signaled. Here, we use model-based fMRI and a task in which participants thought they were playing with another player. In fact, they played with an algorithm that alternated without signaling between cooperative and competitive strategies. We show that a neurocomputational mechanism with arbitration between competitive and cooperative experts outperforms other learning models in predicting choice behavior. At the brain level, the fMRI results show that the ventral striatum and ventromedial prefrontal cortex track the difference of reliability between these experts. When attributing competitive intentions, we find increased coupling between these regions and a network that distinguishes prediction errors related to competition and cooperation. These findings provide a neurocomputational account of how the brain arbitrates dynamically between cooperative and competitive intentions when making adaptive social decisions.

Value Representations of Spite Sensitivity in Psychosis on the Minnesota Trust Game

BACKGROUND

Spite sensitivity provides a valuable construct to understand persecutory ideation and its underlying neural mechanisms. We examined the relationship between persecution and spite sensitivity in psychosis to identify their neural substrates.

METHODS

In a 3T magnetic resonance imaging scanner, 49 participants with psychosis played the Minnesota Trust Game, in which they decided whether to take a small amount of money or trust a partner to choose between fair and unfair distributions of money. In some conditions, the partner benefited from the unfair option, while in others, the partner lost money. Participants who were untrusting in the second condition (suspiciousness) showed heightened sensitivity to spite. Behavioral measures included mistrust during the 2 conditions of the game, which were compared with Brief Psychiatric Rating Scale persecution and computational modeling. Functional connectivity and blood oxygen level-dependent analyses were also conducted on a priori regions during spite-sensitive decisions.

RESULTS

Behavioral results replicated previous findings; participants who experienced more persecutory ideation trusted less, specifically in the suspiciousness condition. Functional connectivity findings showed that decreased connectivity between the orbitofrontal cortex-insula and the left frontoparietal network was associated with increased persecutory ideation and estimated spite-guilt (a marker of spite sensitivity). Additionally, we found differences between conditions in caudate nucleus, medial prefrontal cortex, and lateral orbitofrontal cortex activation.

CONCLUSIONS

These findings provide a new perspective on the origin of positive symptoms by identifying primary brain circuits that are related to both spite sensitivity and persecutory ideation.

Autonomous behaviour and the limits of human volition

Humans and some other animals can autonomously generate action choices that contribute to solving complex problems. However, experimental investigations of the cognitive bases of human autonomy are challenging, because experimental paradigms typically constrain behaviour using controlled contexts, and elicit behaviour by external triggers. In contrast, autonomy and freedom imply unconstrained behaviour initiated by endogenous triggers. Here we propose a new theoretical construct of adaptive autonomy, meaning the capacity to make behavioural choices that are free from constraints of both immediate external triggers and of routine response patterns, but nevertheless show appropriate coordination with the environment. Participants (N = 152) played a competitive game in which they had to choose the right time to act, in the face of an opponent who punished (in separate blocks) either choice biases (such as always responding early), sequential patterns of action timing across trials (such as early, late, early, late…), or predictable action-outcome dependence (such as win-stay, lose-shift). Adaptive autonomy was quantified as the ability to maintain performance when each of these influences on action selection was punished. We found that participants could become free from habitual choices regarding when to act and could also become free from sequential action patterns. However, they were not able to free themselves from influences of action-outcome dependence, even when these resulted in poor performance. These results point to a new concept of autonomous behaviour as flexible adaptation of voluntary action choices in a way that avoids stereotypy. In a sequential analysis, we also demonstrated that participants increased their reliance on belief learning in which they attempt to understand the competitor's beliefs and intentions, when transition bias and reinforcement bias were punished. Taken together, our study points to a cognitive mechanism of adaptive autonomy in which competitive interactions with other agents could promote both social cognition and volition in the form of non-stereotyped action choices.

Neurons in the monkey frontopolar cortex encode learning stage and goal during a fast learning task

The frontopolar cortex (FPC) is, to date, one of the least understood regions of the prefrontal cortex. The current understanding of its function suggests that it plays a role in the control of exploratory behaviors by coordinating the activities of other prefrontal cortex areas involved in decision-making and exploiting actions based on their outcomes. Based on this hypothesis, FPC would drive fast-learning processes through a valuation of the different alternatives. In our study, we used a modified version of a well-known paradigm, the object-in-place (OIP) task, to test this hypothesis in electrophysiology. This paradigm is designed to maximize learning, enabling monkeys to learn in one trial, which is an ability specifically impaired after a lesion of the FPC. We showed that FPC neurons presented an extremely specific pattern of activity by representing the learning stage, exploration versus exploitation, and the goal of the action. However, our results do not support the hypothesis that neurons in the frontal pole compute an evaluation of different alternatives. Indeed, the position of the chosen target was strongly encoded at its acquisition, but the position of the unchosen target was not. Once learned, this representation was also found at the problem presentation, suggesting a monitoring activity of the synthetic goal preceding its acquisition. Our results highlight important features of FPC neurons in fast-learning processes without confirming their role in the disengagement of cognitive control from the current goals.

Exploring the role of mutual prediction in inter-brain synchronization during competitive interactions: an fNIRS hyperscanning investigation

Mutual prediction is crucial for understanding the mediation of bodily actions in social interactions. Despite this importance, limited studies have investigated neurobehavioral patterns under the mutual prediction hypothesis in natural competitive scenarios. To address this gap, our study employed functional near-infrared spectroscopy hyperscanning to examine the dynamics of real-time rock-paper-scissors games using a computerized paradigm with 54 participants. Firstly, our results revealed activations in the right inferior frontal gyrus, bilateral dorsolateral prefrontal cortex, and bilateral frontopolar cortex, each displaying distinct temporal profiles indicative of diverse cognitive processes during the task. Subsequently, a task-related increase in inter-brain synchrony was explicitly identified in the right dorsolateral prefrontal cortex, which supported the mutual prediction hypothesis across the two brains. Moreover, our investigation uncovered a close association between the coherence value in the right dorsolateral prefrontal cortex and the dynamic predictive performances of dyads using inter-subject representational similarity analysis. Finally, heightened inter-brain synchrony values were observed in the right dorsolateral prefrontal cortex before a draw compared to a no-draw scenario in the second block, suggesting that cross-brain signal patterns could be reflected in behavioral responses during competition. In summary, these findings provided initial support for expanding the understanding of cognitive processes underpinning natural competitive engagements.

Mapping causal links between prefrontal cortical regions and intra-individual behavioral variability

Intra-individual behavioral variability is significantly heightened by aging or neuropsychological disorders, however it is unknown which brain regions are causally linked to such variabilities. We examine response time (RT) variability in 21 macaque monkeys performing a rule-guided decision-making task. In monkeys with selective-bilateral lesions in the anterior cingulate cortex (ACC) or in the dorsolateral prefrontal cortex, cognitive flexibility is impaired, but the RT variability is significantly diminished. Bilateral lesions within the frontopolar cortex or within the mid-dorsolateral prefrontal cortex, has no significant effect on cognitive flexibility or RT variability. In monkeys with lesions in the posterior cingulate cortex, the RT variability significantly increases without any deficit in cognitive flexibility. The effect of lesions in the orbitofrontal cortex (OFC) is unique in that it leads to deficits in cognitive flexibility and a significant increase in RT variability. Our findings indicate remarkable dissociations in contribution of frontal cortical regions to behavioral variability. They suggest that the altered variability in OFC-lesioned monkeys is related to deficits in assessing and accumulating evidence to inform a rule-guided decision, whereas in ACC-lesioned monkeys it results from a non-adaptive decrease in decision threshold and consequently immature impulsive responses.

Distinct roles of the orbitofrontal cortex, ventral striatum, and dopamine neurons in counterfactual thinking of decision outcomes

Individuals often assess past decisions by comparing what was gained with what would have been gained had they acted differently. Thoughts of past alternatives that counter what actually happened are called "counterfactuals." Recent theories emphasize the role of the prefrontal cortex in processing counterfactual outcomes in decision-making, although how subcortical regions contribute to this process remains to be elucidated. Here we report a clear distinction among the roles of the orbitofrontal cortex, ventral striatum and midbrain dopamine neurons in processing counterfactual outcomes in monkeys. Our findings suggest that actually gained and counterfactual outcome signals are both processed in the cortico-subcortical network constituted by these regions but in distinct manners and integrated only in the orbitofrontal cortex in a way to compare these outcomes. This study extends the prefrontal theory of counterfactual thinking and provides key insights regarding how the prefrontal cortex cooperates with subcortical regions to make decisions using counterfactual information.

Competitive and cooperative games for probing the neural basis of social decision-making in animals

In a social environment, it is essential for animals to consider the behavior of others when making decisions. To quantitatively assess such social decisions, games offer unique advantages. Games may have competitive and cooperative components, modeling situations with antagonistic and shared objectives between players. Games can be analyzed by mathematical frameworks, including game theory and reinforcement learning, such that an animal's choice behavior can be compared against the optimal strategy. However, so far games have been underappreciated in neuroscience research, particularly for rodent studies. In this review, we survey the varieties of competitive and cooperative games that have been tested, contrasting strategies employed by non-human primates and birds with rodents. We provide examples of how games can be used to uncover neural mechanisms and explore species-specific behavioral differences. We assess critically the limitations of current paradigms and propose improvements. Together, the synthesis of current literature highlights the advantages of using games to probe the neural basis of social decisions for neuroscience studies.

Neural responses in macaque prefrontal cortex are linked to strategic exploration

Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on scarce resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback about the unchosen option. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen, which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.

Neurocomputational mechanisms of young children's observational learning of delayed gratification

The ability to delay gratification is crucial for a successful and healthy life. An effective way for young children to learn this ability is to observe the action of adult models. However, the underlying neurocomputational mechanism remains unknown. Here, we tested the hypotheses that children employed either the simple imitation strategy or the goal-inference strategy when learning from adult models in a high-uncertainty context. Results of computational modeling indicated that children used the goal-inference strategy regardless of whether the adult model was their mother or a stranger. At the neural level, results showed that successful learning of delayed gratification was associated with enhanced interpersonal neural synchronization (INS) between children and the adult models in the dorsal lateral prefrontal cortex but was not associated with children's own single-brain activity. Moreover, the discounting of future reward's value obtained from computational modeling of the goal-inference strategy was positively correlated with the strength of INS. These findings from our exploratory study suggest that, even for 3-year-olds, the goal-inference strategy is used to learn delayed gratification from adult models, and the learning strategy is associated with neural interaction between the brains of children and adult models.

Distinct forms of regret linked to resilience versus susceptibility to stress are regulated by region-specific CREB function in mice

Regret describes recognizing alternative actions could have led to better outcomes. It remains unclear whether regret derives from generalized mistake appraisal or instead comprises dissociable, action-specific processes. Using a neuroeconomic task, we found that mice were sensitive to fundamentally distinct types of regret following exposure to chronic social defeat stress or manipulations of CREB, a transcription factor implicated in stress action. Bias to make compensatory decisions after rejecting high-value offers (regret type I) was unique to stress-susceptible mice. Bias following the converse operation, accepting low-value offers (regret type II), was enhanced in stress-resilient mice and absent in stress-susceptible mice. CREB function in either the prefrontal cortex or nucleus accumbens was required to suppress regret type I but bidirectionally regulated regret type II. We provide insight into how maladaptive stress response traits relate to distinct forms of counterfactual thinking, which could steer therapy for mood disorders, such as depression, toward circuit-specific computations through a careful description of decision narrative.

Hippocampal signals modify orbitofrontal representations to learn new paths

We often remember the consequences of past choices to adapt to changing circumstances. Recalling past events requires the hippocampus (HPC), and using stimuli to anticipate outcome values requires the orbitofrontal cortex (OFC).1-3 Spatial reversal tasks require both structures to navigate newly rewarded paths.4,5 Both HPC place6 and OFC value cells7,8 fire in phase with theta (4-12 Hz) oscillations. Both structures are described as cognitive maps: HPC maps space9 and OFC maps task states.10 These similarities imply that OFC-HPC interactions are crucial for using memory to predict outcomes when circumstances change, but the mechanisms remain largely unknown. To investigate possible interactions, we simultaneously recorded ensembles in OFC and CA1 as rats learned spatial reversals in a plus maze. Striking interactions occurred only while rats learned their first reversal: CA1 population vectors predicted changes in OFC activity but not vice versa, OFC spikes phase locked to hippocampal theta oscillations, mixed pairs of CA1 and OFC neurons fired together within single theta cycles, and CA1 led OFC spikes by ∼30 ms. After the new contingency became familiar, CA1 ensembles stably represented distinct spatial paths, whereas OFC ensembles developed more generalized goal arm representations in different paths to identical rewards. These frontotemporal interactions, engaged selectively when new task features inform decision-making, suggest a mechanism for linking novel episodes with expected outcomes, when HPC signals trigger "cognitive remapping" by OFC.11.

Assessing behavioural profiles following neutral, positive and negative feedback

Previous data suggest zero-value, neutral outcomes (draw) are subjectively assigned negative rather than positive valence. The combined observations of faster rather than slower reaction times, subsequent actions defined by shift rather than stay behaviour, reduced flexibility, and, larger rather than smaller deviations from optimal performance following draws all align with the consequences of explicitly negative outcomes such as losses. We further tested the relationships between neutral, positive and negative outcomes by manipulating value salience and observing their behavioural profiles. Despite speeded reaction times and a non-significant bias towards shift behaviour similar to losses when draws were assigned the value of 0 (Experiment 1), the degree of shift behaviour approached an approximation of optimal performance when the draw value was explicitly positive (+1). This was in contrast to when the draw value was explicitly negative (-1), which led to a significant increase in the degree of shift behaviour (Experiment 2). Similar modifications were absent when the same value manipulations were applied to win or lose trials (Experiment 3). Rather than viewing draws as neutral and valence-free outcomes, the processing cascade generated by draws produces a complex behavioural profile containing elements found in response to both explicitly positive and explicitly negative results.

Neurocomputations of strategic behavior: From iterated to novel interactions

Strategic interactions, where an individual's payoff depends on the decisions of multiple intelligent agents, are ubiquitous among social animals. They span a variety of important social behaviors such as competition, cooperation, coordination, and communication, and often involve complex, intertwining cognitive operations ranging from basic reward processing to higher‐order mentalization. Here, we review the progress and challenges in probing the neural and cognitive mechanisms of strategic behavior of interacting individuals, drawing an analogy to recent developments in studies of reward‐seeking behavior, in particular, how research focuses in the field of strategic behavior have been expanded from adaptive behavior based on trial‐and‐error to flexible decisions based on limited prior experience. We highlight two important research questions in the field of strategic behavior: (i) How does the brain exploit past experience for learning to behave strategically? and (ii) How does the brain decide what to do in novel strategic situations in the absence of direct experience? For the former, we discuss the utility of learning models that have effectively connected various types of neural data with strategic learning behavior and helped elucidate the interplay among multiple learning processes. For the latter, we review the recent evidence and propose a neural generative mechanism by which the brain makes novel strategic choices through simulating others' goal‐directed actions according to rational or bounded‐rational principles obtained through indirect social knowledge.

This article is categorized under:

Economics > Interactive Decision‐Making

Psychology > Reasoning and Decision Making

Neuroscience > Cognition

Taking stock of value in the orbitofrontal cortex

People with damage to the orbitofrontal cortex (OFC) have specific problems making decisions, whereas their other cognitive functions are spared. Neurophysiological studies have shown that OFC neurons fire in proportion to the value of anticipated outcomes. Thus, a central role of the OFC is to guide optimal decision-making by signalling values associated with different choices. Until recently, this view of OFC function dominated the field. New data, however, suggest that the OFC may have a much broader role in cognition by representing cognitive maps that can be used to guide behaviour and that value is just one of many variables that are important for behavioural control. In this Review, we critically evaluate these two alternative accounts of OFC function and examine how they might be reconciled.

Impulsivity and Emotional Dysregulation Predict Choice Behavior During a Mixed-Strategy Game in Adolescents With Borderline Personality Disorder

Impulsivity and emotional dysregulation are two core features of borderline personality disorder (BPD), and the neural mechanisms recruited during mixed-strategy interactions overlap with frontolimbic networks that have been implicated in BPD. We investigated strategic choice patterns during the classic two-player game, Matching Pennies, where the most efficient strategy is to choose each option randomly from trial-to-trial to avoid exploitation by one’s opponent. Twenty-seven female adolescents with BPD (mean age: 16 years) and twenty-seven age-matched female controls (mean age: 16 years) participated in an experiment that explored the relationship between strategic choice behavior and impulsivity in both groups and emotional dysregulation in BPD. Relative to controls, BPD participants showed marginally fewer reinforcement learning biases, particularly decreased lose-shift biases, increased variability in reaction times (coefficient of variation; CV), and a greater percentage of anticipatory decisions. A subset of BPD participants with high levels of impulsivity showed higher overall reward rates, and greater modulation of reaction times by outcome, particularly following loss trials, relative to control and BPD participants with lower levels of impulsivity. Additionally, BPD participants with higher levels of emotional dysregulation showed marginally increased reward rate and increased entropy in choice patterns. Together, our preliminary results suggest that impulsivity and emotional dysregulation may contribute to variability in mixed-strategy decision-making in female adolescents with BPD.

The whole prefrontal cortex is premotor cortex

We propose that the entirety of the prefrontal cortex (PFC) can be seen as fundamentally premotor in nature. By this, we mean that the PFC consists of an action abstraction hierarchy whose core function is the potentiation and depotentiation of possible action plans at different levels of granularity. We argue that the apex of the hierarchy should revolve around the process of goal-selection, which we posit is inherently a form of optimization over action abstraction. Anatomical and functional evidence supports the idea that this hierarchy originates on the orbital surface of the brain and extends dorsally to motor cortex. Accordingly, our viewpoint positions the orbitofrontal cortex in a key role in the optimization of goal-selection policies, and suggests that its other proposed roles are aspects of this more general function. Our proposed perspective will reframe outstanding questions, open up new areas of inquiry and align theories of prefrontal function with evolutionary principles. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Reinforcement learning detuned in addiction: integrative and translational approaches

Suboptimal decision-making strategies have been proposed to contribute to the pathophysiology of addiction. Decision-making, however, arises from a collection of computational components that can independently influence behavior. Disruptions in these different components can lead to decision-making deficits that appear similar behaviorally, but differ at the computational, and likely the neurobiological, level. Here, we discuss recent studies that have used computational approaches to investigate the decision-making processes underlying addiction. Studies in animal models have found that value updating following positive, but not negative, outcomes is predictive of drug use, whereas value updating following negative, but not positive, outcomes is disrupted following drug self-administration. We contextualize these findings with studies on the circuit and biological mechanisms of decision-making to develop a framework for revealing the biobehavioral mechanisms of addiction.

Differential coding of goals and actions in ventral and dorsal corticostriatal circuits during goal-directed behavior

Goal-directed behavior requires identifying objects in the environment that can satisfy internal needs and executing actions to obtain those objects. The current study examines ventral and dorsal corticostriatal circuits that support complementary aspects of goal-directed behavior. We analyze activity from the amygdala, ventral striatum, orbitofrontal cortex, and lateral prefrontal cortex (LPFC) while monkeys perform a three-armed bandit task. Information about chosen stimuli and their value is primarily encoded in the amygdala, ventral striatum, and orbitofrontal cortex, while the spatial information is primarily encoded in the LPFC. Before the options are presented, information about the to-be-chosen stimulus is represented in the amygdala, ventral striatum, and orbitofrontal cortex; at the time of choice, the information is passed to the LPFC to direct a saccade. Thus, learned value information specifying behavioral goals is maintained throughout the ventral corticostriatal circuit, and it is routed through the dorsal circuit at the time actions are selected.

The neural substrate and underlying mechanisms of executive control fluctuations in primates

Trial-by-trial alterations in response time have been linked to fluctuations of executive control and transient lapses of attention. Here, we report remarkable homologies in performance-dependent fluctuations of response time between humans and monkeys. We examined the effects of selective bilateral lesions in four frontal regions on control fluctuations in the context of a rule-shifting task. Lesions within orbitofrontal cortex (OFC), but not within superior-lateral prefrontal cortex, significantly exaggerated the performance-dependent fluctuations of control and prevented its restoration following feedback. Lesions within dorsolateral prefrontal cortex (DLPFC) or within anterior-cingulate cortex (ACC) led to instability of control and disruption of its link with monkeys' upcoming decisions. Examining the activity of DLPFC and OFC cells shed more lights on the underlying neuronal mechanisms by showing that before the start of each trial, OFC cell activity conveyed detailed information regarding the current state of executive control and the likelihood of success or failure in the future decisions. This further emphasizes the crucial role of OFC in the trial-by-trial allocation (setting) of control to the ongoing task. These findings bring insights to the neural architecture of executive control in primates and suggest that DLPFC and ACC support sustained executive control, but OFC is more involved in setting and restoring the control.

Thrifty-Eating Behavior Phenotype at the Food Court - Programming Goes Beyond Food Preferences

INTRODUCTION

Prenatal growth impairment leads to higher preference for palatable foods in comparison to normal prenatal growth subjects, which can contribute to increased body fat mass and a higher risk for developing chronic diseases in small-for-gestational-age (SGA) individuals throughout life. This study aimed to investigate the effect of SGA on feeding behavior in children and adolescents, as well as resting-state connectivity between areas related to reward, self-control, and value determination, such as orbitofrontal cortex (OFC), dorsolateral prefrontal cortex (DL-PFC), amygdala and dorsal striatum (DS).

METHODS

Caregivers and their offspring were recruited from two independent cohorts in Brazil (PROTAIA) and Canada (MAVAN). Both cohorts included anthropometric measurements, food choice tasks, and resting-state functional magnetic resonance imaging (fMRI) data.

RESULTS

In the Brazilian sample (17 ± 0.28 years, n=70), 21.4% of adolescents were classified as SGA. They exhibited lower monetary-related expenditure to buy a snack compared to controls in the food choice test. Decreased functional connectivity (n=40) between left OFC and left DL-PFC; and between right OFC and: left amygdala, right DS, and left DS were observed in the Brazilian SGA participants. Canadian SGA participants (14.9%) had non-significant differences in comparison with controls in a food choice task at 4 years old ( ± 0.01, n=315). At a follow-up brain scan visit (10.21 ± 0.140 years, n=49), SGA participants (28.6%) exhibited higher connectivity between the left OFC and left DL-PFC, also higher connectivity between the left OFC and right DL-PFC. We did not observe significant anthropometric neither nutrients' intake differences between groups in both samples.

CONCLUSIONS

Resting-state fMRI results showed that SGA individuals had altered connectivity between areas involved in encoding the subjective value for available goods and decision-making in both samples, which can pose them in disadvantage when facing food options daily. Over the years, the cumulative exposure to particular food cues together with the altered behavior towards food, such as food purchasing, as seen in the adolescent cohort, can play a role in the long-term risk for developing chronic non-communicable diseases.

The Neural Instantiation of an Abstract Cognitive Map for Economic Choice

Since the discovery of cognitive maps in rodent hippocampus (HC), the cognitive map has evolved from originally referring to spatial representations encoding locations and objects in Euclidean spaces to a general low-dimensional organization of information along selected feature dimensions. A cognitive map includes hypothetical constructs that bridge between environmental stimuli and the final overt behavior. To neuroeconomists, utility and utility functions are such constructs with neurobiological basis that drive choice behavior. Emergence of distinct functional neuron groups in the primate orbitofrontal cortex (OFC) during simple economic choice indicates the formation of an abstract cognitive map for organizing information of goods for value computation. Experimental evidence suggests that organization of neuronal activity in such cognitive map reflects the abstraction of core task features. Thus, such map can be adapted to accommodate economic choices under various task contexts.

Reinforcement learning and its connections with neuroscience and psychology

Reinforcement learning methods have recently been very successful at performing complex sequential tasks like playing Atari games, Go and Poker. These algorithms have outperformed humans in several tasks by learning from scratch, using only scalar rewards obtained through interaction with their environment. While there certainly has been considerable independent innovation to produce such results, many core ideas in reinforcement learning are inspired by phenomena in animal learning, psychology and neuroscience. In this paper, we comprehensively review a large number of findings in both neuroscience and psychology that evidence reinforcement learning as a promising candidate for modeling learning and decision making in the brain. In doing so, we construct a mapping between various classes of modern RL algorithms and specific findings in both neurophysiological and behavioral literature. We then discuss the implications of this observed relationship between RL, neuroscience and psychology and its role in advancing research in both AI and brain science.

Characterizing human random-sequence generation in competitive and non-competitive environments using Lempel-Ziv complexity

The human ability for random-sequence generation (RSG) is limited but improves in a competitive game environment with feedback. However, it remains unclear how random people can be during games and whether RSG during games can improve when explicitly informing people that they must be as random as possible to win the game. Nor is it known whether any such improvement in RSG transfers outside the game environment. To investigate this, we designed a pre/post intervention paradigm around a Rock-Paper-Scissors game followed by a questionnaire. During the game, we manipulated participants’ level of awareness of the computer’s strategy; they were either (a) not informed of the computer’s algorithm or (b) explicitly informed that the computer used patterns in their choice history against them, so they must be maximally random to win. Using a compressibility metric of randomness, our results demonstrate that human RSG can reach levels statistically indistinguishable from computer pseudo-random generators in a competitive-game setting. However, our results also suggest that human RSG cannot be further improved by explicitly informing participants that they need to be random to win. In addition, the higher RSG in the game setting does not transfer outside the game environment. Furthermore, we found that the underrepresentation of long repetitions of the same entry in the series explains up to 29% of the variability in human RSG, and we discuss what might make up the variance left unexplained.

Five Breakthroughs: A First Approximation of Brain Evolution From Early Bilaterians to Humans

Retracing the evolutionary steps by which human brains evolved can offer insights into the underlying mechanisms of human brain function as well as the phylogenetic origin of various features of human behavior. To this end, this article presents a model for interpreting the physical and behavioral modifications throughout major milestones in human brain evolution. This model introduces the concept of a "breakthrough" as a useful tool for interpreting suites of brain modifications and the various adaptive behaviors these modifications enabled. This offers a unique view into the ordered steps by which human brains evolved and suggests several unique hypotheses on the mechanisms of human brain function.

Memory and decision making interact to shape the value of unchosen options

The goal of deliberation is to separate between options so that we can commit to one and leave the other behind. However, deliberation can, paradoxically, also form an association in memory between the chosen and unchosen options. Here, we consider this possibility and examine its consequences for how outcomes affect not only the value of the options we chose, but also, by association, the value of options we did not choose. In five experiments (total n = 612), including a preregistered experiment (n = 235), we found that the value assigned to unchosen options is inversely related to their chosen counterparts. Moreover, this inverse relationship was associated with participants' memory of the pairs they chose between. Our findings suggest that deciding between options does not end the competition between them. Deliberation binds choice options together in memory such that the learned value of one can affect the inferred value of the other.

What Behavioral Abilities Emerged at Key Milestones in Human Brain Evolution? 13 Hypotheses on the 600-Million-Year Phylogenetic History of Human Intelligence

This paper presents 13 hypotheses regarding the specific behavioral abilities that emerged at key milestones during the 600-million-year phylogenetic history from early bilaterians to extant humans. The behavioral, intellectual, and cognitive faculties of humans are complex and varied: we have abilities as diverse as map-based navigation, theory of mind, counterfactual learning, episodic memory, and language. But these faculties, which emerge from the complex human brain, are likely to have evolved from simpler prototypes in the simpler brains of our ancestors. Understanding the order in which behavioral abilities evolved can shed light on how and why our brains evolved. To propose these hypotheses, I review the available data from comparative psychology and evolutionary neuroscience.

Hippocampal neurons construct a map of an abstract value space

The hippocampus is thought to encode a "cognitive map," a structural organization of knowledge about relationships in the world. Place cells, spatially selective hippocampal neurons that have been extensively studied in rodents, are one component of this map, describing the relative position of environmental features. However, whether this map extends to abstract, cognitive information remains unknown. Using the relative reward value of cues to define continuous "paths" through an abstract value space, we show that single neurons in primate hippocampus encode this space through value place fields, much like a rodent's place neurons encode paths through physical space. Value place fields remapped when cues changed but also became increasingly correlated across contexts, allowing maps to become generalized. Our findings help explain the critical contribution of the hippocampus to value-based decision-making, providing a mechanism by which knowledge of relationships in the world can be incorporated into reward predictions for guiding decisions.

Over-representation of fundamental decision variables in the prefrontal cortex underlies decision bias

The brain is organized into anatomically distinct structures consisting of a variety of projection neurons. While such evolutionarily conserved neural circuit organization underlies the innate ability of animals to swiftly adapt to environments, they can cause biased cognition and behavior. Although recent studies have begun to address the causal importance of projection-neuron types as distinct computational units, it remains unclear how projection types are functionally organized in encoding variables during cognitive tasks. This review focuses on the neural computation of decision making in the prefrontal cortex and discusses what decision variables are encoded by single neurons, neuronal populations, and projection type, alongside how specific projection types constrain decision making. We focus particularly on "over-representations" of distinct decision variables in the prefrontal cortex that reflect the biological and subjective significance of the variables for the decision makers. We suggest that task-specific over-representation in the prefrontal cortex involves the refinement of the given decision making, while generalized over-representation of fundamental decision variables is associated with suboptimal decision biases, including pathological ones such as those in patients with psychiatric disorders. Such over-representation of the fundamental decision variables in the prefrontal cortex appear to be tightly constrained by afferent and efferent connections that can be optogenetically intervened on. These ideas may provide critical insights into potential therapeutic targets for psychiatric disorders, including addiction and depression.

Prefrontal tDCS attenuates counterfactual thinking in female individuals prone to self-critical rumination

The tendency to ruminate (i.e., repetitive negative self-referential thoughts that perpetuate depressive mood) is associated with (a) an elevated propensity to maladaptively experience counterfactual thinking (CFT) and regret, and (b) hypo-activity of the left dorsolateral prefrontal cortex (DLPFC). The goal of this study was to investigate whether anodal transcranial direct current stimulation (tDCS) over the left DLPFC, in function of self-critical rumination tendencies, momentarily reduces counterfactual thinking and regret (assessed via self-report and psychophysiological indices). Eighty healthy participants with different levels of self-critical rumination received either anodal or sham tDCS while performing a decision making task in which they were repeatedly confronted with optimal, suboptimal, and non-optimal choice outcomes. The results showed that among rumination-prone individuals, anodal (versus sham) tDCS was associated with decreased CFT and attenuated psychophysiological reactivity to the differential choice outcomes. Conversely, among low rumination-prone individuals, anodal (versus sham) tDCS was associated with increased CFT and regret, but in absence of any effects on psychophysiological reactivity. Potential working mechanisms for these differential tDCS effects are discussed. Taken together, these results provide initial converging evidence for the adaptive effects of left prefrontal tDCS on CFT and regret to personal choice outcomes among individuals prone to engage in self-critical rumination.

Reward and fictive prediction error signals in ventral striatum: asymmetry between factual and counterfactual processing

Reward prediction error, the difference between the expected and obtained reward, is known to act as a reinforcement learning neural signal. In the current study, we propose a model fitting approach that combines behavioral and neural data to fit computational models of reinforcement learning. Briefly, we penalized subject-specific fitted parameters that moved away too far from the group median, except when that deviation led to an improvement in the model's fit to neural responses. By means of a probabilistic monetary learning task and fMRI, we compared our approach with standard model fitting methods. Q-learning outperformed actor-critic at both behavioral and neural level, although the inclusion of neuroimaging data into model fitting improved the fit of actor-critic models. We observed both action-value and state-value prediction error signals in the striatum, while standard model fitting approaches failed to capture state-value signals. Finally, left ventral striatum correlated with reward prediction error while right ventral striatum with fictive prediction error, suggesting a functional hemispheric asymmetry regarding prediction-error driven learning.

The computational origin of representation

Each of our theories of mental representation provides some insight into how the mind works. However, these insights often seem incompatible, as the debates between symbolic, dynamical, emergentist, sub-symbolic, and grounded approaches to cognition attest. Mental representations-whatever they are-must share many features with each of our theories of representation, and yet there are few hypotheses about how a synthesis could be possible. Here, I develop a theory of the underpinnings of symbolic cognition that shows how sub-symbolic dynamics may give rise to higher-level cognitive representations of structures, systems of knowledge, and algorithmic processes. This theory implements a version of conceptual role semantics by positing an internal universal representation language in which learners may create mental models to capture dynamics they observe in the world. The theory formalizes one account of how truly novel conceptual content may arise, allowing us to explain how even elementary logical and computational operations may be learned from a more primitive basis. I provide an implementation that learns to represent a variety of structures, including logic, number, kinship trees, regular languages, context-free languages, domains of theories like magnetism, dominance hierarchies, list structures, quantification, and computational primitives like repetition, reversal, and recursion. This account is based on simple discrete dynamical processes that could be implemented in a variety of different physical or biological systems. In particular, I describe how the required dynamics can be directly implemented in a connectionist framework. The resulting theory provides an "assembly language" for cognition, where high-level theories of symbolic computation can be implemented in simple dynamics that themselves could be encoded in biologically plausible systems.

Variability in competitive decision-making speed and quality against exploiting and exploitative opponents

A presumption in previous work has been that sub-optimality in competitive performance following loss is the result of a reduction in decision-making time (i.e., post-error speeding). The main goal of this paper is to test the relationship between decision-making speed and quality, with the hypothesis that slowing down decision-making should increase the likelihood of successful performance in cases where a model of opponent domination can be implemented. Across Experiments 1–3, the speed and quality of competitive decision-making was examined in a zero-sum game as a function of the nature of the opponent (unexploitable, exploiting, exploitable). Performance was also examined against the nature of a credit (or token) system used as a within-experimental manipulation (no credit, fixed credit, variable credit). To compliment reaction time variation as a function of outcome, both the fixed credit and variable credit conditions were designed to slow down decision-making, relative to a no credit condition where the game could be played in quick succession and without interruption. The data confirmed that (a) self-imposed reductions in processing time following losses (post-error speeding) were causal factors in determining poorer-quality behaviour, (b) the expression of lose-shift was less flexible than the expression of win-stay, and, (c) the use of a variable credit system may enhance the perceived control participants have against exploitable opponents. Future work should seek to disentangle temporal delay and response interruption as determinants of decision-making quality against numerous styles of opponency.

Capuchin and rhesus monkeys show sunk cost effects in a psychomotor task

Human decision-making is often swayed by irrecoverable investments even though it should only be based on future-and not past-costs and benefits. Although this sunk cost effect is widely documented and can lead to devastating losses, the underlying psychological mechanisms are unclear. To tease apart possible explanations through a comparative approach, we assessed capuchin and rhesus monkeys' susceptibility to sunk costs in a psychomotor task. Monkeys needed to track a moving target with a joystick-controlled cursor for variable durations. They could stop at any time, ending the trial without reward. To minimize the work required for a reward, monkeys should have always persisted for at least 1 s, but should have abandoned the trial if that did not yield a reward. Capuchin monkeys and especially rhesus macaques persisted to trial completion even when it was suboptimal, and were more likely to complete the trial the longer they had already tracked the target. These effects were less pronounced, although still present, when the change in expected tracking duration was signalled visually. These results show that sunk cost effects can arise in the absence of human-unique factors and may emerge, in part, because persisting can resolve uncertainty.

Switching Competitors Reduces Win-Stay but Not Lose-Shift Behaviour: The Role of Outcome-Action Association Strength on Reinforcement Learning

Predictability is a hallmark of poor-quality decision-making during competition. One source of predictability is the strong association between current outcome and future action, as dictated by the reinforcement learning principles of win–stay and lose–shift. We tested the idea that predictability could be reduced during competition by weakening the associations between outcome and action. To do this, participants completed a competitive zero-sum game in which the opponent from the current trial was either replayed (opponent repeat) thereby strengthening the association, or, replaced (opponent change) by a different competitor thereby weakening the association. We observed that win–stay behavior was reduced during opponent change trials but lose–shiftbehavior remained reliably predictable. Consistent with the group data, the number of individuals who exhibited predictable behavior following wins decreased for opponent change relative to opponent repeat trials. Our data show that future actions are more under internal control following positive relative to negative outcomes, and that externally breaking the bonds between outcome and action via opponent association also allows us to become less prone to exploitation.

The Rostrolateral Prefrontal Cortex Mediates a Preference for High-Agency Environments

The ability to exert flexible instrumental control over one's environment is a defining feature of adaptive decision-making. Here, we investigated neural substrates mediating a preference for environments with greater instrumental divergence, the distance between outcome probability distributions associated with alternative actions. A formal index of agency, instrumental divergence allows an organism to flexibly obtain the currently most desired outcome as preferences change. As such, it may have intrinsic utility, guiding decisions toward environments that maximize instrumental power. Consistent with this notion, we found that a measure of expected value that treats instrumental divergence as a reward surrogate provided a better account of male and female human participants' choice preferences than did a conventional model, sensitive only to monetary reward. Using model-based fMRI, we found that activity in the rostrolateral and ventromedial PFC, regions associated with abstract cognitive inferences and subjective value computations, respectively, scaled with the divergence-based account of expected value. Implications for a neural common currency of information theoretic and motivational variables are discussed.SIGNIFICANCE STATEMENT Agency is a central concept in philosophy and psychology. While research thus far has focused on cognitive and perceptual measures of agency, recent work demonstrating a strong preference for high-agency environments indicates a salient motivational dimension. Here, using instrumental divergence, the distance between outcome distributions associated with alternative actions, as a formal index of agency, we found that brain regions associated with directed exploration and subjective value computations, respectively, were selectively modulated by a model that treated agency as a reward surrogate, over models that assigned utility only to monetary payoffs. In a subset of regions, such effects were predicted by the influence of instrumental divergence on economic choice preferences. Our results elucidate neural mechanisms mediating the utility of agency.

Closed-Loop Theta Stimulation in the Orbitofrontal Cortex Prevents Reward-Based Learning

Neuronal oscillations in the frontal cortex have been hypothesized to play a role in the organization of high-level cognition. Within the orbitofrontal cortex (OFC), there is a prominent oscillation in the theta frequency (4-8 Hz) during reward-guided behavior, but it is unclear whether this oscillation has causal significance. One methodological challenge is that it is difficult to manipulate theta without affecting other neural signals, such as single-neuron firing rates. A potential solution is to use closed-loop control to record theta in real time and use this signal to control the application of electrical microstimulation to the OFC. Using this method, we show that theta oscillations in the OFC are critically important for reward-guided learning and that they are driven by theta oscillations in the hippocampus (HPC). The ability to disrupt OFC computations via spatially localized and temporally precise stimulation could lead to novel treatment strategies for neuropsychiatric disorders involving OFC dysfunction.

Targeted Stimulation of Human Orbitofrontal Networks Disrupts Outcome-Guided Behavior

Outcome-guided behavior requires knowledge about the current value of expected outcomes. Such behavior can be isolated in the reinforcer devaluation task, which assesses the ability to infer the current value of specific rewards after devaluation. Animal lesion studies demonstrate that orbitofrontal cortex (OFC) is necessary for normal behavior in this task, but a causal role for human OFC in outcome-guided behavior has not been established. Here, we used sham-controlled, non-invasive, continuous theta-burst stimulation (cTBS) to temporarily disrupt human OFC network activity by stimulating a site in the lateral prefrontal cortex that is strongly connected to OFC prior to devaluation of food odor rewards. Subjects in the sham group appropriately avoided Pavlovian cues associated with devalued food odors. However, subjects in the stimulation group persistently chose those cues, even though devaluation of food odors themselves was unaffected by cTBS. This behavioral impairment was mirrored in changes in resting-state functional magnetic resonance imaging (rs-fMRI) activity such that subjects in the stimulation group exhibited reduced OFC network connectivity after cTBS, and the magnitude of this reduction was correlated with choices after devaluation. These findings demonstrate the feasibility of indirectly targeting the human OFC with non-invasive cTBS and indicate that OFC is specifically required for inferring the value of expected outcomes.

Neuroscience: Reevaluating the Role of Orbitofrontal Cortex

A new optogenetic lesion study shows that the orbitofrontal cortex is essential for integrating information about recent rewards - which may either increase or decrease demand for more - with learned preferences to drive behavior.

A novel fMRI paradigm to dissociate the behavioral and neural components of mixed-strategy decision making from non-strategic decisions in humans

During competitive interactions, such as predator-prey or team sports, the outcome of one's actions is dependent on both their own choices and those of their opponents. Success in these rivalries requires that individuals choose dynamically and unpredictably, often adopting a mixed strategy. Understanding the neural basis of strategic decision making is complicated by the fact that it recruits various cognitive processes that are often shared with non-strategic forms of decision making, such as value estimation, working memory, response inhibition, response selection, and reward processes. Although researchers have explored neural activity within key brain regions during mixed-strategy games, how brain activity differs in the context of strategic interactions versus non-strategic choices is not well understood. We developed a novel behavioral paradigm to dissociate choice behavior during mixed-strategy interactions from non-strategic choices, and we used task-based functional magnetic resonance imaging (fMRI) to contrast brain activation. In a block design, participants competed in the classic mixed-strategy game, "matching pennies," against a dynamic computer opponent designed to exploit predictability in players' response patterns. Results were contrasted with a non-strategic task that had comparable sensory input, motor output, and reward rate; thus, differences in behavior and brain activation reflect strategic processes. The mixed-strategy game was associated with activation of a distributed cortico-striatal network compared to the non-strategic task. We propose that choosing in mixed-strategy contexts requires additional cognitive demands present to a lesser degree during the control task, illustrating the strength of this design in probing function of cognitive systems beyond core sensory, motor, and reward processes.

Behavioural and neural limits in competitive decision making: The roles of outcome, opponency and observation

To understand the boundaries we set for ourselves in terms of environmental responsibility during competition, we examined a neural index of outcome valence (feedback-related negativity; FRN) in relation to an early index of visual attention (N1), a later index of motivational significance (P3), and, eventual behaviour. In Experiment 1 (n = 36), participants either were (play) or were not (observe) responsible for action selection. In Experiment 2 (n = 36), opponents additionally either could (exploitable) or could not (unexploitable) be beaten. Various failures in reinforcement learning expression were revealed including large-scale approximations of random behaviour. Against unexploitable opponents, N1 determined the extent to which negative and positive outcomes were perceived as distinct categories by FRN. Against exploitable opponents, FRN determined the extent to which P3 generated neural gain for future events. Differential activation of the N1 - FRN - P3 processing chain provides a framework for understanding the behavioural dynamism observed during competitive decision making.

Behavioural Isomorphism, Cognitive Economy and Recursive Thought in Non-Transitive Game Strategy

Game spaces in which an organism must repeatedly compete with an opponent for mutually exclusive outcomes are critical methodologies for understanding decision-making under pressure. In the non-transitive game rock, paper, scissors (RPS), the only technique that guarantees the lack of exploitation is to perform randomly in accordance with mixed-strategy. However, such behavior is thought to be outside bounded rationality and so decision-making can become deterministic, predictable, and ultimately exploitable. This review identifies similarities across economics, neuroscience, nonlinear dynamics, human, and animal cognition literatures, and provides a taxonomy of RPS strategy. RPS strategies are discussed in terms of (a) whether the relevant computations require sensitivity to item frequency, the cyclic relationships between responses, or the outcome of the previous trial, and (b) whether the strategy is framed around the self or other. The negative implication of this taxonomy is that despite the differences in cognitive economy and recursive thought, many of the identified strategies are behaviorally isomorphic. This makes it difficult to infer strategy from behavior. The positive implication is that this isomorphism can be used as a novel design feature in furthering our understanding of the attribution, agency, and acquisition of strategy in RPS and other game spaces.

A Computational Model of Attention Control in Multi-Attribute, Context-Dependent Decision Making

Real-life decisions often require a comparison of multi-attribute options with various benefits and costs, and the evaluation of each option depends partly on the others in the choice set (i.e., the choice context). Although reinforcement learning models have successfully described choice behavior, how to account for multi-attribute information when making a context-dependent decision remains unclear. Here we develop a computational model of attention control that includes context effects on multi-attribute decisions, linking a context-dependent choice model with a reinforcement learning model. The overall model suggests that the distinctiveness of attributes guides an individual's preferences among multi-attribute options via an attention-control mechanism that determines whether choices are selectively biased toward the most distinctive attribute (selective attention) or proportionally distributed based on the relative distinctiveness of attributes (divided attention). To test the model, we conducted a behavioral experiment in rhesus monkeys, in which they made simple multi-attribute decisions over three conditions that manipulated the degree of distinctiveness between alternatives: (1) four foods of different size and calorie; (2) four pieces of the same food in different colors; and (3) four identical pieces of food. The model simulation of the choice behavior captured the preference bias (i.e., overall preference structure) and the choice persistence (repeated choices) in the empirical data, providing evidence for the respective influences of attention and memory on preference bias and choice persistence. Our study provides insights into computations underlying multi-attribute decisions, linking attentional control to decision-making processes.

Orbitofrontal Circuits Control Multiple Reinforcement-Learning Processes

Adaptive decision making in dynamic environments requires multiple reinforcement-learning steps that may be implemented by dissociable neural circuits. Here, we used a novel directionally specific viral ablation approach to investigate the function of several anatomically defined orbitofrontal cortex (OFC) circuits during adaptive, flexible decision making in rats trained on a probabilistic reversal learning task. Ablation of OFC neurons projecting to the nucleus accumbens selectively disrupted performance following a reversal, by disrupting the use of negative outcomes to guide subsequent choices. Ablation of amygdala neurons projecting to the OFC also impaired reversal performance, but due to disruptions in the use of positive outcomes to guide subsequent choices. Ablation of OFC neurons projecting to the amygdala, by contrast, enhanced reversal performance by destabilizing action values. Our data are inconsistent with a unitary function of the OFC in decision making. Rather, distinct OFC-amygdala-striatal circuits mediate distinct components of the action-value updating and maintenance necessary for decision making.

Dissociation between Private and Social Counterfactual Value Signals Following Ventromedial Prefrontal Cortex Damage

Individuals learn by comparing the outcome of chosen and unchosen actions. A negative counterfactual value signal is generated when this comparison is unfavorable. This can happen in private as well as in social settings—where the foregone outcome results from the choice of another person. We hypothesized that, despite sharing similar features such as supporting learning, these two counterfactual signals might implicate distinct brain networks. We conducted a neuropsychological study on the role of private and social counterfactual value signals in risky decision-making. Patients with lesions in the ventromedial prefrontal cortex (vmPFC), lesion controls, and healthy controls repeatedly chose between lotteries. In private trials, participants could observe the outcomes of their choices and the outcomes of the unselected lotteries. In social trials, participants could also see the other player's choices and outcome. At the time of outcome, vmPFC patients were insensitive to private counterfactual value signals, whereas their responses to social comparison were similar to those of control participants. At the time of choice, intact vmPFC was necessary to integrate counterfactual signals in decisions, although amelioration was observed during the course of the task, possibly driven by social trials. We conclude that if the vmPFC is critical in processing private counterfactual signals and in integrating those signals in decision-making, then distinct brain areas might support the processing of social counterfactual signals.

The macaque anterior cingulate cortex translates counterfactual choice value into actual behavioral change

The neural mechanisms mediating sensory-guided decision making have received considerable attention but animals often pursue behaviors for which there is currently no sensory evidence. Such behaviors are guided by internal representations of choice values that have to be maintained even when these choices are unavailable. We investigated how four macaque monkeys maintained representations of the value of counterfactual choices– choices that could not be taken at the current moment but which could be taken in the future. Using functional magnetic resonance imaging, we found two different patterns of activity co-varying with values of counterfactual choices in a circuit spanning hippocampus, anterior lateral prefrontal cortex, and anterior cingulate cortex (ACC). ACC activity also reflected whether the internal value representations would be translated into actual behavioral change. To establish the causal importance of ACC for this translation process, we used a novel technique, Transcranial Focused Ultrasound Stimulation, to reversibly disrupt ACC activity.

Beyond "incentive hope": Information sampling and learning under reward uncertainty

Information seeking, especially when motivated by strategic learning and intrinsic curiosity, could render the new mechanism "incentive hope" proposed by Anselme & Güntürkün sufficient, but not necessary to explain how reward uncertainty promotes reward seeking and consumption. Naturalistic and foraging-like tasks can help parse motivational processes that bridge learning and foraging behaviors and identify their neural underpinnings.