Behavioral and neurophysiological correlates of regret in rat decision-making on a neuroeconomic task

Hippocampal representations of alternative possibilities are flexibly generated to meet cognitive demands

The cognitive ability to go beyond the present to consider alternative possibilities, including potential futures and counterfactual pasts, can support adaptive decision making. Complex and changing real-world environments, however, have many possible alternatives. Whether and how the brain can select among them to represent alternatives that meet current cognitive needs remains unknown. We therefore examined neural representations of alternative spatial locations in the rat hippocampus during navigation in a complex patch foraging environment with changing reward probabilities. We found representations of multiple alternatives along paths ahead and behind the animal, including in distant alternative patches. Critically, these representations were modulated in distinct patterns across successive trials: alternative paths were represented proportionate to their evolving relative value and predicted subsequent decisions, whereas distant alternatives were prevalent during value updating. These results demonstrate that the brain modulates the generation of alternative possibilities in patterns that meet changing cognitive needs for adaptive behavior.

The complex affective and cognitive capacities of rats

For several decades, although studies of rat physiology and behavior have abounded, research on rat emotions has been limited in scope to fear, anxiety, and pain. Converging evidence for the capacity of many species to share others' affective states has emerged, sparking interest in the empathic capacities of rats. Recent research has demonstrated that rats are a highly cooperative species and are motivated by others' distress to prosocial actions, such as opening a door or pulling a chain to release trapped conspecifics. Studies of rat affect, cognition, and neural function provide compelling evidence that rats have some capacity to represent others' needs, to instrumentally act to improve their well-being, and are thus capable of forms of targeted helping. Rats' complex abilities raise the importance of integrating new measures of rat well-being into scientific research.

Individual differences in decision-making shape how mesolimbic dopamine regulates choice confidence and change-of-mind

Nucleus accumbens dopamine signaling is an important neural substrate for decision-making. Dominant theories generally discretize and homogenize decision-making, when it is in fact a continuous process, with evaluation and re-evaluation components that extend beyond simple outcome prediction into consideration of past and future value. Extensive work has examined mesolimbic dopamine in the context of reward prediction error, but major gaps persist in our understanding of how dopamine regulates volitional and self-guided decision-making. Moreover, there is little consideration of individual differences in value processing that may shape how dopamine regulates decision-making. Here, using an economic foraging task in mice, we found that dopamine dynamics in the nucleus accumbens core reflected decision confidence during evaluation of decisions, as well as both past and future value during re-evaluation and change-of-mind. Optogenetic manipulations of mesolimbic dopamine release selectively altered evaluation and re-evaluation of decisions in mice whose dopamine dynamics and behavior reflected future value.

Sex differences in change-of-mind neuroeconomic decision-making is modulated by LINC00473 in medial prefrontal cortex

Changing one's mind is a complex cognitive phenomenon involving a continuous re-appraisal of the trade-off between past costs and future value. Recent work modeling this behavior across species has established associations between aspects of this choice process and their contributions to altered decision-making in psychopathology. Here, we investigated the actions in medial prefrontal cortex (mPFC) neurons of long intergenic non-coding RNA, LINC00473, known to induce stress resilience in a striking sex-dependent manner, but whose role in cognitive function is unknown. We characterized complex decision-making behavior in male and female mice longitudinally in our neuroeconomic foraging paradigm, Restaurant Row, following virus-mediated LINC00473 expression in mPFC neurons. On this task, mice foraged for their primary source of food among varying costs (delays) and subjective value (flavors) while on a limited time-budget during which decisions to accept and wait for rewards were separated into discrete stages of primary commitments and secondary re-evaluations. We discovered important differences in decision-making behavior between female and male mice. LINC00473 expression selectively influenced multiple features of re-evaluative choices, without affecting primary decisions, in female mice only. These behavioral effects included changing how mice (i) cached the value of the passage of time and (ii) weighed their history of economically disadvantageous choices. Both processes were uniquely linked to change-of-mind decisions and underlie the computational bases of distinct aspects of counterfactual thinking. These findings reveal a key bridge between a molecular driver of stress resilience and psychological mechanisms underlying sex-specific decision-making proclivities.

A Double Hit of Social and Economic Stress in Mice Precipitates Changes in Decision-Making Strategies

BACKGROUND

Economic stress can serve as a second hit for people who have already accumulated a history of adverse life experiences. How one recovers from a setback is a core feature of resilience but is seldom captured in animal studies.

METHODS

We challenged mice in a novel 2-hit stress model by first exposing them to chronic social defeat stress and then testing adaptations to increasing reward scarcity on a neuroeconomic task. Mice were tested across months on the Restaurant Row task, during which they foraged daily for their primary source of food while on a limited time budget in a closed-economy system. An abrupt transition into a reward-scarce environment elicits an economic challenge, precipitating a drop in food intake and body weight to which mice must respond to survive.

RESULTS

We found that mice with a history of social stress mounted a robust behavioral response to this economic challenge that was achieved through a complex redistribution of time allocation among competing opportunities. Interestingly, we found that mice with a history of social defeat displayed changes in the development of decision-making policies during the recovery process that are important not only for ensuring food security necessary for survival but also prioritizing subjective value and that these changes emerged only for certain types of choices.

CONCLUSIONS

These findings indicate that an individual's capacity to recover from economic challenges depends on that person's prior history of stress and can affect multiple decision-making aspects of subjective well-being, thus highlighting a motivational balance that may be altered in stress-related disorders such as depression.

Ketamine reverses stress-induced hypersensitivity to sunk costs

How mood interacts with information processing in the brain is thought to mediate the maladaptive behaviors observed in depressed individuals. However, the neural mechanisms underlying impairments in emotion-cognition interactions are poorly understood. This includes influencing the balance between how past-sensitive vs. future-looking one is during decision-making. Recent insights from the field of neuroeconomics offer novel approaches to study changes in such valuation processes in a manner that is biologically tractable and readily translatable across species. We recently discovered that rodents are sensitive to "sunk costs" - a feature of higher cognition previously thought to be unique to humans. The sunk costs bias describes the phenomenon in which an individual overvalues and escalates commitment to continuing an ongoing endeavor, even if suboptimal, as a function of irrecoverable past (sunk) losses - information that, according to classic economic theory, should be ignored. In the present study, mice were exposed to chronic social defeat stress paradigm, a well-established animal model used for the study of depression. Mice were then tested on our longitudinal neuroeconomic foraging task, Restaurant Row. We found mice exposed to this severe stressor displayed an increased sensitivity to sunk costs, without altering overall willingness to wait. Mice were then randomly assigned to receive a single intraperitoneal injection of either saline or ketamine (20 mg/kg). We discovered that stress-induced hypersensitivity to sunk costs was renormalized following a single dose of ketamine. Interestingly, in non-defeated mice, ketamine treatment completely abolished sunk cost sensitivity, causing mice to no longer value irrecoverable losses during re-evaluation decisions who instead based choices solely on the future investment required to obtain a goal. These findings suggest that the antidepressant effects of ketamine may be mediated in part through changes in the processing of past-sensitive information during on-going decision-making, reducing its weight as a potential source of cognitive dissonance that could modulate behavior and instead promoting more future-thinking behavior.

Measuring excitation-inhibition balance through spectral components of local field potentials

The balance between excitation and inhibition is critical to brain functioning, and dysregulation of this balance is a hallmark of numerous psychiatric conditions. Measuring this excitation-inhibition (E:I) balance in vivo has remained difficult, but theoretical models have proposed that characteristics of local field potentials (LFP) may provide an accurate proxy. To establish a conclusive link between LFP and E:I balance, we recorded single units and LFP from the prefrontal cortex (mPFC) of rats during decision making. Dynamic measures of synaptic coupling strength facilitated direct quantification of E:I balance and revealed a strong inverse relationship to broadband spectral power of LFP. These results provide a critical link between LFP and underlying network properties, opening the door for non-invasive recordings to measure E:I balance in clinical settings.

Diabetes alters neuroeconomically dissociable forms of mental accounting

Those with diabetes mellitus are at high-risk of developing psychiatric disorders, yet the link between hyperglycemia and alterations in motivated behavior has not been explored in detail. We characterized value-based decision-making behavior of a streptozocin-induced diabetic mouse model on a naturalistic neuroeconomic foraging paradigm called Restaurant Row. Mice made self-paced choices while on a limited time-budget accepting or rejecting reward offers as a function of cost (delays cued by tone-pitch) and subjective value (flavors), tested daily in a closed-economy system across months. We found streptozocin-treated mice disproportionately undervalued less-preferred flavors and inverted their meal-consumption patterns shifted toward a more costly strategy that overprioritized high-value rewards. We discovered these foraging behaviors were driven by impairments in multiple decision-making systems, including the ability to deliberate when engaged in conflict and cache the value of the passage of time in the form of sunk costs. Surprisingly, diabetes-induced changes in behavior depended not only on the type of choice being made but also the salience of reward-scarcity in the environment. These findings suggest complex relationships between glycemic regulation and dissociable valuation algorithms underlying unique cognitive heuristics and sensitivity to opportunity costs can disrupt fundamentally distinct computational processes and could give rise to psychiatric vulnerabilities.

Psychometric validation and clinical correlates of an experiential foraging task

Measuring the function of decision-making systems is a central goal of computational psychiatry. Individual measures of decisional function could be used to describe neurocognitive profiles that underpin psychopathology and offer insights into deficits that are shared across traditional diagnostic classes. However, there are few demonstrably reliable and mechanistically relevant metrics of decision making that can accurately capture the complex overlapping domains of cognition whilst also quantifying the heterogeneity of function between individuals. The WebSurf task is a reverse-translational human experiential foraging paradigm which indexes naturalistic and clinically relevant decision-making. To determine its potential clinical utility, we examined the psychometric properties and clinical correlates of behavioural parameters extracted from WebSurf in an initial exploratory experiment and a pre-registered validation experiment. Behaviour was stable over repeated administrations of the task, as were individual differences. The ability to measure decision making consistently supports the potential utility of the task in predicting an individual's propensity for response to psychiatric treatment, in evaluating clinical change during treatment, and in defining neurocognitive profiles that relate to psychopathology. Specific aspects of WebSurf behaviour also correlate with anhedonic and externalising symptoms. Importantly, these behavioural parameters may measure dimensions of psychological variance that are not captured by traditional rating scales. WebSurf and related paradigms might therefore be useful platforms for computational approaches to precision psychiatry.

The road not taken: Common and distinct neural correlates of regret and relief

Humans anticipate and evaluate both obtained and counterfactual outcomes - outcomes that could have been had an alternate decision been taken - and experience associated emotions of regret and relief. Although many functional magnetic resonance imaging (fMRI) studies have examined the neural correlates of these emotions, there is substantial heterogeneity in their results. We conducted coordinate-based ALE and network-based ANM meta-analysis of fMRI studies of experienced regret and relief to examine commonalities and differences in their neural correlates. Regionally, we observed that the experience of both regret and relief was associated with greater activation in the right ventral striatum (VS), which is implicated in tracking reward prediction error. At the network level, regret and relief shared the reward-sensitive mesocorticolimbic network with preferential activation of the medial orbitofrontal cortex (mOFC) for regret processing and medial cingulate cortex (MCC) for relief processing. Our research identified shared and separable brain systems subserving regret and relief experience, which may inform the treatment of regret-related mood disorders.

Distinct value computations support rapid sequential decisions

The value of the environment determines animals' motivational states and sets expectations for error-based learning1-3. How are values computed? Reinforcement learning systems can store or cache values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them4-8. Here we show that rats use distinct value computations for sequential decisions within single trials. We used high-throughput training to collect statistically powerful datasets from 291 rats performing a temporal wagering task with hidden reward states. Rats adjusted how quickly they initiated trials and how long they waited for rewards across states, balancing effort and time costs against expected rewards. Statistical modeling revealed that animals computed the value of the environment differently when initiating trials versus when deciding how long to wait for rewards, even though these decisions were only seconds apart. Moreover, value estimates interacted via a dynamic learning rate. Our results reveal how distinct value computations interact on rapid timescales, and demonstrate the power of using high-throughput training to understand rich, cognitive behaviors.

Distinct roles of the orbitofrontal cortex, ventral striatum, and dopamine neurons in counterfactual thinking of decision outcomes

Individuals often assess past decisions by comparing what was gained with what would have been gained had they acted differently. Thoughts of past alternatives that counter what actually happened are called "counterfactuals." Recent theories emphasize the role of the prefrontal cortex in processing counterfactual outcomes in decision-making, although how subcortical regions contribute to this process remains to be elucidated. Here we report a clear distinction among the roles of the orbitofrontal cortex, ventral striatum and midbrain dopamine neurons in processing counterfactual outcomes in monkeys. Our findings suggest that actually gained and counterfactual outcome signals are both processed in the cortico-subcortical network constituted by these regions but in distinct manners and integrated only in the orbitofrontal cortex in a way to compare these outcomes. This study extends the prefrontal theory of counterfactual thinking and provides key insights regarding how the prefrontal cortex cooperates with subcortical regions to make decisions using counterfactual information.

Distinct value computations support rapid sequential decisions

The value of the environment determines animals' motivational states and sets expectations for error-based learning1-3. How are values computed? Reinforcement learning systems can store or "cache" values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them4-8. Here we show that rats use distinct value computations for sequential decisions within single trials. We used high-throughput training to collect statistically powerful datasets from 291 rats performing a temporal wagering task with hidden reward states. Rats adjusted how quickly they initiated trials and how long they waited for rewards across states, balancing effort and time costs against expected rewards. Statistical modeling revealed that animals computed the value of the environment differently when initiating trials versus when deciding how long to wait for rewards, even though these decisions were only seconds apart. Moreover, value estimates interacted via a dynamic learning rate. Our results reveal how distinct value computations interact on rapid timescales, and demonstrate the power of using high-throughput training to understand rich, cognitive behaviors.

Pausing and reorienting behaviors enhance the performance of a spatial working memory task

During spatial working memory tasks, animals need to retain information about a previous trial in order to successfully select their next trajectory. Specifically, the delayed non-match to position task requires rats to follow a cued sample trajectory, then select the opposite route after a delay period. When faced with this choice, rats will occasionally exhibit complex behaviors, such as pausing and sweeping their head back and forth. These behaviors, called vicarious trial and error (VTE), are thought to be a behavioral manifestation of deliberation. However, we identified similarly complex behaviors during sample-phase traversals, despite the fact that these laps do not require a decision. First, we identified that these behaviors occurred more often after incorrect trials than before them, indicating that rats are retaining information between trials. Next, we determined that these pause-and-reorient (PAR) behaviors increased the likelihood of the next choice being selected correctly, suggesting that these behaviors assist the rat in successful task performance. Finally, we identified similarities between PARs and choice-phase VTEs, suggesting that VTEs may not only be reflective of deliberation, but may also contribute to a strategy for successful performance of spatial working memory tasks.

Increased cortical structural covariance correlates with anhedonia in schizophrenia

Anhedonia is a common symptom in schizophrenia and is closely related to poor functional outcomes. Several lines of evidence reveal that the orbitofrontal cortex plays an important role in anhedonia. In the present study, we aimed to investigate abnormalities in structural covariance within the orbitofrontal subregions, and to further study their role in anticipatory and consummatory anhedonia in schizophrenia. T1 images of 35 schizophrenia patients and 45 healthy controls were obtained. The cortical thickness of 68 cerebral regions parcellated by the Desikan-Killiany (DK) atlas was calculated. The structural covariance within the orbitofrontal subregions was calculated in both schizophrenia and healthy control groups. Stepwise linear regression was performed to examine the relationship between structural covariance and anhedonia in schizophrenia patients. Patients with schizophrenia exhibited higher structural covariance between the left and right medial orbitofrontal thickness, the left lateral orbitofrontal thickness and left pars orbitalis thickness compared to healthy controls (p < 0.05, FDR corrected). This results imply that the increased structural covariance in orbitofrontal thickness may be involved in the process of developing anhedonia in schizophrenia. The result indicated that the increased structural covariance between the left and right medial orbitofrontal thickness might be a protective factor for anticipatory pleasure (B' = 0.420, p = 0.012).

AutoRG: An automatized reach-to-grasp platform technology for assessing forelimb motor function, neural circuit activation, and cognition in rodents

BACKGROUND

Rodent reach-to-grasp function assessment is a translationally powerful model for evaluating neurological function impairments and recovery responses. Existing assessment platforms are experimenter-dependent, costly, or low-throughput with limited output measures. Further, a direct histologic comparison of neural activation has never been conducted between any novel, automated platform and the well-established single pellet skilled reach task (SRT).

NEW METHOD

To address these technological and knowledge gaps, we designed an open-source, low-cost Automatized Reach-to-Grasp (AutoRG) pull platform that reduces experimenter interventions and variability. We assessed reach-to-grasp function in rats across seven progressively difficult stages using AutoRG. We mapped AutoRG and SRT-activated motor circuitries in the rat brain using volumetric imaging of the immediate early gene-encoded Arc (activity-regulated cytoskeleton-associated) protein.

RESULTS

Rats demonstrated robust forelimb reaching and pulling behavior after training in AutoRG. Reliable force versus time responses were recorded for individual reach events in real time, which were used to derive several secondary functional measures of performance. Moreover, we provide the first demonstration that for a training period of 30 min, AutoRG and SRT both engage similar neural responses in the caudal forelimb area (CFA), rostral forelimb area (RFA), and sensorimotor area (S1).

CONCLUSION

AutoRG is the first low-cost, open-source pull system designed for the scale-up of volitional forelimb motor function testing and characterization of rodent reaching behavior. The similarities in neuronal activation patterns observed in the rat motor cortex after SRT and AutoRG assessments validate the AutoRG as a rigorously characterized, scalable alternative to the conventional SRT and expensive commercial systems.

On the Role of Theory and Modeling in Neuroscience

In recent years, the field of neuroscience has gone through rapid experimental advances and a significant increase in the use of quantitative and computational methods. This growth has created a need for clearer analyses of the theory and modeling approaches used in the field. This issue is particularly complex in neuroscience because the field studies phenomena that cross a wide range of scales and often require consideration at varying degrees of abstraction, from precise biophysical interactions to the computations they implement. We argue that a pragmatic perspective of science, in which descriptive, mechanistic, and normative models and theories each play a distinct role in defining and bridging levels of abstraction, will facilitate neuroscientific practice. This analysis leads to methodological suggestions, including selecting a level of abstraction that is appropriate for a given problem, identifying transfer functions to connect models and data, and the use of models themselves as a form of experiment.

Neural responses in macaque prefrontal cortex are linked to strategic exploration

Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on scarce resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback about the unchosen option. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen, which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.

Differential processing of decision information in subregions of rodent medial prefrontal cortex

Decision-making involves multiple cognitive processes requiring different aspects of information about the situation at hand. The rodent medial prefrontal cortex (mPFC) has been hypothesized to be central to these abilities. Functional studies have sought to link specific processes to specific anatomical subregions, but past studies of mPFC have yielded controversial results, leaving the precise nature of mPFC function unclear. To settle this debate, we recorded from the full dorso-ventral extent of mPFC in each of 8 rats, as they performed a complex economic decision task. These data revealed four distinct functional domains within mPFC that closely mirrored anatomically identified subregions, including novel evidence to divide prelimbic cortex into dorsal and ventral components. We found that dorsal aspects of mPFC (ACC, dPL) were more involved in processing information about active decisions, while ventral aspects (vPL, IL) were more engaged in motivational factors.

Imagination as a fundamental function of the hippocampus

Imagination is a biological function that is vital to human experience and advanced cognition. Despite this importance, it remains unknown how imagination is realized in the brain. Substantial research focusing on the hippocampus, a brain structure traditionally linked to memory, indicates that firing patterns in spatially tuned neurons can represent previous and upcoming paths in space. This work has generally been interpreted under standard views that the hippocampus implements cognitive abilities primarily related to actual experience, whether in the past (e.g. recollection, consolidation), present (e.g. spatial mapping) or future (e.g. planning). However, relatively recent findings in rodents identify robust patterns of hippocampal firing corresponding to a variety of alternatives to actual experience, in many cases without overt reference to the past, present or future. Given these findings, and others on hippocampal contributions to human imagination, we suggest that a fundamental function of the hippocampus is to generate a wealth of hypothetical experiences and thoughts. Under this view, traditional accounts of hippocampal function in episodic memory and spatial navigation can be understood as particular applications of a more general system for imagination. This view also suggests that the hippocampus contributes to a wider range of cognitive abilities than previously thought. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.

Counterfactual curiosity: motivated thinking about what might have been

Counterfactual information, information about what might have been, forms the content of counterfactual thoughts and emotions like regret and relief. Recent research suggests that human adults and children, as well as rhesus monkeys, demonstrate 'counterfactual curiosity': they are motivated to seek out counterfactual information after making decisions. Based on contemporary theories of curiosity and information seeking and a broad range of empirical literature, we suggest multiple heterogeneous psychological processes that contribute to people's motivation for counterfactual information. This includes processes that are identified in the curiosity literature more generally-the potential use of counterfactual information for adaptive decision making (its long-term instrumental value) and the drive to reduce uncertainty. Additionally, we suggest that counterfactual information may be particularly alluring because of its role in causal reasoning; its relationship with prediction and decision making; and its potential to fulfil emotion regulation and self-serving goals. Some future directions have been suggested, including investigating the role of individual differences in counterfactual curiosity on learning and wellbeing. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.

Pushing the boundaries of behavioral analysis could aid psychiatric drug discovery

Drug discovery for psychiatric conditions is stagnating. Behavioral changes could be used as a primary phenotypic screen for new drug candidates, if enough useful data can be generated from behavioral models. Could machine learning be the answer to extracting the data we need?

Sunk cost sensitivity during change-of-mind decisions is informed by both the spent and remaining costs

Sunk cost sensitivity describes escalating decision commitment with increased spent resources. On neuroeconomic foraging tasks, mice, rats, and humans show similar escalations from sunk costs while quitting an ongoing countdown to reward. In a new analysis taken across computationally parallel foraging tasks across species and laboratories, we find that these behaviors primarily occur on choices that are economically inconsistent with the subject's other choices, and that they reflect not only the time spent, but also the time remaining, suggesting that these are change-of-mind re-evaluation processes. Using a recently proposed change-of-mind drift-diffusion model, we find that the sunk cost sensitivity in this model arises from decision-processes that directly take into account the time spent (costs sunk). Applying these new insights to experimental data, we find that sensitivity to sunk costs during re-evaluation decisions depends on the information provided to the subject about the time spent and the time remaining.

Bridging the Gap: Human Emotions and Animal Emotions

Our experiences of the conscious mental states that we call emotions drive our interest in whether such states also exist in other animals. Because linguistic report can be used as a gold standard (albeit indirect) indicator of subjective emotional feelings in humans but not other species, how can we investigate animal emotions and what exactly do we mean when we use this term? Linguistic reports of human emotion give rise to emotion concepts (discrete emotions; dimensional models), associated objectively measurable behavioral and bodily emotion indicators, and understanding of the emotion contexts that generate specific states. We argue that many animal studies implicitly translate human emotion concepts, indicators and contexts, but that explicit consideration of the underlying pathways of inference, their theoretical basis, assumptions, and pitfalls, and how they relate to conscious emotional feelings, is needed to provide greater clarity and less confusion in the conceptualization and scientific study of animal emotion.

Distinct forms of regret linked to resilience versus susceptibility to stress are regulated by region-specific CREB function in mice

Regret describes recognizing alternative actions could have led to better outcomes. It remains unclear whether regret derives from generalized mistake appraisal or instead comprises dissociable, action-specific processes. Using a neuroeconomic task, we found that mice were sensitive to fundamentally distinct types of regret following exposure to chronic social defeat stress or manipulations of CREB, a transcription factor implicated in stress action. Bias to make compensatory decisions after rejecting high-value offers (regret type I) was unique to stress-susceptible mice. Bias following the converse operation, accepting low-value offers (regret type II), was enhanced in stress-resilient mice and absent in stress-susceptible mice. CREB function in either the prefrontal cortex or nucleus accumbens was required to suppress regret type I but bidirectionally regulated regret type II. We provide insight into how maladaptive stress response traits relate to distinct forms of counterfactual thinking, which could steer therapy for mood disorders, such as depression, toward circuit-specific computations through a careful description of decision narrative.

Hippocampal signals modify orbitofrontal representations to learn new paths

We often remember the consequences of past choices to adapt to changing circumstances. Recalling past events requires the hippocampus (HPC), and using stimuli to anticipate outcome values requires the orbitofrontal cortex (OFC).1-3 Spatial reversal tasks require both structures to navigate newly rewarded paths.4,5 Both HPC place6 and OFC value cells7,8 fire in phase with theta (4-12 Hz) oscillations. Both structures are described as cognitive maps: HPC maps space9 and OFC maps task states.10 These similarities imply that OFC-HPC interactions are crucial for using memory to predict outcomes when circumstances change, but the mechanisms remain largely unknown. To investigate possible interactions, we simultaneously recorded ensembles in OFC and CA1 as rats learned spatial reversals in a plus maze. Striking interactions occurred only while rats learned their first reversal: CA1 population vectors predicted changes in OFC activity but not vice versa, OFC spikes phase locked to hippocampal theta oscillations, mixed pairs of CA1 and OFC neurons fired together within single theta cycles, and CA1 led OFC spikes by ∼30 ms. After the new contingency became familiar, CA1 ensembles stably represented distinct spatial paths, whereas OFC ensembles developed more generalized goal arm representations in different paths to identical rewards. These frontotemporal interactions, engaged selectively when new task features inform decision-making, suggest a mechanism for linking novel episodes with expected outcomes, when HPC signals trigger "cognitive remapping" by OFC.11.

Taking stock of value in the orbitofrontal cortex

People with damage to the orbitofrontal cortex (OFC) have specific problems making decisions, whereas their other cognitive functions are spared. Neurophysiological studies have shown that OFC neurons fire in proportion to the value of anticipated outcomes. Thus, a central role of the OFC is to guide optimal decision-making by signalling values associated with different choices. Until recently, this view of OFC function dominated the field. New data, however, suggest that the OFC may have a much broader role in cognition by representing cognitive maps that can be used to guide behaviour and that value is just one of many variables that are important for behavioural control. In this Review, we critically evaluate these two alternative accounts of OFC function and examine how they might be reconciled.

The rodent medial prefrontal cortex and associated circuits in orchestrating adaptive behavior under variable demands

Emerging evidence implicates rodent medial prefrontal cortex (mPFC) in tasks requiring adaptation of behavior to changing information from external and internal sources. However, the computations within mPFC and subsequent outputs that determine behavior are incompletely understood. We review the involvement of mPFC subregions, and their projections to the striatum and amygdala in two broad types of tasks in rodents: 1) appetitive and aversive Pavlovian and operant conditioning tasks that engage mPFC-striatum and mPFC-amygdala circuits, and 2) foraging-based tasks that require decision making to optimize reward. We find support for region-specific function of the mPFC, with dorsal mPFC and its projections to the dorsomedial striatum supporting action control with higher cognitive demands, and ventral mPFC engagement in translating affective signals into behavior via discrete projections to the ventral striatum and amygdala. However, we also propose that defined mPFC subdivisions operate as a functional continuum rather than segregated functional units, with crosstalk that allows distinct subregion-specific inputs (e.g., internal, affective) to influence adaptive behavior supported by other subregions.

Computational validity: using computation to translate behaviours across species

We propose a new conceptual framework (computational validity) for translation across species and populations based on the computational similarity between the information processing underlying parallel tasks. Translating between species depends not on the superficial similarity of the tasks presented, but rather on the computational similarity of the strategies and mechanisms that underlie those behaviours. Computational validity goes beyond construct validity by directly addressing questions of information processing. Computational validity interacts with circuit validity as computation depends on circuits, but similar computations could be accomplished by different circuits. Because different individuals may use different computations to accomplish a given task, computational validity suggests that behaviour should be understood through the subject's point of view; thus, behaviour should be characterized on an individual level rather than a task level. Tasks can constrain the computational algorithms available to a subject and the observed subtleties of that behaviour can provide information about the computations used by each individual. Computational validity has especially high relevance for the study of psychiatric disorders, given the new views of psychiatry as identifying and mediating information processing dysfunctions that may show high inter-individual variability, as well as for animal models investigating aspects of human psychiatric disorders. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.

Apparent sunk cost effect in rational agents

Rational decision makers aim to maximize their gains, but humans and other animals often fail to do so, exhibiting biases and distortions in their choice behavior. In a recent study of economic decisions, humans, mice, and rats were reported to succumb to the sunk cost fallacy, making decisions based on irrecoverable past investments to the detriment of expected future returns. We challenge this interpretation because it is subject to a statistical fallacy, a form of attrition bias, and the observed behavior can be explained without invoking a sunk cost-dependent mechanism. Using a computational model, we illustrate how a rational decision maker with a reward-maximizing decision strategy reproduces the reported behavioral pattern and propose an improved task design to dissociate sunk costs from fluctuations in decision valuation. Similar statistical confounds may be common in analyses of cognitive behaviors, highlighting the need to use causal statistical inference and generative models for interpretation.

Subpopulations of neurons in lOFC encode previous and current rewards at time of choice

Studies of neural dynamics in lateral orbitofrontal cortex (lOFC) have shown that subsets of neurons that encode distinct aspects of behavior, such as value, may project to common downstream targets. However, it is unclear whether reward history, which may subserve lOFC's well-documented role in learning, is represented by functional subpopulations in lOFC. Previously, we analyzed neural recordings from rats performing a value-based decision-making task, and we documented trial-by-trial learning that required lOFC (Constantinople et al., 2019). Here, we characterize functional subpopulations of lOFC neurons during behavior, including their encoding of task variables. We found five distinct clusters of lOFC neurons, either based on clustering of their trial-averaged peristimulus time histograms (PSTHs), or a feature space defined by their average conditional firing rates aligned to different task variables. We observed weak encoding of reward attributes, but stronger encoding of reward history, the animal's left or right choice, and reward receipt across all clusters. Only one cluster, however, encoded the animal's reward history at the time shortly preceding the choice, suggesting a possible role in integrating previous and current trial outcomes at the time of choice. This cluster also exhibits qualitatively similar responses to identified corticostriatal projection neurons in a recent study (Hirokawa et al., 2019), and suggests a possible role for subpopulations of lOFC neurons in mediating trial-by-trial learning.

Five Breakthroughs: A First Approximation of Brain Evolution From Early Bilaterians to Humans

Retracing the evolutionary steps by which human brains evolved can offer insights into the underlying mechanisms of human brain function as well as the phylogenetic origin of various features of human behavior. To this end, this article presents a model for interpreting the physical and behavioral modifications throughout major milestones in human brain evolution. This model introduces the concept of a "breakthrough" as a useful tool for interpreting suites of brain modifications and the various adaptive behaviors these modifications enabled. This offers a unique view into the ordered steps by which human brains evolved and suggests several unique hypotheses on the mechanisms of human brain function.

Memory and decision making interact to shape the value of unchosen options

The goal of deliberation is to separate between options so that we can commit to one and leave the other behind. However, deliberation can, paradoxically, also form an association in memory between the chosen and unchosen options. Here, we consider this possibility and examine its consequences for how outcomes affect not only the value of the options we chose, but also, by association, the value of options we did not choose. In five experiments (total n = 612), including a preregistered experiment (n = 235), we found that the value assigned to unchosen options is inversely related to their chosen counterparts. Moreover, this inverse relationship was associated with participants' memory of the pairs they chose between. Our findings suggest that deciding between options does not end the competition between them. Deliberation binds choice options together in memory such that the learned value of one can affect the inferred value of the other.

What Behavioral Abilities Emerged at Key Milestones in Human Brain Evolution? 13 Hypotheses on the 600-Million-Year Phylogenetic History of Human Intelligence

This paper presents 13 hypotheses regarding the specific behavioral abilities that emerged at key milestones during the 600-million-year phylogenetic history from early bilaterians to extant humans. The behavioral, intellectual, and cognitive faculties of humans are complex and varied: we have abilities as diverse as map-based navigation, theory of mind, counterfactual learning, episodic memory, and language. But these faculties, which emerge from the complex human brain, are likely to have evolved from simpler prototypes in the simpler brains of our ancestors. Understanding the order in which behavioral abilities evolved can shed light on how and why our brains evolved. To propose these hypotheses, I review the available data from comparative psychology and evolutionary neuroscience.

Over-representation of fundamental decision variables in the prefrontal cortex underlies decision bias

The brain is organized into anatomically distinct structures consisting of a variety of projection neurons. While such evolutionarily conserved neural circuit organization underlies the innate ability of animals to swiftly adapt to environments, they can cause biased cognition and behavior. Although recent studies have begun to address the causal importance of projection-neuron types as distinct computational units, it remains unclear how projection types are functionally organized in encoding variables during cognitive tasks. This review focuses on the neural computation of decision making in the prefrontal cortex and discusses what decision variables are encoded by single neurons, neuronal populations, and projection type, alongside how specific projection types constrain decision making. We focus particularly on "over-representations" of distinct decision variables in the prefrontal cortex that reflect the biological and subjective significance of the variables for the decision makers. We suggest that task-specific over-representation in the prefrontal cortex involves the refinement of the given decision making, while generalized over-representation of fundamental decision variables is associated with suboptimal decision biases, including pathological ones such as those in patients with psychiatric disorders. Such over-representation of the fundamental decision variables in the prefrontal cortex appear to be tightly constrained by afferent and efferent connections that can be optogenetically intervened on. These ideas may provide critical insights into potential therapeutic targets for psychiatric disorders, including addiction and depression.

Disrupting the medial prefrontal cortex with designer receptors exclusively activated by designer drug alters hippocampal sharp-wave ripples and their associated cognitive processes

The hippocampus and medial prefrontal cortex (mPFC) interact during a myriad of cognitive processes including decision-making and long-term memory consolidation. Exactly how the mPFC and hippocampus interact during goal-directed decision-making remains to be fully elucidated. During periods of rest, bursts of high-frequency oscillations, termed sharp-wave ripple (SWR), appear in the local field potential. Impairing SWRs on the maze or during post-learning rest can interfere with memory-guided decision-making and memory consolidation. We hypothesize that the hippocampus and mPFC bidirectionally interact during SWRs to support memory consolidation and decision-making. Rats were trained on the neuroeconomic spatial decision-making task, Restaurant Row, to make serial stay-skip decisions where the amount of effort (delay to reward) varied upon entry to each restaurant. Hippocampal cells and SWRs were recorded in rats with the mPFC transduced with inhibitory DREADDs. We found that disrupting the mPFC impaired consolidating SWRs in the hippocampus. Hippocampal SWR rates depended on the internalized value of the reward (derived from individual flavor preferences), a parameter important in decision-making, and disrupting the mPFC changed this relationship. Additionally, we found a dissociation between SWRs that occurred while rats were on the maze dependent upon whether those SWRs occurred while the rat was anticipating food reward or during post-reward consumption.

Forest Manners Exchange: Forest as a Place to Remedy Risky Behaviour of Adolescents: Mixed Methods Approach

This paper evaluates the impact of the forest environment on aggressive manifestations in adolescents. A remedial educative programme was performed with 68 teenagers from institutions with substitute social care with diagnoses F 30.0 (affective disorders) and F 91.0 (family-related behavioural disorders), aged 12–16 years. Adolescents observed patterns of prosocial behaviour in forest animals (wolves, wild boars, deer, bees, ants, squirrels and birds), based on the fact that processes and interactions in nature are analogous to proceedings and bonds in human society. The methodology is based on qualitative and quantitative research. Projective tests (Rorschach Test, Hand Test, Thematic Apperception Test) were used as a diagnostic tool for aggressive manifestations before and after forest therapies based on Shinrin-yoku, wilderness therapy, observational learning and forest pedagogy. Probands underwent 16 therapies lasting for two hours each. The experimental intervention has a statistically significant effect on the decreased final values relating to psychopathology, irritability, restlessness, emotional instability, egocentrism, relativity, and negativism. Forest animals demonstrated to these adolescents ways of communication, cooperation, adaptability, and care for others, i.e., characteristics without which no community can work.

Vicarious Trial-and-Error Is Enhanced During Deliberation in Human Virtual Navigation in a Translational Foraging Task

Foraging tasks provide valuable insights into decision-making as animals decide how to allocate limited resources (such as time). In rodents, vicarious trial-and-error (back and forth movements), or VTE, is an important behavioral measure of deliberation which is enhanced early in learning and when animals are presented with difficult decisions. Using new translational versions of a rodent foraging task (the "Movie Row" and "Candy Row"), humans navigated a virtual maze presented on standard computers to obtain rewards (either short videos or candy) offered after a variable delay. Decision latencies were longer when participants were presented with difficult offers, overrode their preferences, and when they accepted an offer after rejecting a previous offer. In these situations, humans showed VTE-like behavior, where they were more likely to pause and/or reorient one or more times before making a decision. Behavior on these tasks replicated previous results from the rodent foraging task ("Restaurant Row") and a human version lacking a navigation component ("Web-Surf") and revealed some species differences. Compared to survey measures of delay-discounting, willingness to wait for rewards in the foraging task was not related to willingness to wait for hypothetical rewards. And, smoking status (use of cigarettes or e-cigarettes) was associated with stronger discounting of hypothetical future rewards, but was not well-related to performance on the foraging tasks. In contrast, individuals with overweight or obese BMI (≥25) did not show stronger delay-discounting, but individuals with BMI ≥ 25, and especially females, showed reduced sensitivity to sunk-costs (where their decisions were less sensitive to irrecoverable investments of effort) and less deliberation when presented with difficult offers. These data indicate that VTE is a behavioral index of deliberation in humans, and further support the Movie and Candy Row as translational tools to study decision-making in humans with the potential to provide novel insights about decision-making that are relevant to public health.

The Versatile Wayfinder: Prefrontal Contributions to Spatial Navigation

The prefrontal cortex (PFC) supports decision-making, goal tracking, and planning. Spatial navigation is a behavior that taxes these cognitive processes, yet the role of the PFC in models of navigation has been largely overlooked. In humans, activity in dorsolateral PFC (dlPFC) and ventrolateral PFC (vlPFC) during detours, reveal a role in inhibition and replanning. Dorsal anterior cingulate cortex (dACC) is implicated in planning and spontaneous internally-generated changes of route. Orbitofrontal cortex (OFC) integrates representations of the environment with the value of actions, providing a 'map' of possible decisions. In rodents, medial frontal areas interact with hippocampus during spatial decisions and switching between navigation strategies. In reviewing these advances, we provide a framework for how different prefrontal regions may contribute to different stages of navigation.

Sensitivity to Sunk Costs Depends on Attention to the Delay

In the WebSurf task, humans forage for videos paying costs in terms of wait times on a time-limited task. A variant of the task in which demands during the wait time were manipulated revealed the role of attention in susceptibility to sunk costs. Consistent with parallel tasks in rodents, previous studies have found that humans (undergraduates measured in lab) preferred shorter delays, but waited longer for more preferred videos, suggesting that they were treating the delays economically. In an Amazon Mechanical Turk (mTurk) sample, we replicated these predicted economic behaviors for a majority of participants. In the lab, participants showed susceptibility to sunk costs in this task, basing their decisions in part on time they have already waited, which we also observed in the subset of the mTurk sample that behaved economically. In another version of the task, we added an attention check to the wait phase of the delay. While that attention check further increased the proportion of subjects with predicted economic behaviors, it also removed the susceptibility to sunk costs. These findings have important implications for understanding how cognitive processes, such as the deployment of attention, are key to driving re-evaluation and susceptibility to sunk costs.

Parametric shift from rational to irrational decisions in mice

In the classical view of economic choices, subjects make rational decisions evaluating the costs and benefits of options in order to maximize their overall income. Nonetheless, subjects often fail to reach optimal outcomes. The overt value of an option drives the direction of decisions, but covert factors such as emotion and sensitivity to sunk cost are thought to drive the observed deviations from optimality. Many questions remain to be answered as to (1) which contexts contribute the most to deviation from an optimal solution; and (2) the extent of these effects. In order to tackle these questions, we devised a decision-making task for mice, in which cost and benefit parameters could be independently and flexibly adjusted and for which a tractable optimal solution was known. Comparing mouse behavior with this optimal solution across parameter settings revealed that the factor most strongly contributing to suboptimal performance was the cost parameter. The quantification of sensitivity to sunk cost, a covert factor implicated in our task design, revealed it as another contributor to reduced optimality. In one condition where the large reward option was particularly unattractive and the small reward cost was low, the sensitivity to sunk cost and the cost-led suboptimality almost vanished. In this regime and this regime only, mice could be viewed as close to rational (here, 'rational' refers to a state in which an animal makes decisions basing on objective valuation, not covert factors). Taken together, our results suggest that "rationality" is a task-specific construct even in mice.

"Should've known better": Counterfactual processing in disordered gambling

Counterfactual thinking is a component of human decision-making that entails "if only" thinking about unselected choices and outcomes. It is associated with strong emotional responses of regret (when the obtained outcome is inferior to the counterfactual) and relief (vice versa). Counterfactual thinking may play a role in various cognitive phenomena in disordered gambling, such as the effects of near-misses. This study compared individuals with gambling disorder (n = 46) and healthy controls (n = 25) on a behavioural economic choice task that entailed choosing between two gambles, designed to measure counterfactual thinking. Participants provided affect ratings following both the obtained and the non-obtained outcomes. Choices were analyzed using a computational model that derived parameters reflecting sensitivity to expected value, risk variance, and anticipated regret. In the computational choice model, the group with gambling disorder showed increased sensitivity to anticipated regret, reduced sensitivity to expected value, and increased preference for high risk-variance gambles. On the affect ratings, the group with gambling disorder displayed blunted emotional sensitivity to obtained and counterfactual outcomes. Effect sizes of the group differences were modest. Participants with gambling disorder show wide-ranging alterations in decision-making processes and emotional reactivity to choice outcomes. Altered sensitivity to anticipatory regret in gambling disorder may contribute to the development of gambling-related cognitive distortions, and the influences of gambling marketing.

Neural signatures underlying deliberation in human foraging decisions

Humans have a remarkable capacity to mentally project themselves far ahead in time. This ability, which entails the mental simulation of events, is thought to be fundamental to deliberative decision making, as it allows us to search through and evaluate possible choices. Many decisions that humans make are foraging decisions, in which one must decide whether an available offer is worth taking, when compared to unknown future possibilities (i.e., the background). Using a translational decision-making paradigm designed to reveal decision preferences in rats, we found that humans engaged in deliberation when making foraging decisions. A key feature of this task is that preferences (and thus, value) are revealed as a function of serial choices. Like rats, humans also took longer to respond when faced with difficult decisions near their preference boundary, which was associated with prefrontal and hippocampal activation, exemplifying cross-species parallels in deliberation. Furthermore, we found that voxels within the visual cortices encoded neural representations of the available possibilities specifically following regret-inducing experiences, in which the subject had previously rejected a good offer only to encounter a low-valued offer on the subsequent trial.

Spatial encoding in dorsomedial prefrontal cortex and hippocampus is related during deliberation

Deliberation is thought to involve the internal simulation of the outcomes of candidate actions, the valuation of those outcomes, and the selection of the actions with the highest expected value. While it is known that deliberation involves prefrontal cortical areas, specifically the dorsomedial prefrontal cortex (dmPFC), as well as the hippocampus (HPC) and other brain regions, how these areas process prospective information and select actions is not well understood. We recorded simultaneously from ensembles in dmPFC and CA1 of dorsal HPC in rats during performance of a spatial contingency switching task, and examined the relationships between spatial and reward encoding in these two areas during deliberation at the choice point. We found that CA1 and dmPFC represented either goal locations or the current position simultaneously, but that when goal locations were encoded, HPC and dmPFC did not always represent the same goal location. Ensemble activity in dmPFC predicted when HPC would represent goal locations, but on a broad timescale on the order of seconds. Also, reward encoding in dmPFC increased during hippocampal theta cycles where CA1 ensembles represented the goal location. These results suggest that dmPFC and HPC share prospective information during deliberation, that dmPFC may influence whether HPC represents prospective information, and that information recalled about goal locations by HPC may be integrated into dmPFC reward representations on fast timescales.

Behavior- and Modality-General Representation of Confidence in Orbitofrontal Cortex

Every decision we make is accompanied by a sense of confidence about its likely outcome. This sense informs subsequent behavior, such as investing more-whether time, effort, or money-when reward is more certain. A neural representation of confidence should originate from a statistical computation and predict confidence-guided behavior. An additional requirement for confidence representations to support metacognition is abstraction: they should emerge irrespective of the source of information and inform multiple confidence-guided behaviors. It is unknown whether neural confidence signals meet these criteria. Here, we show that single orbitofrontal cortex neurons in rats encode statistical decision confidence irrespective of the sensory modality, olfactory or auditory, used to make a choice. The activity of these neurons also predicts two confidence-guided behaviors: trial-by-trial time investment and cross-trial choice strategy updating. Orbitofrontal cortex thus represents decision confidence consistent with a metacognitive process that is useful for mediating confidence-guided economic decisions.

Temporal Junctures in the Mind

Humans can imagine what happened in the past and what will happen in the future, but also what did not happen and what might happen. We reflect on envisioned events from alternative timelines, while knowing that we only ever live on one timeline. Considering alternative timelines rests on representations of temporal junctures, or points in time at which possible versions of reality diverge. These representations become increasingly sophisticated over childhood, first enabling preparation for mutually exclusive future possibilities and later the experience of counterfactual emotions like regret. By contrast, it remains unclear whether non-human animals represent temporal junctures at all. The emergence of these representations may have been a prime mover in human evolution.