Optogenetic Inhibition Reveals Distinct Roles for Basolateral Amygdala Activity at Discrete Time Points during Risky Decision Making

A roadmap for transformative translational research on gambling disorder in the UK

The UK has one of the highest rates of recreational gambling in the world. Some vulnerable individuals progressively lose control over gambling and develop at-risk gambling or gambling disorder (GD), characterised by the compulsive pursuit of gambling. GD destroys lives and incurs massive costs to societies, yet only a few treatments are available. Failure to develop a wider range of interventions is in part due to a lack of funding that has slowed progress in the translational research necessary to understand the individual vulnerability to switch from controlled to compulsive gambling. Current preclinical models of GD do not operationalise the key clinical features of the human condition. The so-called "gambling tasks" for non-human mammals almost exclusively assess probabilistic decision-making, which is not real-world gambling. While they have provided insights into the psychological and neural mechanisms involved in the processing of gains and losses, these tasks have failed to capture those underlying real-world gambling and its compulsive manifestation in humans. Here, we highlight the strengths and weaknesses of current gambling-like behaviour tasks and suggest how their translational validity may be improved. We then propose a theoretical framework, the incentive habit theory of GD, which may prove useful for the operationalisation of the biobehavioural mechanisms of GD in preclinical models. We conclude with a list of recommendations for the development of next-generation preclinical models of GD and discuss how modern techniques in animal behavioural experimentation can be deployed in the context of GD preclinical research to bolster the translational pipeline.

Inhibition of the basolateral amygdala to prelimbic cortex pathway enhances risk-taking during risky decision-making shock task in rats

Many animal studies have explored decision-making under risk and punishment, particularly regarding potential rewards, but less focus has been placed on contexts involving net losses. Understanding decision-making under net loss conditions can shed light on the neural mechanisms involved. The basolateral amygdala to prelimbic cortex (BLA→PL) pathway is crucial for risky decision-making. In this study, we investigated how rats make decisions under no-reward but shock conditions, specifically examining the role of the BLA→PL pathway. In the risky decision-making shock task (RDST), rats chose between a "small/certain" lever, which consistently delivered one pellet, and a "large/risky" lever, offering variable rewards with a 50 % probability of reward and a 50 % probability of 1-s foot-shock. The results showed that the shock condition decreased the preference for the large/risky lever, despite increasing rewards. Importantly, inhibiting the BLA→PL pathway significantly increased the selection of the "large/risky" lever compared to the control. Although rats in the clozapine N-oxide (CNO) group did not exhibit significant differences in response latency between levers, they exhibited heightened sensitivity to rewards and losses, suggesting that the BLA→PL pathway affects the encoding of the relationship between aversive stimuli and reward-seeking. Overall, these results provide valuable insights into the neural mechanisms of risk-taking, particularly regarding how inhibition in the BLA→PL pathway can influence reward processing and decision-making under no-reward but shock conditions, with implications for understanding risk-related psychiatric disorders.

A distinct hypothalamus-habenula circuit governs risk preference

Appropriate risk evaluation is essential for survival in complex, uncertain environments. Confronted with choosing between certain (safe) and uncertain (risky) options, animals show strong preference for either option consistently across extended time periods. How such risk preference is encoded in the brain remains elusive. A candidate region is the lateral habenula (LHb), which is prominently involved in value-guided behavior. Here, using a balanced two-alternative choice task and longitudinal two-photon calcium imaging in mice, we identify risk-preference-selective activity in LHb neurons reflecting individual risk preference before action selection. By using whole-brain anatomical tracing, multi-fiber photometry and projection-specific and cell-type-specific optogenetics, we find glutamatergic LHb projections from the medial (MH) but not lateral (LH) hypothalamus providing behavior-relevant synaptic input before action selection. Optogenetic stimulation of MH→LHb axons evoked excitatory and inhibitory postsynaptic responses, whereas LH→LHb projections were excitatory. We thus reveal functionally distinct hypothalamus-habenula circuits for risk preference in habitual economic decision-making.

Social Risk Coding by Amygdala Activity and Connectivity with the Dorsal Anterior Cingulate Cortex

Risk is a fundamental factor affecting individual and social economic decisions, but its neural correlates are largely unexplored in the social domain. The amygdala, together with the dorsal anterior cingulate cortex (dACC), is thought to play a central role in risk-taking. Here, we investigated in human volunteers (n = 20; 11 females) how risk (defined as the variance of reward probability distributions) in a social situation affects decisions and concomitant neural activity as measured with fMRI. We found separate variance-risk signals for social and nonsocial outcomes in the amygdala. Specifically, amygdala activity increased parametrically with social reward variance of presented choice options and on separate trials with nonsocial reward variance. Behaviorally, 75% of participants were averse to social risk as estimated in a Becker-DeGroot-Marschak auction-like procedure. The stronger this aversion, the more negative the coupling between risk-related amygdala regions and dACC. This negative relation was significant for social risk attitude but not for the attitude toward variance-risk in juice outcomes. Our results indicate that the amygdala and its coupling with dACC process objective and subjectively evaluated social risk. Moreover, while social risk can be captured with a framework originally established by finance theory for nonsocial risk, the amygdala appears to process social risk largely separately from nonsocial risk.

Sex differences in sensitivity to dopamine receptor manipulations of risk-based decision making in rats

Risky decision making involves the ability to weigh risks and rewards associated with different options to make adaptive choices. Previous work has established a necessary role for the basolateral amygdala (BLA) in mediating effective decision making under risk of punishment, but the mechanisms by which the BLA mediates this process are less clear. Because this form of decision making is profoundly sensitive to dopaminergic (DA) manipulations, we hypothesized that DA receptors in the BLA may be involved in risk-taking behavior. To test this hypothesis, male and female Long-Evans rats were trained in a decision-making task in which rats chose between a small, safe food reward and a larger food reward that was associated with a variable risk of footshock punishment. Once behavioral stability emerged, rats received intra-BLA infusions of ligands targeting distinct dopamine receptor subtypes prior to behavioral testing. Intra-BLA infusions of the dopamine D2 receptor (D2R) agonist quinpirole decreased risk taking in females at all doses, and this reduction in risk taking was accompanied by an increase in sensitivity to punishment. In males, decreased risk taking was only observed at the highest dose of quinpirole. In contrast, intra-BLA manipulations of dopamine D1 or D3 receptors (D1R and D3R, respectively) had no effect on risk taking. Considered together, these data suggest that differential D2R sensitivity in the BLA may contribute to the well-established sex differences in risk taking. Neither D1Rs nor D3Rs, however, appear to contribute to risky decision making in either sex.

Activation of the mPFC-NAc Pathway Reduces Motor Impulsivity but Does Not Affect Risk-Related Decision-Making in Innately High-Impulsive Male Rats

Attention-deficit/hyperactivity disorder (ADHD) and substance use disorders (SUD) are characterized by exacerbated motor and risk-related impulsivities, which are associated with decreased cortical activity. In rodents, the medial prefrontal cortex (mPFC) and nucleus accumbens (NAc) have been separately implicated in impulsive behaviors, but studies on the specific role of the mPFC-NAc pathway in these behaviors are limited. Here, we investigated whether heightened impulsive behaviors are associated with reduced mPFC activity in rodents and determined the involvement of the mPFC-NAc pathway in motor and risk-related impulsivities. We used the Roman High- (RHA) and Low-Avoidance (RLA) rat lines, which display divergent phenotypes in impulsivity. To investigate alterations in cortical activity in relation to impulsivity, regional brain glucose metabolism was measured using positron emission tomography and [18F]-fluorodeoxyglucose ([18F]FDG). Using chemogenetics, the activity of the mPFC-NAc pathway was either selectively activated in high-impulsive RHA rats or inhibited in low-impulsive RLA rats, and the effects of these manipulations on motor and risk-related impulsivity were concurrently assessed using the rat gambling task. We showed that basal [18F]FDG uptake was lower in the mPFC and NAc of RHA compared to RLA rats. Activation of the mPFC-NAc pathway in RHA rats reduced motor impulsivity, without affecting risk-related decision-making. Conversely, inhibition of the mPFC-NAc pathway had no effect in RLA rats. Our results suggest that the mPFC-NAc pathway controls motor impulsivity, but has limited involvement in risk-related decision-making in our current model. Our findings suggest that reducing fronto-striatal activity may help attenuate motor impulsivity in patients with impulse control dysregulation.

Ventral Pallidum and Amygdala Cooperate to Restrain Reward Approach under Threat

Foraging decisions involve assessing potential risks and prioritizing food sources, which can be challenging when confronted with changing and conflicting circumstances. A crucial aspect of this decision-making process is the ability to actively overcome defensive reactions to threats and focus on achieving specific goals. The ventral pallidum (VP) and basolateral amygdala (BLA) are two brain regions that play key roles in regulating behavior motivated by either rewards or threats. However, it is unclear whether these regions are necessary in decision-making processes involving competing motivational drives during conflict. Our aim was to investigate the requirements of the VP and BLA for foraging choices in conflicts involving overcoming defensive responses. Here, we used a novel foraging task and pharmacological techniques to inactivate either the VP or BLA or to disconnect these brain regions before conducting a conflict test in male rats. Our findings showed that BLA is necessary for making risky choices during conflicts, whereas VP is necessary for invigorating the drive to obtain food, regardless of the presence of conflict. Importantly, our research revealed that the connection between VP and BLA is critical in controlling risky food-seeking choices during conflict situations. This study provides a new perspective on the collaborative function of VP and BLA in driving behavior, aimed at achieving goals in the face of dangers.

Effects of systemic oxytocin receptor activation and blockade on risky decision making in female and male rats

The neuropeptide oxytocin is traditionally known for its roles in parturition, lactation, and social behavior. Other data, however, show that oxytocin can modulate behaviors outside of these contexts, including drug self-administration and some aspects of cost-benefit decision making. Here we used a pharmacological approach to investigate the contributions of oxytocin signaling to decision making under risk of explicit punishment. Female and male Long-Evans rats were trained on a risky decision-making task in which they chose between a small, "safe" food reward and a large, "risky" food reward that was accompanied by varying probabilities of mild footshock. Once stable choice behavior emerged, rats were tested in the task following acute intraperitoneal injections of oxytocin or the oxytocin receptor antagonist L-368,899. Neither drug affected task performance in males. In females, however, both oxytocin and L-368,899 caused a dose-dependent reduction in preference for large risky reward. Control experiments showed that these effects could not be accounted for by alterations in food motivation or shock sensitivity. Together, these results reveal a sex-dependent effect of oxytocin signaling on risky decision making in rats.

Lateral Orbitofrontal Cortex Encodes Presence of Risk and Subjective Risk Preference During Decision-Making

Adaptive decision-making requires consideration of objective risks and rewards associated with each option, as well as subjective preference for risky/safe alternatives. Inaccurate risk/reward estimations can engender excessive risk-taking, a central trait in many psychiatric disorders. The lateral orbitofrontal cortex (lOFC) has been linked to many disorders associated with excessively risky behavior and is ideally situated to mediate risky decision-making. Here, we used single-unit electrophysiology to measure neuronal activity from lOFC of freely moving rats performing in a punishment-based risky decision-making task. Subjects chose between a small, safe reward and a large reward associated with either 0% or 50% risk of concurrent punishment. lOFC activity repeatedly encoded current risk in the environment throughout the decision-making sequence, signaling risk before, during, and after a choice. In addition, lOFC encoded reward magnitude, although this information was only evident during action selection. A Random Forest classifier successfully used neural data accurately to predict the risk of punishment in any given trial, and the ability to predict choice via lOFC activity differentiated between and risk-preferring and risk-averse rats. Finally, risk preferring subjects demonstrated reduced lOFC encoding of risk and increased encoding of reward magnitude. These findings suggest lOFC may serve as a central decision-making hub in which external, environmental information converges with internal, subjective information to guide decision-making in the face of punishment risk.

Effects of optogenetic silencing the anterior cingulate cortex in a delayed non-match to trajectory task

Working memory is a fundamental cognitive ability, allowing us to keep information in memory for the time needed to perform a given task. A complex neural circuit fulfills these functions, among which is the anterior cingulate cortex (CG). Functionally and anatomically connected to the medial prefrontal, retrosplenial, midcingulate and hippocampus, as well as motor cortices, CG has been implicated in retrieving appropriate information when needed to select and control appropriate behavior. The role of cingulate cortex in working memory-guided behaviors remains unclear due to the lack of studies reversibly interfering with its activity during specific epochs of working memory. We used eNpHR3.0 to silence cingulate neurons while animals perform a standard delayed non-match to trajectory task, and found that, while not causing an absolute impairment in working memory, silencing cingulate neurons during retrieval decreases the mean performance if compared to silencing during encoding. Such retrieval-associated changes are accompanied by longer delays observed when light is delivered to control animals, when compared to eNpHR3.0+ ones, consistent with an adaptive recruitment of additional cognitive resources.

Effects of reproductive experience on cost-benefit decision making in female rats

Many individuals undergo mating and/or other aspects of reproductive experience at some point in their lives, and pregnancy and childbirth in particular are associated with alterations in the prevalence of several psychiatric disorders. Research in rodents shows that maternal experience affects spatial learning and other aspects of hippocampal function. In contrast, there has been little work in animal models concerning how reproductive experience affects cost-benefit decision making, despite the relevance of this aspect of cognition for psychiatric disorders. To begin to address this issue, reproductively experienced (RE) and reproductively naïve (RN) female Long-Evans rats were tested across multiple tasks that assess different forms of cost-benefit decision making. In a risky decision-making task, in which rats chose between a small, safe food reward and a large food reward accompanied by variable probabilities of punishment, RE females chose the large risky reward significantly more frequently than RN females (greater risk taking). In an intertemporal choice task, in which rats chose between a small, immediate food reward and a large food reward delivered after a variable delay period, RE females chose the large reward less frequently than RN females. Together, these results show distinct effects of reproductive experience on different forms of cost-benefit decision making in female rats, and highlight reproductive status as a variable that could influence aspects of cognition relevant for psychiatric disorders.

Effects of fentanyl self-administration on risk-taking behavior in male rats

RATIONALE

Individuals with opioid use disorder (OUD) exhibit impaired decision making and elevated risk-taking behavior. In contrast to the effects of natural and semi-synthetic opioids, however, the impact of synthetic opioids on decision making is still unknown.

OBJECTIVES

The objective of the current study was to determine how chronic exposure to the synthetic opioid fentanyl alters risk-based decision making in adult male rats.

METHODS

Male rats underwent 14 days of intravenous fentanyl or oral sucrose self-administration. After 3 weeks of abstinence, rats were tested in a decision-making task in which they chose between a small, safe food reward and a large food reward accompanied by variable risk of footshock punishment. Following testing in the decision-making task, rats were tested in control assays that assessed willingness to work for food and shock reactivity. Lastly, rats were tested on a probabilistic reversal learning task to evaluate enduring effects of fentanyl on behavioral flexibility.

RESULTS

Relative to rats in the sucrose group, rats in the fentanyl group displayed greater choice of the large, risky reward (risk taking), an effect that was present as long as 7 weeks into abstinence. This increased risk taking was driven by enhanced sensitivity to the large rewards and diminished sensitivity to punishment. The fentanyl-induced elevation in risk taking was not accompanied by alterations in food motivation or shock reactivity or impairments in behavioral flexibility.

CONCLUSIONS

Results from the current study reveal that the synthetic opioid fentanyl leads to long-lasting increases in risk taking in male rats. Future experiments will extend this work to females and identify neural mechanisms that underlie these drug-induced changes in risk taking.

The rat frontal orienting field dynamically encodes value for economic decisions under risk

Frontal and parietal cortex are implicated in economic decision-making, but their causal roles are untested. Here we silenced the frontal orienting field (FOF) and posterior parietal cortex (PPC) while rats chose between a cued lottery and a small stable surebet. PPC inactivations produced minimal short-lived effects. FOF inactivations reliably reduced lottery choices. A mixed-agent model of choice indicated that silencing the FOF caused a change in the curvature of the rats' utility function (U = Vρ). Consistent with this finding, single-neuron and population analyses of neural activity confirmed that the FOF encodes the lottery value on each trial. A dynamical model, which accounts for electrophysiological and silencing results, suggests that the FOF represents the current lottery value to compare against the remembered surebet value. These results demonstrate that the FOF is a critical node in the neural circuit for the dynamic representation of action values for choice under risk.

Circuit and Cell-Specific Contributions to Decision Making Involving Risk of Explicit Punishment in Male and Female Rats

Decision making is a complex cognitive process that recruits a distributed network of brain regions, including the basolateral amygdala (BLA) and nucleus accumbens shell (NAcSh). Recent work suggests that communication between these structures, as well as activity of cells expressing dopamine (DA) D2 receptors (D2R) in the NAcSh, are necessary for some forms of decision making; however, the contributions of this circuit and cell population during decision making under risk of punishment are unknown. The current experiments addressed this question using circuit-specific and cell type-specific optogenetic approaches in rats during a decision making task involving risk of punishment. In experiment 1, Long-Evans rats received intra-BLA injections of halorhodopsin or mCherry (control) and in experiment 2, D2-Cre transgenic rats received intra-NAcSh injections of Cre-dependent halorhodopsin or mCherry. Optic fibers were implanted in the NAcSh in both experiments. Following training in the decision making task, BLA→NAcSh or D2R-expressing neurons were optogenetically inhibited during different phases of the decision process. Inhibition of the BLA→NAcSh during deliberation (the time between trial initiation and choice) increased preference for the large, risky reward (increased risk taking). Similarly, inhibition during delivery of the large, punished reward increased risk taking, but only in males. Inhibition of D2R-expressing neurons in the NAcSh during deliberation increased risk taking. In contrast, inhibition of these neurons during delivery of the small, safe reward decreased risk taking. These findings extend our knowledge of the neural dynamics of risk taking, revealing sex-dependent circuit recruitment and dissociable activity of selective cell populations during decision making.SIGNIFICANCE STATEMENT Until recently, the ability to dissect the neural substrates of decision making involving risk of punishment (risk taking) in a circuit-specific and cell-specific manner has been limited by the tools available for use in rats. Here, we leveraged the temporal precision of optogenetics, together with transgenic rats, to probe contributions of a specific circuit and cell population to different phases of risk-based decision making. Our findings reveal basolateral amygdala (BLA)→nucleus accumbens shell (NAcSh) is involved in evaluation of punished rewards in a sex-dependent manner. Further, NAcSh D2 receptor (D2R)-expressing neurons make unique contributions to risk taking that vary across the decision making process. These findings advance our understanding of the neural principles of decision making and provide insight into how risk taking may become compromised in neuropsychiatric diseases.

Description and comparison of brain distribution of oxytocin receptors in Rhabdomys pumillio and Rhabdomys dilectus

Oxytocin receptor (OXTR) distribution in the brain has been associated with different reproductive and social strategies of species. Rhabdomys pumilio (R. pumilio) and Rhabdomys dilectus (R. dilectus) are two sister rodent species that live in large/medium (but flexible) or small (mostly solitary) social groups respectively. In this study, we describe and compare the distribution of OXTR in these two species. OXTR binding in the brain of R. pumilio (8 females and 5 males) and R. dilectus (8 females and 5 males) adults was determined using autoradiography. Our results revealed significant differences in the nucleus accumbens, diagonal band, medial preoptic area, lateral habenula, superior colliculus, periaqueductal area and anterior paraventricular nucleus (higher in R. dilectus), and the dorsal lateral septum and anterior bed nucleus of the stria terminalis (higher in R. pumilio). OXTR density in other brain regions, such as the amygdala nuclei and hippocampus, did not differ between the two species. Sex differences were found in the medial preoptic area and ventral region of the lateral septum in R. pumilio (OXTR density higher in males) and in the anterior paraventricular thalamic nucleus, ventromedial nucleus of the hypothalamus and basolateral amygdala of R. dilectus (OXTR density higher in females). A sex difference in the density of OXTR was also found in the posterior region of the bed nucleus of the stria terminalis, where it was higher in males than in females of both species. This study shows species-specific brain distribution of OXTR in R. pumilio and R. dilectus that are unique, but with similarities with other polygynous or promiscuous rodent species that live in variable size groups, such as R. norvergicus, C. sociabilis, S. teguina and M. musculus.

Temporal Dynamics Underlying Prelimbic Prefrontal Cortical Regulation of Action Selection and Outcome Evaluation during Risk/Reward Decision-Making

Risk/reward decision-making is a dynamic process that includes periods of deliberation before action selection and evaluation of the action outcomes that bias subsequent choices. Inactivation of the prelimbic (PL) cortex has revealed its integral role in updating decision biases in the face of changes in probabilistic reward contingencies, yet how phasic PL signals during different phases of the decision process influence choice remains unclear. We used temporally specific optogenetic inhibition to selectively disrupt PL activity coinciding with action selection and outcome phases to examine how these signals influence choice. Male rats expressing the inhibitory opsin eArchT within PL excitatory neurons were well trained on a probabilistic discounting task, entailing choice between small/certain versus large/risky rewards, the probability of which varied over a session (50-12.5%). During testing, brief light pulses suppressed PL activity before choice or after different outcomes. Prechoice suppression reduced bias toward more preferred/higher utility options and disrupted how recent outcomes influenced subsequent choice. Inhibition during risky losses induced a similar profile, but here, the impact of reward omissions were either amplified or diminished, relative to the context of the estimated profitability of the risky option. Inhibition during large or small reward receipt reduced risky choice when this option was more profitable, suggesting these signals can both reinforce rewarded risky choices and also act as a relative value comparator signal that augments incentive for larger rewards. These findings reveal multifaceted contributions by the PL in implementing decisions and integrating action-outcome feedback to assign context to the decision space.SIGNIFICANCE STATEMENT The PL prefrontal cortex plays an integral role in guiding risk/reward decisions, but how activity in this region during different phases of the decision process influences choice is unclear. By using temporally specific optogenetic manipulations of this activity, the present study unveiled previously uncharacterized and differential contributions by PL in implementing decision policies and how evaluation of decision outcomes shape subsequent choice. These findings provide novel insight into the dynamic processes engaged by the PL that underlie action selection in situations involving reward uncertainty that may aid in understanding the mechanism underlying normal and aberrant decision-making processes.

A neural substrate of sex-dependent modulation of motivation

While there is emerging evidence of sex differences in decision-making behavior, the neural substrates that underlie such differences remain largely unknown. Here we demonstrate that in mice performing a value-based decision-making task, while choices are similar between the sexes, motivation to engage in the task is modulated by action value more strongly in females than in males. Inhibition of activity in anterior cingulate cortex (ACC) neurons that project to the dorsomedial striatum (DMS) preferentially disrupts this relationship between value and motivation in females, without affecting choice in either sex. In line with these effects, in females compared to males, ACC-DMS neurons have stronger representations of negative outcomes and more neurons are active when the value of the chosen option is low. By contrast, the representation of each choice is similar between the sexes. Thus, we identify a neural substrate that contributes to sex-specific modulation of motivation by value.

Circuit and cell-specific contributions to decision making involving risk of explicit punishment in male and female rats

Decision making is a complex cognitive process that recruits a distributed network of brain regions, including the basolateral amygdala (BLA) and nucleus accumbens shell (NAcSh). Recent work suggests that communication between these structures, as well as activity of cells expressing dopamine D2 receptors (D2R) in the NAcSh, are necessary for some forms of decision making; however, the contributions of this circuit and cell population during decision making under risk of punishment are unknown. The current experiments addressed this question using circuit- and cell type-specific optogenetic approaches in rats during a decision-making task involving risk of punishment. In Experiment 1, Long-Evans rats received intra-BLA injections of halorhodopsin or mCherry (control) and in Experiment 2, D2-Cre transgenic rats received intra-NAcSh injections of Cre-dependent halorhodopsin or mCherry. Optic fibers were implanted in the NAcSh in both experiments. Following training in the decision-making task, BLA→NAcSh or D2R-expressing neurons were optogenetically inhibited during different phases of the decision process. Inhibition of the BLA→NAcSh during deliberation (the time between trial initiation and choice) increased choice of the large, risky reward (increased risk taking). Similarly, inhibition during delivery of the large, punished reward increased risk taking, but only in males. Inhibition of D2R-expressing neurons in the NAcSh during deliberation increased risk taking. In contrast, inhibition of these neurons during delivery of the small, safe reward decreased risk taking. These findings extend our knowledge of the neural dynamics of risk taking, revealing sex-dependent circuit recruitment and dissociable activity of selective cell populations during decision making.

Age-Related Changes in Risky Decision Making and Associated Neural Circuitry in a Rat Model

Altered decision making at advanced ages can have a significant impact on an individual's quality of life and the ability to maintain personal independence. Relative to young adults, older adults make less impulsive and less risky choices; although these changes in decision making could be considered beneficial, they can also lead to choices with potentially negative consequences (e.g., avoidance of medical procedures). Rodent models of decision making have been invaluable for dissecting cognitive and neurobiological mechanisms that contribute to age-related changes in decision making, but they have predominantly used costs related to timing or probability of reward delivery and have not considered other equally important costs, such as the risk of adverse consequences. The current study therefore used a rat model of decision making involving risk of explicit punishment to examine age-related changes in this form of choice behavior in male rats, and to identify potential cognitive and neurobiological mechanisms that contribute to these changes. Relative to young rats, aged rats displayed greater risk aversion, which was not attributable to reduced motivation for food, changes in shock sensitivity, or impaired cognitive flexibility. Functional MRI analyses revealed that, overall, functional connectivity was greater in aged rats compared with young rats, particularly among brain regions implicated in risky decision making such as basolateral amygdala, orbitofrontal cortex, and ventral tegmental area. Collectively, these findings are consistent with greater risk aversion found in older humans, and reveal age-related changes in brain connectivity.

Amygdala-cortical collaboration in reward learning and decision making

Adaptive reward-related decision making requires accurate prospective consideration of the specific outcome of each option and its current desirability. These mental simulations are informed by stored memories of the associative relationships that exist within an environment. In this review, I discuss recent investigations of the function of circuitry between the basolateral amygdala (BLA) and lateral (lOFC) and medial (mOFC) orbitofrontal cortex in the learning and use of associative reward memories. I draw conclusions from data collected using sophisticated behavioral approaches to diagnose the content of appetitive memory in combination with modern circuit dissection tools. I propose that, via their direct bidirectional connections, the BLA and OFC collaborate to help us encode detailed, outcome-specific, state-dependent reward memories and to use those memories to enable the predictions and inferences that support adaptive decision making. Whereas lOFC→BLA projections mediate the encoding of outcome-specific reward memories, mOFC→BLA projections regulate the ability to use these memories to inform reward pursuit decisions. BLA projections to lOFC and mOFC both contribute to using reward memories to guide decision making. The BLA→lOFC pathway mediates the ability to represent the identity of a specific predicted reward and the BLA→mOFC pathway facilitates understanding of the value of predicted events. Thus, I outline a neuronal circuit architecture for reward learning and decision making and provide new testable hypotheses as well as implications for both adaptive and maladaptive decision making.

Lateral Orbitofrontal Cortex and Basolateral Amygdala Regulate Sensitivity to Delayed Punishment during Decision-making

In real-world decision-making scenarios, negative consequences do not always occur immediately after a choice. This delay between action and outcome drives the underestimation, or "delay discounting", of punishment. While the neural substrates underlying sensitivity to immediate punishment have been well-studied, there has been minimal investigation of delayed consequences. Here, we assessed the role of lateral orbitofrontal cortex (LOFC) and basolateral amygdala (BLA), two regions implicated in cost/benefit decision-making, in sensitivity to delayed vs immediate punishment. The delayed punishment decision-making task (DPDT) was used to measure delay discounting of punishment in rodents. During DPDT, rats choose between a small, single pellet reward and a large, three pellet reward accompanied by a mild foot shock. As the task progresses, the shock is preceded by a delay that systematically increases or decreases throughout the session. We observed that rats avoid choices associated with immediate punishment, then shift preference toward these options when punishment is delayed. LOFC inactivation did not influence choice of rewards with immediate punishment, but decreased choice of delayed punishment. We also observed that BLA inactivation reduced choice of delayed punishment for ascending but not descending delays. Inactivation of either brain region produced comparable effects on decision-making in males and females, but there were sex differences observed in omissions and latency to make a choice. In summary, both LOFC and BLA contribute to the delay discounting of punishment and may serve as promising therapeutic targets to improve sensitivity to delayed punishment during decision-making.Significance StatementNegative consequences occurring after a delay are often underestimated, which can lead to maladaptive decision-making. While sensitivity to immediate punishment during reward-seeking has been well-studied, the neural substrates underlying sensitivity to delayed punishment remain unclear. Here, we used the Delayed Punishment Decision-making Task to determine that lateral orbitofrontal cortex and basolateral amygdala both regulate the discounting of delayed punishment, suggesting that these regions may be potential targets to improve decision-making in psychopathology.

Regulation of sex differences in risk-based decision making by gonadal hormones: Insights from rodent models

Men and women differ in their ability to evaluate options that vary in their rewards and the risks that are associated with these outcomes. Most studies have shown that women are more risk averse than men and that gonadal hormones significantly contribute to this sex difference. Gonadal hormones can influence risk-based decision making (i.e., risk taking) by modulating the neurobiological substrates underlying this cognitive process. Indeed, estradiol, progesterone and testosterone modulate activity in the prefrontal cortex, amygdala and nucleus accumbens associated with reward and risk-related information. The use of animal models of decision making has advanced our understanding of the intersection between the behavioral, neural and hormonal mechanisms underlying sex differences in risk taking. This review will outline the current state of this literature, identify the current gaps in knowledge and suggest the neurobiological mechanisms by which hormones regulate risky decision making. Collectively, this knowledge can be used to understand the potential consequences of significant hormonal changes, whether endogenously or exogenously induced, on risk-based decision making as well as the neuroendocrinological basis of neuropsychiatric diseases that are characterized by impaired risk taking, such as substance use disorder and schizophrenia.

Instrumental aversion coding in the basolateral amygdala and its reversion by a benzodiazepine

Punishment involves learning the relationship between actions and their adverse consequences. Both the acquisition and expression of punishment learning depend on the basolateral amygdala (BLA), but how BLA supports punishment remains poorly understood. To address this, we measured calcium (Ca²⁺) transients in BLA principal neurons during punishment. Male rats were trained to press two individually presented levers for food; when one of these levers also yielded aversive footshock, responding on this punished lever decreased relative to the other, unpunished lever. In rats with the Ca²⁺ indicator GCaMP6f targeted to BLA principal neurons, we observed excitatory activity transients to the footshock punisher and inhibitory transients to lever-presses earning a reward. Critically, as rats learned punishment, activity around the punished response transformed from inhibitory to excitatory and similarity analyses showed that these punished lever-press transients resembled BLA transients to the punisher itself. Systemically administered benzodiazepine (midazolam) selectively alleviated punishment. Moreover, the degree to which midazolam alleviated punishment was associated with how much punished response-related BLA transients reverted to their pre-punishment state. Together, these findings show that punishment learning is supported by aversion-coding of instrumental responses in the BLA and that the anti-punishment effects of benzodiazepines are associated with a reversion of this aversion coding.

Choose your path: Divergent basolateral amygdala efferents differentially mediate incentive motivation, flexibility and decision-making

To survive in a complex environment, individuals form associations between environmental stimuli and rewards to organize and optimize reward seeking behaviors. The basolateral amygdala (BLA) uses these learned associations to inform decision-making processes. In this review, we describe functional projections between BLA and its cortical and striatal targets that promote learning and motivational processes central to decision-making. Specifically, we compare and contrast divergent projections from the BLA to the orbitofrontal (OFC) and to the nucleus accumbens (NAc) and examine the roles of these pathways in associative learning, value-guided decision-making, choice behaviors, as well as cue and context-driven drug seeking. Finally, we consider how these projections are involved in disorders of motivation, with a focus on Substance Use Disorder.

Reward/Punishment-Based Decision Making in Rodents

Deficits in decision making are at the heart of many psychiatric diseases, such as substance abuse disorders and attention deficit hyperactivity disorder. Consequently, rodent models of decision making are germane to understanding the neural mechanisms underlying adaptive choice behavior and how such mechanisms can become compromised in pathological conditions. A critical factor that must be integrated with reward value to ensure optimal decision making is the occurrence of consequences, which can differ based on probability (risk of punishment) and temporal contiguity (delayed punishment). This article will focus on two models of decision making that involve explicit punishment, both of which recapitulate different aspects of consequences during human decision making. We will discuss each behavioral protocol, the parameters to consider when designing an experiment, and finally how such animal models can be utilized in studies of psychiatric disease. © 2020 Wiley Periodicals LLC. Basic Protocol 1: Behavioral training Support Protocol: Equipment testing Alternate Protocol: Reward discrimination Basic Protocol 2: Risky decision-making task (RDT) Basic Protocol 3: Delayed punishment decision-making task (DPDT).

A bidirectional corticoamygdala circuit for the encoding and retrieval of detailed reward memories

Adaptive reward-related decision making often requires accurate and detailed representation of potential available rewards. Environmental reward-predictive stimuli can facilitate these representations, allowing one to infer which specific rewards might be available and choose accordingly. This process relies on encoded relationships between the cues and the sensory-specific details of the rewards they predict. Here, we interrogated the function of the basolateral amygdala (BLA) and its interaction with the lateral orbitofrontal cortex (lOFC) in the ability to learn such stimulus-outcome associations and use these memories to guide decision making. Using optical recording and inhibition approaches, Pavlovian cue-reward conditioning, and the outcome-selective Pavlovian-to-instrumental transfer (PIT) test in male rats, we found that the BLA is robustly activated at the time of stimulus-outcome learning and that this activity is necessary for sensory-specific stimulus-outcome memories to be encoded, so they can subsequently influence reward choices. Direct input from the lOFC was found to support the BLA in this function. Based on prior work, activity in BLA projections back to the lOFC was known to support the use of stimulus-outcome memories to influence decision making. By multiplexing optogenetic and chemogenetic inhibition we performed a serial circuit disconnection and found that the lOFC→BLA and BLA→lOFC pathways form a functional circuit regulating the encoding (lOFC→BLA) and subsequent use (BLA→lOFC) of the stimulus-dependent, sensory-specific reward memories that are critical for adaptive, appetitive decision making.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.

Ventral Pallidum GABA Neurons Mediate Motivation Underlying Risky Choice

Pursuing rewards while avoiding danger is an essential function of any nervous system. Here, we examine a new mechanism helping rats negotiate the balance between risk and reward when making high-stakes decisions. Specifically, we focus on GABA neurons within an emerging mesolimbic circuit nexus: the ventral pallidum (VP). These neurons play a distinct role from other VP neurons in simple motivated behaviors in mice, but their role in more complex motivated behaviors is unknown. Here, we interrogate the behavioral functions of VPGABA neurons in male and female transgenic GAD1:Cre rats (and WT littermates), using a reversible chemogenetic inhibition approach. Using a behavioral assay of risky decision-making, and of the food-seeking and shock-avoidance components of this task, we show that engaging inhibitory Gi/o signaling specifically in VPGABA neurons suppresses motivation to pursue highly salient palatable foods, and possibly also motivation to avoid being shocked. In contrast, inhibiting these neurons did not affect seeking of low-value food, free consumption of palatable food, or unconditioned affective responses to shock. Accordingly, when rats considered whether to pursue food despite potential for shock in a risky decision-making task, inhibiting VPGABA neurons caused them to more readily select a small but safe reward over a large but dangerous one, an effect not seen in the absence of shock threat. Together, results indicate that VPGABA neurons are critical for high-stakes adaptive responding that is necessary for survival, but which may also malfunction in psychiatric disorders.SIGNIFICANCE STATEMENT In a dynamic world, it is essential to implement appropriate behaviors under circumstances involving rewards, threats, or both. Here, we demonstrate a crucial role for VPGABA neurons in high-stakes motivated behavior of several types. We show that this VPGABA role in motivation impacts decision-making, as inhibiting these neurons yields a conservative, risk-averse strategy not seen when the task is performed without threat of shock. These new roles for VPGABA neurons in behavior may inform future strategies for treating addiction, and other disorders of maladaptive decision-making.

The ventral hippocampus is necessary for cue-elicited, but not outcome driven approach-avoidance conflict decisions: a novel operant choice decision-making task

Approach-avoidance conflict is induced when an organism encounters a stimulus that carries both positive and negative attributes. Accumulating evidence implicates the ventral hippocampus (VH) in the detection and resolution of approach-avoidance conflict, largely on the basis of maze-based tasks assaying innate and conditioned responses to situations of conflict. However, its role in discrete trial approach-avoidance decision-making has yet to be elucidated. In this study, we designed a novel cued operant conflict decision-making task in which rats were required to choose and respond for a low reward option or high reward option paired with varying shock intensities on a differential reinforcement of low rates of responding schedule. Post training, the VH was chemogenetically inhibited while animals performed the task with the usual outcomes delivered, and with the presentation of cues associated with the reward vs. conflict options only (extinction condition). We found that VH inhibition led to an avoidance of the conflict option and longer latency to choose this option when decision-making was being made on the basis of cues alone with no outcomes. Consistent with these findings, VH-inhibited animals spent more time in the central component of the elevated plus maze (EPM), indicating a potential deficit in decision-making under innate forms of approach-avoidance conflict. Taken together, these findings implicate the VH in cue-driven approach-avoidance decisions in the face of motivational conflict.

Regulation of risky decision making by gonadal hormones in males and females

Psychiatric diseases characterized by dysregulated risky decision making are differentially represented in males and females. The factors that govern such sex differences, however, remain poorly understood. Using a task in which rats make discrete trial choices between a small, "safe" food reward and a large food reward accompanied by varying probabilities of footshock punishment, we recently showed that females are more risk averse than males. The objective of the current experiments was to test the extent to which these sex differences in risky decision making are mediated by gonadal hormones. Male and female rats were trained in the risky decision-making task, followed by ovariectomy (OVX), orchiectomy (ORX), or sham surgery. Rats were then retested in the task, under both baseline conditions and following administration of estradiol and/or testosterone. OVX increased choice of the large, risky reward (increased risky choice), an effect that was attenuated by estradiol administration. In contrast, ORX decreased risky choice, but testosterone administration was without effect in either ORX or sham males. Estradiol, however, decreased risky choice in both groups of males. Importantly, none of the effects of hormonal manipulation on risky choice were due to altered shock sensitivity or food motivation. These data show that gonadal hormones are required for maintaining sex-typical profiles of risk-taking behavior in both males and females, and that estradiol is sufficient to promote risk aversion in both sexes. The findings provide novel information about the mechanisms supporting sex differences in risk taking and may prove useful in understanding sex differences in the prevalence of psychiatric diseases associated with altered risk taking.

Neural Computations of Threat

A host of learning, memory, and decision-making processes form the individual's response to threat and may be disrupted in anxiety and post-trauma psychopathology. Here we review the neural computations of threat, from the first encounter with a dangerous situation, through learning, storing, and updating cues that predict it, to making decisions about the optimal course of action. The overview highlights the interconnected nature of these processes and their reliance on shared neural and computational mechanisms. We propose an integrative approach to the study of threat-related processes, in which specific computations are studied across the various stages of threat experience rather than in isolation. This approach can generate new insights about the evolution, diagnosis, and treatment of threat-related psychopathology.

Touchscreen-based assessment of risky-choice in mice

Addictions are characterized by choices made to satisfy the addiction despite the risk it could produce an adverse consequence. Here, we developed a murine version of a 'risky decision-making' task (RDT), in which mice could respond on a touchscreen panel to obtain either a large milkshake reward associated with varying probability of footshock, or a smaller amount of the same reward that was never punished. Results showed that (the following font is incorrectly smaller/subscripted) mice shifted choice from the large to small reward stimulus as shock probability increased. Immunohistochemical analysis revealed more Fos-positive cells in prelimbic cortex (PL) and basal amygdala (BA) after RDT testing, and a strong anti-correlation between infralimbic cortex (IL) activity and choice of the large reward stimulus under likely (75-100 % probability) punishment. These findings establish an assay for risky choice in mice and provide preliminary insight into the underlying neural substrates.

Basolateral amygdala - nucleus accumbens circuitry regulates optimal cue-guided risk/reward decision making

Maladaptive decision making is a characteristic feature of substance use disorder and pathological gambling. Studies in humans and animals have implicated neural circuits that include the basolateral amygdala (BLA) and nucleus accumbens (NAc) in facilitating risk/reward decision making. However, the preclinical literature has focussed primarily on situations where animals use internally-generated information to adapt to changes in reward likelihood, whereas many real-life situations require the use of external stimuli to facilitate context-appropriate behavior. We recently developed the "Blackjack" task, to measure cued risk/reward decision making requiring rats to chose between Small/Certain and Large/Risky rewards, with auditory cues at the start of each trial explicitly informing that the probability of obtaining a large reward was either good (50%) or poor (12.5%). Here we investigated the contribution of the BLA and its interaction with the NAc in guiding these types of decisions. In well-trained male rats, bilateral inactivation of the BLA induced suboptimal decision making, primarily by reducing risky choice on good-odds trials. In comparison, pharmacological disconnection of the BLA and NAc-shell also induced suboptimal decision making, diverting choice from more preferred option by reducing or increasing risky choice on good vs. poor odds trials respectively. Together, these results suggest that the BLA-NAc circuitry plays a crucial role in integrating information provided by discriminative stimuli. Furthermore, this circuitry may aid in guiding action selection of advantageous options in situations to maximize rewards. Finally, they suggest that perturbations in optimal decision making observed in substance abuse and gambling disorders may be driven in part by dysfunction within this circuitry.

Deconstructing value-based decision making via temporally selective manipulation of neural activity: Insights from rodent models

The ability to choose among options that differ in their rewards and costs (value-based decision making) has long been a topic of interest for neuroscientists, psychologists, and economists alike. This is likely because this is a cognitive process in which all animals (including humans) engage on a daily basis, be it routine (which road to take to work) or consequential (which graduate school to attend). Studies of value-based decision making (particularly at the preclinical level) often treat it as a uniform process. The results of such studies have been invaluable for our understanding of the brain substrates and neurochemical systems that contribute to decision making involving a range of different rewards and costs. Value-based decision making is not a unitary process, however, but is instead composed of distinct cognitive operations that function in concert to guide choice behavior. Within this conceptual framework, it is therefore important to consider that the known neural substrates supporting decision making may contribute to temporally distinct and dissociable components of the decision process. This review will describe this approach for investigating decision making, drawing from published studies that have used techniques that allow temporal dissection of the decision process, with an emphasis on the literature in animal models. The review will conclude with a discussion of the implications of this work for understanding pathological conditions that are characterized by impaired decision making.

[Physiological centers of decision-making: manipulation of neural activity in insular cortex by AAV]

Decision-making is a key activity process that influences many aspects of daily living and both mental and physical health. In general, healthy participants reveal rational choice, but patients with neuropsychiatric disorders reveal irrational and risky choice in decision-making. Addiction is one of typical diseases revealed risky decision-making, addicts select risky action and options that confer short-term rewards at the cost of long-term disadvantages. Thus, irrational and risky decision-making is recognized as a core problem in patients with neuropsychiatric disorders, and a better understanding of the mechanisms underlying altered decision-making would provide insights into potential therapeutic approaches for these diseases. However, the neural pathway and substrates underlying these deficits are particularly unknown. Recently, we found that insular cortex is one of key regions for risky decision-making in an animal model of methamphetamine addiction, by using the designer receptor exclusively activated by designer drug (DREADD) technology, and that GABAergic dysfunction in insular cortex is involved in evaluating the subjective value of reward and reward prediction error. These brain dysfunctions would be related to risk taking behavior in addiction. In this review, we introduced the possible neural pathway related to risky decision-making and behavioral changes in choice strategy using adeno associated virus (AAV).

Quantity versus quality: Convergent findings in effort-based choice tasks

Organisms must frequently make cost-benefit decisions based on time, risk, and effort in choosing rewards to pursue. Various tasks have been developed to assess effort-based choice in rats, and experimenters have found largely similar results across tasks and brain regions. In this review, we focus primarily on the convergence of different effort-based choice tasks where quality or quantity of reward are manipulated. In the former, the rat is typically presented with the option to work for a preferred reward or select a less preferred, but freely-available reward. In such paradigms, the rewards are of different identities but are confirmed to differ qualitatively in value by a food preference task when both are freely-available. In the latter task type, rats are required to select between higher magnitude versus lower magnitudes of the same reward, but each with a similar effort requirement. We discuss the strengths/limitations of these paradigms, and describe brain regions that have been probed that result in converging or equivocal findings. Results are also reviewed with reference to a need for future work, and the broader impacts and implications of studies probing the mechanisms of effort.

Optogenetic dissection of basolateral amygdala contributions to intertemporal choice in young and aged rats

Across species, aging is associated with an increased ability to choose delayed over immediate gratification. These experiments used young and aged rats to test the role of the basolateral amygdala (BLA) in intertemporal decision making. An optogenetic approach was used to inactivate the BLA in young and aged rats at discrete time points during choices between levers that yielded a small, immediate vs. a large, delayed food reward. BLA inactivation just prior to decisions attenuated impulsive choice in both young and aged rats. In contrast, inactivation during receipt of the small, immediate reward increased impulsive choice in young rats but had no effect in aged rats. BLA inactivation during the delay or intertrial interval had no effect at either age. These data demonstrate that the BLA plays multiple, temporally distinct roles during intertemporal choice, and show that the contribution of BLA to choice behavior changes across the lifespan.

Distinct cortical-amygdala projections drive reward value encoding and retrieval

The value of an anticipated rewarding event is a crucial component of the decision to engage in its pursuit. But little is known of the networks responsible for encoding and retrieving this value. By using biosensors and pharmacological manipulations, we found that basolateral amygdala (BLA) glutamatergic activity tracks and mediates encoding and retrieval of the state-dependent incentive value of a palatable food reward. Projection-specific, bidirectional chemogenetic and optogenetic manipulations revealed that the orbitofrontal cortex (OFC) supports the BLA in these processes. Critically, the function of ventrolateral and medial OFC→BLA projections is doubly dissociable. Whereas lateral OFC→BLA projections are necessary and sufficient for encoding of the positive value of a reward, medial OFC→BLA projections are necessary and sufficient for retrieving this value from memory. These data reveal a new circuit for adaptive reward valuation and pursuit and provide insight into the dysfunction in these processes that characterizes myriad psychiatric diseases.

Recent Updates in Modeling Risky Decision Making in Rodents

Excessive preference for risky over safe options is a hallmark of several psychiatric disorders. Here we describe a behavioral task that models such risky decision making in rats. In this task, rats are given choices between small, safe rewards and large rewards accompanied by risk of footshock punishment. The risk of punishment changes within a test session, allowing quantification of decision making at different levels of risk. Importantly, this task can yield a wide degree of reliable individual variability, allowing the characterization of rats as "risk-taking" or "risk-averse." The task has been demonstrated to be effective for testing the effects of pharmacological agents and neurobiological manipulations, and the individual variability (which mimics the human population) allows assessment of behavioral and neurobiological distinctions among subjects based on their risk-taking profile.

Optogenetic Dissection of Temporal Dynamics of Amygdala-Striatal Interplay during Risk/Reward Decision Making

Decision making often requires weighing costs and benefits of different options that vary in terms of reward magnitude and uncertainty. Previous studies using pharmacological inactivations have shown that the basolateral amygdala (BLA) to nucleus accumbens (NAc) pathway promotes choice towards larger/riskier rewards. Neural activity in BLA and NAc shows distinct, phasic changes in firing prior to choice and following action outcomes, yet, how these temporally-discrete patterns of activity within BLA→NAc circuitry influence choice is unclear. We assessed how optogenetic silencing of BLA terminals in the NAc altered action selection during probabilistic decision making. Rats received intra-BLA infusions of viruses encoding the inhibitory opsin eArchT and were well trained on a probabilistic discounting task, where they chose between smaller/certain rewards and larger rewards delivered in a probabilistic manner, with the odds of obtaining the larger reward changing over a session (50–12.5%). During testing, activity of BLA→NAc inputs were suppressed with 4- to 7-s pulses of light delivered via optic fibers into the NAc during discrete task events: prior to choice or after choice outcomes. Inhibition prior to choice reduced selection of the preferred option, suggesting that during deliberation, BLA→NAc activity biases choice towards preferred rewards. Inhibition during reward omissions increased risky choice during the low-probability block, indicating that activity after non-rewarded actions serves to modify subsequent choice. In contrast, silencing during rewarded outcomes did not reliably affect choice. These data demonstrate how patterns of activity in BLA→NAc circuitry convey different types of information that guide action selection in situations involving reward uncertainty.