Selective involvement by the medial orbitofrontal cortex in biasing risky, but not impulsive, choice

Lesions of the Orbitofrontal but Not Medial Prefrontal Cortex Affect Cognitive Judgment Bias in Rats

Neuroimaging studies in humans have recently shown that the prefrontal cortex (PFC) and orbitofrontal cortex (OFC) mediate bias in the judgment of forthcoming events. In the present study, we sought to determine whether cognitive judgment bias (CJB) is also dependent on these prefrontal regions in non-human animals. For this, we trained a cohort of rats in the ambiguous-cue interpretation (ACI) paradigm, subjected them to excitotoxic lesions in the medial PFC (mPFC) and OFC, and tested the effects of neuronal loss within these regions on CJB. Comparison of the lesions' behavioral effects in the ACI paradigm revealed that neuronal loss within the OFC but not within the mPFC influences the interpretation of ambiguous cues by animals. Our findings demonstrate the specific involvement of the OFC in CJB in rats.

A framework for understanding and advancing intertemporal choice research using rodent models

Intertemporal choices are common and consequential to private and public life. Thus, there is considerable interest in understanding the neural basis of intertemporal decision making. In this minireview, we briefly describe conceptual and psychological perspectives on intertemporal choice and then provide a comprehensive evaluation of the neural structures and signals that comprise the underlying cortico-limbic-striatal circuit. Even though great advances have been made, our understanding of the neurobiology of intertemporal choice is still in its infancy because of the complex and dynamic nature of this form of decision making. We close by briefly discussing recommendations for the future study of intertemporal choice research.

Neural substrates underlying effort, time, and risk-based decision making in motivated behavior

All mobile organisms rely on adaptive motivated behavior to overcome the challenges of living in an environment in which essential resources may be limited. A variety of influences ranging from an organism's environment, experiential history, and physiological state all influence a cost-benefit analysis which allows motivation to energize behavior and direct it toward specific goals. Here we review the substantial amount of research aimed at discovering the interconnected neural circuits which allow organisms to carry-out the cost-benefit computations which allow them to behave in adaptive ways. We specifically focus on how the brain deals with different types of costs, including effort requirements, delays to reward and payoff riskiness. An examination of this broad literature highlights the importance of the extended neural circuits which enable organisms to make decisions about these different types of costs. This involves Cortical Structures, including the Anterior Cingulate Cortex (ACC), the Orbital Frontal Cortex (OFC), the Infralimbic Cortex (IL), and prelimbic Cortex (PL), as well as the Baso-Lateral Amygdala (BLA), the Nucleus Accumbens (NAcc), the Ventral Pallidal (VP), the Sub Thalamic Nucleus (STN) among others. Some regions are involved in multiple aspects of cost-benefit computations while the involvement of other regions is restricted to information relating to specific types of costs.

Multifaceted Contributions by Different Regions of the Orbitofrontal and Medial Prefrontal Cortex to Probabilistic Reversal Learning

Different subregions of the prefrontal cortex (PFC) contribute to the ability to respond flexibly to changes in reward contingencies, with the medial versus orbitofrontal cortex (OFC) subregions contributing differentially to processes such as set-shifting and reversal learning. To date, the manner in which these regions may facilitate reversal learning in situations involving reward uncertainty remains relatively unexplored. We investigated the involvement of five distinct regions of the rat OFC (lateral and medial) and medial PFC (prelimbic, infralimbic, and anterior cingulate) on probabilistic reversal learning wherein "correct" versus "incorrect" responses were rewarded on 80% and 20% of trials, respectively. Contingencies were reversed repeatedly within a session. In well trained rats, inactivation of the medial or lateral OFC induced dissociable impairments in performance (indexed by fewer reversals completed) when outcomes were probabilistic, but not when they were assured. Medial OFC inactivation impaired probabilistic learning during the first discrimination, increased perseverative responding and reduced sensitivity to positive and negative feedback, suggestive of a deficit in incorporating information about previous action outcomes to guide subsequent behavior. Lateral OFC inactivation preferentially impaired performance during reversal phases. In contrast, prelimbic inactivation caused an apparent improvement in performance by increasing the number of reversals completed. This was associated with enhanced sensitivity to recently rewarded actions and reduced sensitivity to negative feedback. Infralimbic inactivation had no effect, whereas the anterior cingulate appeared to play a permissive role in this form of reversal learning. These results clarify the dissociable contributions of different regions of the frontal lobes to probabilistic learning.

SIGNIFICANCE STATEMENT

The ability to adjust behavior in response to changes involving uncertain or probabilistic reward contingencies is an essential survival skill that is impaired in a variety of psychiatric disorders. It is well established that different forms of cognitive flexibility are mediated by anatomically distinct regions of the frontal lobes when reinforcement contingencies are assured, however, less is known about the contribution of these regions to probabilistic reinforcement learning. Here we show that different regions of the orbitofrontal and medial prefrontal cortex make distinct contributions to probabilistic reversal learning. These findings provide novel information about the complex interplay between frontal lobe regions in mediating these processes and accordingly provide insight into possible pathophysiology that underlies impairments in cognitive flexibility observed in mental illnesses.

Impaired discrimination learning in interneuronal NMDAR-GluN2B mutant mice

Previous studies have established a role for N-methyl-D-aspartate receptor (NMDAR) containing the GluN2B subunit in efficient learning behavior on a variety of tasks. Recent findings have suggested that NMDAR on GABAergic interneurons may underlie the modulation of striatal function necessary to balance efficient action with cortical excitatory input. Here we investigated how loss of GluN2B-containing NMDAR on GABAergic interneurons altered corticostriatal-mediated associative learning. Mutant mice (floxed-GluN2B×Ppp1r2-Cre) were generated to produce loss of GluN2B on forebrain interneurons and phenotyped on a touchscreen-based pairwise visual learning paradigm. We found that the mutants showed normal performance during Pavlovian and instrumental pretraining, but were significantly impaired on a discrimination learning task. Detailed analysis of the microstructure of discrimination performance revealed reduced win→stay behavior in the mutants. These results further support the role of NMDAR, and GluN2B in particular, on modulation of striatal function necessary for efficient choice behavior and suggest that NMDAR on interneurons may play a critical role in associative learning.

Individual differences in impulsive action and dopamine transporter function in rat orbitofrontal cortex

Impulsivity, which can be subdivided into impulsive action and impulsive choice, is implicated as a factor underlying drug abuse vulnerability. Although previous research has shown that dopamine (DA) systems in prefrontal cortex are involved in impulsivity and substance abuse, it is not known if inherent variation in DA transporter (DAT) function contributes to impulsivity. The current study determined if individual differences in either impulsive action or impulsive choice are related to DAT function in orbitofrontal (OFC) and/or medial prefrontal cortex (mPFC). Rats were first tested both for impulsive action in a cued go/no-go task and for impulsive choice in a delay-discounting task. Following behavioral evaluation, in vitro [(3)H]DA uptake assays were performed in OFC and mPFC isolated from individual rats. Vmax in OFC, but not mPFC, was correlated with performance in the cued go/no-go task, with decreased OFC DAT function being associated with high impulsive action. In contrast, Vmax in OFC and mPFC was not correlated with performance in the delay-discounting task. The current results demonstrate that impulsive behavior in cued go/no-go performance is associated with decreased DAT function in OFC, suggesting that hyperdopaminergic tone in this prefrontal subregion mediates, at least in part, increased impulsive action.

The neural dynamics of reward value and risk coding in the human orbitofrontal cortex

The orbitofrontal cortex is known to carry information regarding expected reward, risk and experienced outcome. Yet, due to inherent limitations in lesion and neuroimaging methods, the neural dynamics of these computations has remained elusive in humans. Here, taking advantage of the high temporal definition of intracranial recordings, we characterize the neurophysiological signatures of the intact orbitofrontal cortex in processing information relevant for risky decisions. Local field potentials were recorded from the intact orbitofrontal cortex of patients suffering from drug-refractory partial epilepsy with implanted depth electrodes as they performed a probabilistic reward learning task that required them to associate visual cues with distinct reward probabilities. We observed three successive signals: (i) around 400 ms after cue presentation, the amplitudes of the local field potentials increased with reward probability; (ii) a risk signal emerged during the late phase of reward anticipation and during the outcome phase; and (iii) an experienced value signal appeared at the time of reward delivery. Both the medial and lateral orbitofrontal cortex encoded risk and reward probability while the lateral orbitofrontal cortex played a dominant role in coding experienced value. The present study provides the first evidence from intracranial recordings that the human orbitofrontal cortex codes reward risk both during late reward anticipation and during the outcome phase at a time scale of milliseconds. Our findings offer insights into the rapid mechanisms underlying the ability to learn structural relationships from the environment.

Age-related changes in prefrontal norepinephrine transporter density: The basis for improved cognitive flexibility after low doses of atomoxetine in adolescent rats

Adolescence is a period of major behavioral and brain reorganization. As diagnoses and treatment of disorders like attention deficit hyperactivity disorder (ADHD) often occur during adolescence, it is important to understand how the prefrontal cortices change and how these changes may influence the response to drugs during development. The current study uses an adolescent rat model to study the effect of standard ADHD treatments, atomoxetine and methylphenidate on attentional set shifting and reversal learning. While both of these drugs act as norepinephrine reuptake inhibitors, higher doses of atomoxetine and all doses of methylphenidate also block dopamine transporters (DAT). Low doses of atomoxetine, were effective at remediating cognitive rigidity found in adolescents. In contrast, methylphenidate improved performance in rats unable to form an attentional set due to distractibility but was without effect in normal subjects. We also assessed the effects of GBR 12909, a selective DAT inhibitor, but found no effect of any dose on behavior. A second study in adolescent rats investigated changes in norepinephrine transporter (NET) and dopamine beta hydroxylase (DBH) density in five functionally distinct sub-regions of the prefrontal cortex: infralimbic, prelimbic, anterior cingulate, medial and lateral orbitofrontal cortices. These regions are implicated in impulsivity and distractibility. We found that NET, but not DBH, changed across adolescence in a regionally selective manner. The prelimbic cortex, which is critical to cognitive rigidity, and the lateral orbitofrontal cortex, critical to reversal learning and some forms of response inhibition, showed higher levels of NET at early than mid- to late adolescence. This article is part of a Special Issue entitled SI: Noradrenergic System.

A Framework for Understanding the Emerging Role of Corticolimbic-Ventral Striatal Networks in OCD-Associated Repetitive Behaviors

Significant interest in the mechanistic underpinnings of obsessive-compulsive disorder (OCD) has fueled research on the neural origins of compulsive behaviors. Converging clinical and preclinical evidence suggests that abnormal repetitive behaviors are driven by dysfunction in cortico-striatal-thalamic-cortical (CSTC) circuits. These findings suggest that compulsive behaviors arise, in part, from aberrant communication between lateral orbitofrontal cortex (OFC) and dorsal striatum. An important body of work focused on the role of this network in OCD has been instrumental to progress in the field. Disease models focused primarily on these regions, however, fail to capture an important aspect of the disorder: affective dysregulation. High levels of anxiety are extremely prevalent in OCD, as is comorbidity with major depressive disorder. Furthermore, deficits in processing rewards and abnormalities in processing emotional stimuli are suggestive of aberrant encoding of affective information. Accordingly, OCD can be partially characterized as a disease in which behavioral selection is corrupted by exaggerated or dysregulated emotional states. This suggests that the networks producing OCD symptoms likely expand beyond traditional lateral OFC and dorsal striatum circuit models, and highlights the need to cast a wider net in our investigation of the circuits involved in generating and sustaining OCD symptoms. Here, we address the emerging role of medial OFC, amygdala, and ventral tegmental area projections to the ventral striatum (VS) in OCD pathophysiology. The VS receives strong innervation from these affect and reward processing regions, and is therefore poised to integrate information crucial to the generation of compulsive behaviors. Though it complements functions of dorsal striatum and lateral OFC, this corticolimbic-VS network is less commonly explored as a potential source of the pathology underlying OCD. In this review, we discuss this network's potential role as a locus of OCD pathology and effective treatment.

Medial Orbitofrontal Cortex Mediates Outcome Retrieval in Partially Observable Task Situations

Choice between actions often requires the ability to retrieve action consequences in circumstances where they are only partially observable. This capacity has recently been argued to depend on orbitofrontal cortex; however, no direct evidence for this hypothesis has been reported. Here, we examined whether activity in the medial orbitofrontal cortex (mOFC) underlies this critical determinant of decision-making in rats. First, we simulated predictions from this hypothesis for various tests of goal-directed action by removing the assumption that rats could retrieve partially observable outcomes and then tested those predictions experimentally using manipulations of the mOFC. The results closely followed predictions; consistent deficits only emerged when action consequences had to be retrieved. Finally, we put action selection based on observable and unobservable outcomes into conflict and found that whereas intact rats selected actions based on the value of retrieved outcomes, mOFC rats relied solely on the value of observable outcomes.

Age-Dependent D1-D2 Receptor Coactivation in the Lateral Orbitofrontal Cortex Potentiates NMDA Receptors and Facilitates Cognitive Flexibility

The orbitofrontal cortex (OFC) integrates information about the environment to guide decision-making. Glutamatergic synaptic transmission mediated through N-methyl-d-aspartate receptors is required for optimal functioning of the OFC. Additionally, abnormal dopamine signaling in this region has been implicated in impulsive behavior and poor cognitive flexibility. Yet, despite the high prevalence of psychostimulants prescribed for attention deficit/hyperactivity disorder, there is little information on how dopamine modulates synaptic transmission in the juvenile or the adult OFC. Using whole-cell patch-clamp recordings in OFC pyramidal neurons, we demonstrated that while dopamine or selective D2-like receptor (D2R) agonists suppress excitatory synaptic transmission of juvenile or adult lateral OFC neurons; in juvenile lateral OFC neurons, higher concentrations of dopamine can target dopamine receptors that couple to a phospholipase C (PLC) signaling pathway to enhance excitatory synaptic transmission. Interfering with the formation of a putative D1R-D2R interaction blocked the potentiation of excitatory synaptic transmission. Furthermore, targeting the putative D1R-D2R complex with a biased agonist, SKF83959, not only enhanced excitatory synaptic transmission in a PLC-dependent manner, but also improved the performance of juvenile rats on a reversal-learning task. Our results demonstrate that dopamine signaling in the lateral OFC differs between juveniles and adults, through potential crosstalk between dopamine receptor subtypes.

Hippocampal interplay with the nucleus accumbens is critical for decisions about time

Human cognition depends upon the capacity to make decisions in the present that bear upon outcomes in the future. The nucleus accumbens, a recipient of direct projections from both the hippocampus and orbitofrontal cortex, is known to contribute to these aspects of decision-making. Here we demonstrate that interaction of the nucleus accumbens with the hippocampus, but not the orbitofrontal cortex, is critical in shaping decisions that involve time trade-offs. Compared with controls, rats with a disrupted hippocampal-accumbens interaction were strongly biased toward choosing stimuli that led to small and immediate food rewards over large and delayed ones. We show that this pattern of behavior cannot be ascribed to the impaired representation of stimulus value, the incapacity to wait, or a general disruption of decision-making. These results identify a hippocampal-accumbens circuit that may underlie a range of problems in which daily decisions are marked by a shift toward immediate gratification.

Noradrenergic modulation of risk/reward decision making

RATIONALE

Catecholamine transmission modulates numerous cognitive and reward-related processes that can subserve more complex functions such as cost/benefit decision making. Dopamine has been shown to play an integral role in decisions involving reward uncertainty, yet there is a paucity of research investigating the contributions of noradrenaline (NA) transmission to these functions.

OBJECTIVES

The present study was designed to elucidate the contribution of NA to risk/reward decision making in rats, assessed with a probabilistic discounting task.

METHODS

We examined the effects of reducing noradrenergic transmission with the α2 agonist clonidine (10-100 μg/kg), and increasing activity at α2A receptor sites with the agonist guanfacine (0.1-1 mg/kg), the α2 antagonist yohimbine (1-3 mg/kg), and the noradrenaline transporter (NET) inhibitor atomoxetine (0.3-3 mg/kg) on probabilistic discounting. Rats chose between a small/certain reward and a larger/risky reward, wherein the probability of obtaining the larger reward either decreased (100-12.5 %) or increased (12.5-100 %) over a session.

RESULTS

In well-trained rats, clonidine reduced risky choice by decreasing reward sensitivity, whereas guanfacine did not affect choice behavior. Yohimbine impaired adjustments in decision biases as reward probability changed within a session by altering negative feedback sensitivity. In a subset of rats that displayed prominent discounting of probabilistic rewards, the lowest dose of atomoxetine increased preference for the large/risky reward when this option had greater long-term utility.

CONCLUSIONS

These data highlight an important and previously uncharacterized role for noradrenergic transmission in mediating different aspects of risk/reward decision making and mediating reward and negative feedback sensitivity.

A functional difference in information processing between orbitofrontal cortex and ventral striatum during decision-making behaviour

Both orbitofrontal cortex (OFC) and ventral striatum (vStr) have been identified as key structures that represent information about value in decision-making tasks. However, the dynamics of how this information is processed are not yet understood. We recorded ensembles of cells from OFC and vStr in rats engaged in the spatial adjusting delay-discounting task, a decision-making task that involves a trade-off between delay to and magnitude of reward. Ventral striatal neural activity signalled information about reward before the rat's decision, whereas such reward-related signals were absent in OFC until after the animal had committed to its decision. These data support models in which vStr is directly involved in action selection, but OFC processes decision-related information afterwards that can be used to compare the predicted and actual consequences of behaviour.

Neural mechanisms regulating different forms of risk-related decision-making: Insights from animal models

Over the past 20 years there has been a growing interest in the neural underpinnings of cost/benefit decision-making. Recent studies with animal models have made considerable advances in our understanding of how different prefrontal, striatal, limbic and monoaminergic circuits interact to promote efficient risk/reward decision-making, and how dysfunction in these circuits underlies aberrant decision-making observed in numerous psychiatric disorders. This review will highlight recent findings from studies exploring these questions using a variety of behavioral assays, as well as molecular, pharmacological, neurophysiological, and translational approaches. We begin with a discussion of how neural systems related to decision subcomponents may interact to generate more complex decisions involving risk and uncertainty. This is followed by an overview of interactions between prefrontal-amygdala-dopamine and habenular circuits in regulating choice between certain and uncertain rewards and how different modes of dopamine transmission may contribute to these processes. These data will be compared with results from other studies investigating the contribution of some of these systems to guiding decision-making related to rewards vs. punishment. Lastly, we provide a brief summary of impairments in risk-related decision-making associated with psychiatric disorders, highlighting recent translational studies in laboratory animals.

Dissociable roles of dopamine and serotonin transporter function in a rat model of negative urgency

Negative urgency is a facet of impulsivity that reflects mood-based rash action and is associated with various maladaptive behaviors in humans. However, the underlying neural mechanisms of negative urgency are not fully understood. Several brain regions within the mesocorticolimbic pathway, as well as the neurotransmitters dopamine (DA) and serotonin (5-HT), have been implicated in impulsivity. Extracellular DA and 5-HT concentrations are regulated by DA transporters (DAT) and 5-HT transporters (SERT); thus, these transporters may be important molecular mechanisms underlying individual differences in negative urgency. The current study employed a reward omission task to model negative urgency in rats. During reward trials, a cue light signaled the non-contingent delivery of one sucrose pellet; immediately following the non-contingent reward, rats responded on a lever to earn sucrose pellets (operant phase). Omission trials were similar to reward trials, except that non-contingent sucrose was omitted following the cue light prior to the operant phase. As expected, contingent responding was higher following omission of expected reward than following delivery of expected reward, thus reflecting negative urgency. Upon completion of behavioral training, Vmax and Km were obtained from kinetic analysis of [(3)H]DA and [(3)H]5-HT uptake using synaptosomes prepared from nucleus accumbens (NAc), dorsal striatum (Str), medial prefrontal cortex (mPFC), and orbitofrontal cortex (OFC) isolated from individual rats. Vmax for DAT in NAc and for SERT in OFC were positively correlated with negative urgency scores. The current findings suggest that mood-based impulsivity (negative urgency) is associated with enhanced DAT function in NAc and SERT function in OFC.

Dopamine D1/D2 Receptor Activity in the Nucleus Accumbens Core But Not in the Nucleus Accumbens Shell and Orbitofrontal Cortex Modulates Risk-Based Decision Making

BACKGROUND

It is well known that brain dopamine (DA) signals support risk-based decision making; however, the specific terminal regions of midbrain DA neurons through which DA signals mediate risk-based decision making are unknown.

METHODS

Using microinfusions of the D1/D2 receptor antagonist flupenthixol, we sought to explore the role of D1/D2 receptor activity in the rat orbitofrontal cortex (OFC) and core and shell regions of the nucleus accumbens (AcbC and AcbS, respectively) in the regulation of risky choices. A risk-discounting task was used that involves choices between a certain small-reward lever that always delivered 1 pellet or a risky large-reward lever which delivered 4 pellets but had a decreasing probability of receiving the reward across 4 subsequent within-session trial blocks (100%, 50%, 25%, 12.5%). To validate task sensitivity to experimental manipulations of DA activity, we also examined the effects of systemic amphetamine and flupenthixol.

RESULTS

Systemic amphetamine increased while systemic flupenthixol reduced risky choices. Results further demonstrate that rats that received intra-AcbC flupenthixol were able to track increasing risk associated with the risky lever but displayed a generally reduced preference for the risky lever across all trial blocks, including in the initial trial block (large reward at 100%). Microinfusions of flupenthixol into the AcbS or OFC did not alter risk-based decision making.

CONCLUSIONS

Our data suggest that intra-AcbC D1/D2 receptor signaling does not support the ability to track shifts in reward probabilities but does bias risk-based decision making. That is, it increased the rats' preference for the response option known to be associated with higher risk-related costs.

Dissociable roles for the basolateral amygdala and orbitofrontal cortex in decision-making under risk of punishment

Several neuropsychiatric disorders are associated with abnormal decision-making involving risk of punishment, but the neural basis of this association remains poorly understood. Altered activity in brain systems including the basolateral amygdala (BLA) and orbitofrontal cortex (OFC) can accompany these same disorders, and these structures are implicated in some forms of decision-making. The current study investigated the role of the BLA and OFC in decision-making under risk of explicit punishment. Rats were trained in the risky decision-making task (RDT), in which they chose between two levers, one that delivered a small safe reward, and the other that delivered a large reward accompanied by varying risks of footshock punishment. Following training, they received sham or neurotoxic lesions of BLA or OFC, followed by RDT retesting. BLA lesions increased choice of the large risky reward (greater risk-taking) compared to both prelesion performance and sham controls. When reward magnitudes were equated, both BLA lesion and control groups shifted their choice to the safe (no shock) reward lever, indicating that the lesions did not impair punishment sensitivity. In contrast to BLA lesions, OFC lesions significantly decreased risk-taking compared with sham controls, but did not impair discrimination between different reward magnitudes or alter baseline levels of anxiety. Finally, neither lesion significantly affected food-motivated lever pressing under various fixed ratio schedules, indicating that lesion-induced alterations in risk-taking were not secondary to changes in appetitive motivation. Together, these findings indicate distinct roles for the BLA and OFC in decision-making under risk of explicit punishment.

Age differences in strategy selection and risk preference during risk-based decision making

Studies of the effects of aging on decision making suggest that choices can be altered in a variety of ways depending on the situation, the nature of the outcome and risk, or certainty levels. To better characterize how aging impacts decision making in rodents, young and aged Fischer 344 rats underwent a series of probabilistic discounting tasks in which reward magnitude and probabilities were manipulated. Young rats tended to choose 1 of 2 different strategies: (a) to press for the large/uncertain reward, regardless of the reward probability; or (b) to continually adapt their behavior according to the odds of winning. The first strategy was adopted by about half of the younger rats, the second by the remaining young animals and the entire group of aged rats. Additionally, we found that when the odds of winning were varied from uncertain to certain during a session, aged rats chose most often the lever associated with the small/certain reward. This is consistent with an interpretation of increased risk aversion. When this behavior was further characterized using a lose-shift analysis, it appears that older rats exhibited an increased sensitivity to negative feedback. In contrast, sensitivity to wins was unaltered in aged rats compared with young, suggesting that aging selectively impacts rat's behavior following losses. In line with some human aging studies, it appears that aged rats are either more risk averse or have a greater certainty bias, which may result from age differences in emotion regulation.

Relative gains, losses, and reference points in probabilistic choice in rats

Theoretical reference points have been proposed to differentiate probabilistic gains from probabilistic losses in humans, but such a phenomenon in non-human animals has yet to be thoroughly elucidated. Three experiments evaluated the effect of reward magnitude on probabilistic choice in rats, seeking to determine reference point use by examining the effect of previous outcome magnitude(s) on subsequent choice behavior. Rats were trained to choose between an outcome that always delivered reward (low-uncertainty choice) and one that probabilistically delivered reward (high-uncertainty). The probability of high-uncertainty outcome receipt and the magnitudes of low-uncertainty and high-uncertainty outcomes were manipulated within and between experiments. Both the low- and high-uncertainty outcomes involved variable reward magnitudes, so that either a smaller or larger magnitude was probabilistically delivered, as well as reward omission following high-uncertainty choices. In Experiments 1 and 2, the between groups factor was the magnitude of the high-uncertainty-smaller (H-S) and high-uncertainty-larger (H-L) outcome, respectively. The H-S magnitude manipulation differentiated the groups, while the H-L magnitude manipulation did not. Experiment 3 showed that manipulating the probability of differential losses as well as the expected value of the low-uncertainty choice produced systematic effects on choice behavior. The results suggest that the reference point for probabilistic gains and losses was the expected value of the low-uncertainty choice. Current theories of probabilistic choice behavior have difficulty accounting for the present results, so an integrated theoretical framework is proposed. Overall, the present results have implications for understanding individual differences and corresponding underlying mechanisms of probabilistic choice behavior.

Dopaminergic circuitry and risk/reward decision making: implications for schizophrenia

Abnormal reinforcement learning and representations of reward value are present in schizophrenia, and these impairments can manifest as deficits in risk/reward decision making. These abnormalities may be due in part to dopaminergic dysfunction within cortico-limbic-striatal circuitry. Evidence from studies with laboratory animal have revealed that normal DA activity within different nodes of these circuits is critical for mediating dissociable processes that can refine decision biases. Moreover, both phasic and tonic dopamine transmission appear to play separate yet complementary roles in these processes. Tonic dopamine release within the prefrontal cortex and nucleus accumbens, serves as a "running rate-meter" of reward and reflects contextual information such as reward uncertainty and overt choice behavior. On the other hand, manipulations of outcome-related phasic dopamine bursts and dips suggest these signals provide rapid feedback to allow for quick adjustments in choice as reward contingencies change. The lateral habenula is a key input to the DA system that phasic signals is necessary for expressing subjective decision biases; as suppression of activity within this nucleus leads to catastrophic impairments in decision making and random patterns of choice behavior. As schizophrenia is characterized by impairments in using positive and negative feedback to appropriately guide decision making, these findings suggest that these deficits in these processes may be mediated, at least in part, by abnormalities in both tonic and phasic dopamine transmission.

Recent Translational Findings on Impulsivity in Relation to Drug Abuse

Impulsive behavior is strongly implicated in drug abuse, as both a cause and a consequence of drug use. To understand how impulsive behaviors lead to and result from drug use, translational evidence from both human and non-human animal studies is needed. Here, we review recent (2009 or later) studies that have investigated two major components of impulsive behavior, inhibitory control and impulsive choice, across preclinical and clinical studies. We concentrate on the stop-signal task as the measure of inhibitory control and delay discounting as the measure of impulsive choice. Consistent with previous reports, recent studies show greater impulsive behavior in drug users compared with non-users. Additionally, new evidence supports the prospective role of impulsive behavior in drug abuse, and has begun to identify the neurobiological mechanisms underlying impulsive behavior. We focus on the commonalities and differences in findings between preclinical and clinical studies, and suggest future directions for translational research.

Role of medial prefrontal and orbitofrontal monoamine transporters and receptors in performance in an adjusting delay discounting procedure

Performance in an adjusting delay discounting procedure is predictive of drug abuse vulnerability; however, the shared underlying specific prefrontal neural systems linking delay discounting and increased addiction-like behaviors are unclear. Rats received direct infusions of methylphenidate (MPH; 6.25, 25.0, or 100μg), amphetamine (AMPH; 0.25, 1.0, or 4.0μg), or atomoxetine (ATO; 1.0, 4.0, or 16.0μg) into either medial prefrontal cortex (mPFC) or orbitofrontal cortex (OFC) immediately prior to performance in an adjusting delay task. These drugs were examined because they are efficacious in treating impulse control disorders. Because dopamine (DA) and serotonin (5-HT) receptors are implicated in impulsive behavior, separate groups of rats received microinfusions of the DA receptor-selective drugs SKF 81297 (0.1 or 0.4µg), SCH 23390 (0.25 or 1.0µg), quinpirole (1.25 or 5.0µg), and eticlopride (0.25 or 1.0µg), or received microinfusions of the 5-HT receptor-selective drugs 8-OH-DPAT (0.025 or 0.1μg), WAY 100635 (0.01 or 0.04μg), DOI (2.5 or 10.0μg), and ketanserin (0.1 or 0.4μg). Impulsive choice was not altered significantly by MPH, AMPH, or ATO into either mPFC or OFC, indicating that neither of these prefrontal regions alone may mediate the systemic effect of ADHD medications on impulsive choice. However, quinpriole (1.25μg) and eticlopride infused into mPFC increased impulsive choice, whereas 8-OH-DPAT infused into OFC decreased impulsive choice. These latter results demonstrate that blockade of DA D2 receptors in mPFC or activation of 5-HT1A receptors in OFC increases impulsive choice in the adjusting delay procedure.

Affective and cognitive mechanisms of risky decision making

The ability to make advantageous decisions under circumstances in which there is a risk of adverse consequences is an important component of adaptive behavior; however, extremes in risk taking (either high or low) can be maladaptive and are characteristic of a number of neuropsychiatric disorders. To better understand the contributions of various affective and cognitive factors to risky decision making, cohorts of male Long-Evans rats were trained in a "Risky Decision making Task" (RDT), in which they made discrete trial choices between a small, "safe" food reward and a large, "risky" food reward accompanied by varying probabilities of footshock. Experiment 1 evaluated the relative contributions of the affective stimuli (i.e., punishment vs. reward) to RDT performance by parametrically varying the magnitudes of the footshock and large reward. Varying the shock magnitude had a significant impact on choice of the large, "risky" reward, such that greater magnitudes were associated with reduced choice of the large reward. In contrast, varying the large, "risky" reward magnitude had minimal influence on reward choice. Experiment 2 compared individual variability in RDT performance with performance in an attentional set shifting task (assessing cognitive flexibility), a delayed response task (assessing working memory), and a delay discounting task (assessing impulsive choice). Rats characterized as risk averse in the RDT made more perseverative errors on the set shifting task than did their risk taking counterparts, whereas RDT performance was not related to working memory abilities or impulsive choice. In addition, rats that showed greater delay discounting (greater impulsive choice) showed corresponding poorer performance in the working memory task. Together, these results suggest that reward-related decision making under risk of punishment is more strongly influenced by the punishment than by the reward, and that risky and impulsive decision making are associated with distinct components of executive function.

Circuit dynamics of adaptive and maladaptive behaviour

The recent development of technologies for investigating specific components of intact biological systems has allowed elucidation of the neural circuitry underlying adaptive and maladaptive behaviours. Investigators are now able to observe and control, with high spatio-temporal resolution, structurally defined intact pathways along which electrical activity flows during and after the performance of complex behaviours. These investigations have revealed that control of projection-specific dynamics is well suited to modulating behavioural patterns that are relevant to a broad range of psychiatric diseases. Structural dynamics principles have emerged to provide diverse, unexpected and causal insights into the operation of intact and diseased nervous systems, linking form and function in the brain.

Noradrenergic α2A-receptor stimulation in the ventral hippocampus reduces impulsive decision-making

RATIONALE

Guanfacine, an α2A-adrenergic receptor agonist, is currently in use for treatment of a variety of psychiatric disorders that are associated with impulsive decision-making (e.g., attention-deficit hyperactivity disorder; ADHD). In animals and humans, the behavioral effects of adrenergic agents are presumed to involve neuromodulation of the prefrontal cortex, consistent with the demonstrated actions of dopaminergic agents. However, recent experimental work has shown that the ventral hippocampus (vHC) contributes to decision-making and impulse control, raising the possibility that the hippocampus may be an important site of action for these drugs.

OBJECTIVE

The purpose of this study was to examine the effect of local vHC infusions of guanfacine and other neuropharmacological agents on behavioral decisions that involve a trade-off between reward size and delay.

METHODS

Different cohorts of rats were implanted with bilateral guide cannulae targeting the vHC. We examined the animals' behavior in a touchscreen version of a delay discounting task following intra-vHC infusions of: (a) guanfacine (α2A-adrenergic receptor agonist), (b) SCH 23390 (dopamine D1 receptor antagonist), and (c) muscimol/baclofen (GABAA/B agonists).

RESULTS

Guanfacine led to a dose-dependent reduction in impulsive decision-making, increasing the animals' tolerance for delay in exchange for a larger reward. By contrast, infusion of SCH 23390 had no behavioral effects. Consistent with previous lesion studies, reversible pharmacological inactivation with muscimol/baclofen increased impulsive decision-making.

CONCLUSIONS

These data provide the first evidence that guanfacine, a commonly used treatment for ADHD, may derive its clinical benefits through hippocampal stimulation, via α2A-adrenergic receptors.

The orbitofrontal cortex as part of a hierarchical neural system mediating choice between two good options

Animals rely on environmental cues to identify potential rewards and select the best reward available. The orbitofrontal cortex (OFC) is proposed to encode sensory-specific representations of expected outcome. However, its contribution to the selection of a preferred outcome among different reward options is still unclear. We investigated the effect of transient OFC inactivation (achieved by presession injection of muscimol and baclofen) in a novel two-reward choice task. In discrete trials, rats could choose between a solution of polycose and an equally caloric, but highly preferred, solution of sucrose by visiting one of two liquid dispensers after the presentation of a specific cue signaling the availability of one or both of the solutions. We found that OFC inactivation did not affect outcome preference: rats maintained high preference for sucrose and adapted their behavioral responding when the cue-outcome contingencies were reversed. However, when rats were tested drug-free 24 h after OFC inactivation and reversal learning, memory for the newly learned contingencies was poor. These results suggest a potential conflict between OFC (encoding pre-reversal contingencies) and other brain circuits (encoding the new contingencies). Remarkably, repeating the OFC inactivation before the reversal memory test restored normal behavior, confirming the hypothesis of a dominant impact of OFC on other decision-making circuits. These results indicate that the representations encoded in the OFC, while not essential to the expression of outcome preference, exert hierarchical control on downstream decision-making circuits.

Increased firing to cues that predict low-value reward in the medial orbitofrontal cortex

Anatomical, imaging, and lesion work have suggested that medial and lateral aspects of orbitofrontal cortex (OFC) play different roles in reward-guided decision-making, yet few single-neuron recording studies have examined activity in more medial parts of the OFC (mOFC) making it difficult to fully assess its involvement in motivated behavior. Previously, we have shown that neurons in lateral parts of the OFC (lOFC) selectively fire for rewards of different values. In that study, we trained rats to respond to different fluid wells for rewards of different sizes or delivered at different delays. Rats preferred large over small reward, and rewards delivered after short compared with long delays. Here, we recorded from single neurons in rat rostral mOFC as they performed the same task. Similar to the lOFC, activity was attenuated for rewards that were delivered after long delays and was enhanced for delivery of larger rewards. However, unlike lOFC, odor-responsive neurons in the mOFC were more active when cues predicted low-value outcomes. These data suggest that odor-responsive mOFC neurons signal the association between environmental cues and unfavorable outcomes during decision making.