Selective involvement by the medial orbitofrontal cortex in biasing risky, but not impulsive, choice

Perturbations in risk/reward decision making and frontal cortical catecholamine regulation induced by mild traumatic brain injury

Mild traumatic brain injury (mTBI) disrupts cognitive processes that influence risk taking behavior. Little is known regarding the effects of repetitive mild injury (rmTBI) or whether these outcomes are sex specific. Risk/reward decision making is mediated by the prefrontal cortex (PFC), which is densely innervated by catecholaminergic fibers. Aberrant PFC catecholamine activity has been documented following TBI and may underlie TBI-induced risky behavior. The present study characterized the effects of rmTBI on risk/reward decision making behavior and catecholamine transmitter regulatory proteins within the PFC. Rats were exposed to sham, single (smTBI), or three closed-head controlled cortical impact (CH-CCI) injuries and assessed for injury-induced effects on risk/reward decision making using a probabilistic discounting task (PDT). In the first week post-final surgery, mTBI increased risky choice preference. By the fourth week, males exhibited increased latencies to make risky choices following rmTBI, demonstrating a delayed effect on processing speed. When levels of tyrosine hydroxylase (TH) and the norepinephrine reuptake transporter (NET) were measured within subregions of the PFC, females exhibited dramatic increases of TH levels within the orbitofrontal cortex (OFC) following smTBI. However, both males and females demonstrated reduced levels of OFC NET following rmTBI. These results indicate the OFC is susceptible to catecholamine instability after rmTBI and suggests that not all areas of the PFC contribute equally to TBI-induced imbalances. Overall, the CH-CCI model of rmTBI has revealed time-dependent and sex-specific changes in risk/reward decision making and catecholamine regulation following repetitive mild head injuries.

A critical review of risk-sensitive foraging

Foraging is risk sensitive if choices depend on the variability of returns from the options as well as their mean return. Risk-sensitive foraging is important in behavioural ecology, psychology and neurophysiology. It has been explained both in terms of mechanisms and in terms of evolutionary advantage. We provide a critical review, evaluating both mechanistic and evolutionary accounts. Some derivations of risk sensitivity from mechanistic models based on psychophysics are not convincing because they depend on an inappropriate use of Jensen's inequality. Attempts have been made to link risk sensitivity to the ecology of a species, but again these are not convincing. The field of risk-sensitive foraging has provided a focus for theoretical and empirical work and has yielded important insights, but we lack a simple and empirically defendable general account of it in either mechanistic or evolutionary terms. However, empirical analysis of choice sequences under theoretically motivated experimental designs and environmental settings appears a promising avenue for mapping the scope and relative merits of existing theories. Simply put, the devil is in the sequence.

Postnatal Phencyclidine-Induced Deficits in Decision Making Are Ameliorated by Optogenetic Inhibition of Ventromedial Orbitofrontal Cortical Glutamate Neurons

Background

The orbitofrontal cortex (OFC) is essential for decision making, and functional disruptions within the OFC are evident in schizophrenia. Postnatal phencyclidine (PCP) administration in rats is a neurodevelopmental manipulation that induces schizophrenia-relevant cognitive impairments. We aimed to determine whether manipulating OFC glutamate cell activity could ameliorate postnatal PCP-induced deficits in decision making.

Methods

Male and female Wistar rats (n = 110) were administered saline or PCP on postnatal days 7, 9, and 11. In adulthood, we expressed YFP (yellow fluorescent protein) (control), ChR2 (channelrhodopsin-2) (activation), or eNpHR 3.0 (enhanced halorhodopsin) (inhibition) in glutamate neurons within the ventromedial OFC (vmOFC). Rats were tested on the probabilistic reversal learning task once daily for 20 days while we manipulated the activity of vmOFC glutamate cells. Behavioral performance was analyzed using a Q-learning computational model of reinforcement learning.

Results

Compared with saline-treated rats expressing YFP, PCP-treated rats expressing YFP completed fewer reversals, made fewer win-stay responses, and had lower learning rates. We induced similar performance impairments in saline-treated rats by activating vmOFC glutamate cells (ChR2). Strikingly, PCP-induced performance deficits were ameliorated when the activity of vmOFC glutamate cells was inhibited (halorhodopsin).

Conclusions

Postnatal PCP-induced deficits in decision making are associated with hyperactivity of vmOFC glutamate cells. Thus, normalizing vmOFC activity may represent a potential therapeutic target for decision-making deficits in patients with schizophrenia.

Selective chemogenetic inactivation of corticoaccumbal projections disrupts trait choice impulsivity

Impulsive choice has enduring trait-like characteristics and is defined by preference for small immediate rewards over larger delayed ones. Importantly, it is a determining factor in the development and persistence of substance use disorder (SUD). Emerging evidence from human and animal studies suggests frontal cortical regions exert influence over striatal reward processing areas during decision-making in impulsive choice or delay discounting (DD) tasks. The goal of this study was to examine how these circuits are involved in decision-making in animals with defined trait impulsivity. To this end, we trained adolescent male rats to stable behavior on a DD procedure and then re-trained them in adulthood to assess trait-like, conserved impulsive choice across development. We then used chemogenetic tools to selectively and reversibly target corticostriatal projections during performance of the DD task. The prelimbic region of the medial prefrontal cortex (mPFC) was injected with a viral vector expressing inhibitory designer receptors exclusively activated by designer drugs (Gi-DREADD), and then mPFC projections to the nucleus accumbens core (NAc) were selectively suppressed by intra-NAc administration of the Gi-DREADD actuator clozapine-n-oxide (CNO). Inactivation of the mPFC-NAc projection elicited a robust increase in impulsive choice in rats with lower vs. higher baseline impulsivity. This demonstrates a fundamental role for mPFC afferents to the NAc during choice impulsivity and suggests that maladaptive hypofrontality may underlie decreased executive control in animals with higher levels of choice impulsivity. Results such as these may have important implications for the pathophysiology and treatment of impulse control, SUDs, and related psychiatric disorders.

Medial orbitofrontal cortical regulation of different aspects of Pavlovian and instrumental reward seeking

RATIONALE

The medial subregion of the orbitofrontal cortex (mOFC) is thought to play an important role representing the expected outcome of a given course of action, as lesioning or inactivating this cortical region results in the adoption of choice strategies based more on observable (rather than previously learned) information. Despite this, its role in mediating basic associative learning remains to be fully clarified.

OBJECTIVE

The present series of experiments examined the role of the mOFC in (1) Pavlovian conditioned approach, (2) conditioned reinforcement, (3) extinction, and (4) cue-induced reinstatement of food-seeking behavior.

METHODS

Separate cohorts of rats went through Pavlovian or instrumental training. Intra-mOFC infusions of either saline or GABA agonists (to temporarily inactivate neural activity) were given prior to Pavlovian approach, conditioned reinforcement, first or second day of instrumental extinction training, or cue-induced reinstatement test days.

RESULTS

mOFC inactivation increased lever-CS contacts in Pavlovian conditioned approach and (2) had no effect on conditioned reinforcement. These manipulations (3) accelerated within-session instrumental extinction during the initial extinction session, but impaired subsequent extinction learning on drug-free days. (4) mOFC inactivation induced differential effects on reinstatement that depended on baseline performance. mOFC inactivation abolished reinstatement in "Reinstater" rats (who displayed robust responding under control conditions) and robustly increased reinstatement in "Non-Reinstater" rats (who showed little reinstatement under control conditions) suggesting that individual differences in reinstatement may be supported by differences in mOFC mediated representations of expected outcomes.

CONCLUSIONS

These findings have important implications for understanding how the mOFC uses stimulus-outcome and action-outcome expectancies to guide behavior, and how dysfunction within this region may contribute to pathological patterns of reward seeking.

The medial orbitofrontal cortex governs reward-related circuits in an age-dependent manner

Nucleus accumbens (NAc) neurons integrate excitatory inputs from cortical and limbic structures, contributing to critical cognitive functions, including decision-making. As these afferents mature from adolescence through adulthood, incoming signals to the NAc may summate differently between age groups. Decision-making evaluates both reward and risk before action selection, suggesting an interplay between reward- and risk-related circuits. Medial orbitofrontal cortex (MO)-NAc circuits permit risk assessment behaviors and likely underlie risk information incorporation. As adolescents make reward-centric choices regardless of risk, we hypothesized the impact of MO activity alters reward-related NAc circuits in an age-dependent manner. To test this hypothesis, we used single-unit electrophysiology to measure MO train stimulation's effect on reward-related pathways, specifically the basolateral amygdala (BLA)-NAc circuit, in adult and adolescent rats. MO train stimulation altered the strength but not the timing of BLA-NAc interactions in a frequency-dependent manner. In adults, MO train stimulation produced a frequency-dependent, bidirectional effect on BLA-evoked NAc AP probability. Contrastingly, MO train stimulation uniformly attenuated BLA-NAc interactions in adolescents. While the mature MO can govern reward-related circuits in an activity-dependent manner, perhaps to adapt to positive or negative decision-making outcomes, the adolescent MO may be less able to bidirectionally impact reward-related pathways resulting in biased decision-making.

Contrasting effects of DOI and lisuride on impulsive decision-making in delay discounting task

RATIONALE

The 5-HT2A receptor is the major target of classic hallucinogens. Both DOI (2,5-dimethoxy-4-iodoamphetamine) and lisuride act at 5-HT2A receptors, and lisuride shares comparable affinity with DOI and acts as a partial agonist at 5-HT2A receptors. However, not like DOI, lisuride lacks hallucinogenic properties. Impulsive decision-making refers to the preference for an immediate small reinforcer (SR) over a delayed large reinforcer (LR).

OBJECTIVES

The current study aims to compare the effects of DOI and lisuride on impulsive decision-making and further to investigate the possible receptor mechanisms responsible for the actions of the two drugs.

METHODS

Impulsive decision-making was evaluated in male Sprague-Dawley rats by the percentage of choice for the LR in delay discounting task (DDT). Delay to the LR changed in an ascending order (0, 4, 8, 16, and 32 s) across one session.

RESULTS

DOI (0.5 and 1.0 mg/kg) increased impulsive decision-making, and the effects of DOI (1.0 mg/kg) were blocked by the 5-HT2A receptor antagonist ketanserin (1.0 mg/kg) rather than the 5-HT2C receptor antagonist SB-242084 (1.0 mg/kg). Contrarily, lisuride (0.1, 0.3, and 0.5 mg/kg) decreased impulsive decision-making. The effects of lisuride (0.3 mg/kg) were not antagonized by ketanserin (1.0 mg/kg), selective 5-HT1A antagonist WAY-100635 (1.0 mg/kg), or selective dopamine D4 receptor antagonist L-745870 (1.0 mg/kg) but were attenuated by the selective dopamine D2/D3 receptor antagonist tiapride (40 mg/kg).

CONCLUSIONS

DOI and lisuride have contrasting effects on impulsive decision-making via distinct receptors. DOI-induced increase of impulsivity is mediated by the 5-HT2A receptor, while lisuride-induced inhibition of impulsivity is regulated by the dopamine D2/D3 receptor.

Medial orbitofrontal neurotrophin systems integrate hippocampal input into outcome-specific value representations

In everyday life, we mentally represent possible consequences of our behaviors and integrate specific outcome values into existing knowledge to inform decisions. The medial orbitofrontal cortex (MO) is necessary to adapt behaviors when outcomes are not immediately available-when they and their values need to be envisioned. Nevertheless, neurobiological mechanisms remain unclear. We find that the neuroplasticity-associated neurotrophin receptor tropomyosin receptor kinase B (TrkB) is necessary for mice to integrate outcome-specific value information into choice behavior. This function appears attributable to memory updating (and not retrieval) and the stabilization of dendritic spines on excitatory MO neurons, which led us to investigate inputs to the MO. Ventral hippocampal (vHC)-to-MO projections appear conditionally necessary for value updating, involved in long-term aversion-based value memory updating. Furthermore, vHC-MO-mediated control of choice is TrkB dependent. Altogether, we reveal a vHC-MO connection by which specific value memories are updated, and we position TrkB within this functional circuit.

Amygdala-cortical collaboration in reward learning and decision making

Adaptive reward-related decision making requires accurate prospective consideration of the specific outcome of each option and its current desirability. These mental simulations are informed by stored memories of the associative relationships that exist within an environment. In this review, I discuss recent investigations of the function of circuitry between the basolateral amygdala (BLA) and lateral (lOFC) and medial (mOFC) orbitofrontal cortex in the learning and use of associative reward memories. I draw conclusions from data collected using sophisticated behavioral approaches to diagnose the content of appetitive memory in combination with modern circuit dissection tools. I propose that, via their direct bidirectional connections, the BLA and OFC collaborate to help us encode detailed, outcome-specific, state-dependent reward memories and to use those memories to enable the predictions and inferences that support adaptive decision making. Whereas lOFC→BLA projections mediate the encoding of outcome-specific reward memories, mOFC→BLA projections regulate the ability to use these memories to inform reward pursuit decisions. BLA projections to lOFC and mOFC both contribute to using reward memories to guide decision making. The BLA→lOFC pathway mediates the ability to represent the identity of a specific predicted reward and the BLA→mOFC pathway facilitates understanding of the value of predicted events. Thus, I outline a neuronal circuit architecture for reward learning and decision making and provide new testable hypotheses as well as implications for both adaptive and maladaptive decision making.

Medial orbitofrontal cortex and nucleus accumbens mediation in risk assessment behaviors in adolescents and adults

Risk assessment behaviors are necessary for gathering risk information and guiding decision-making. Risky decision-making heightens during adolescence, possibly as a result of low risk awareness and an increase in sensitivity to reward-associated cues and experiences. Higher adolescent engagement in high-risk behaviors may be, in part, due to developing circuits that contribute to risk assessment behaviors. Nucleus accumbens (NAc) activity is linked to risky decision-making and receives inputs carrying sensory and emotional information. Namely, the medial orbitofrontal cortex (MO) contributes to behavior guided by reward probability and sends direct projections to the NAc (MO→NAc), which may permit risk assessment in a mature circuit. Here, we evaluated risk assessment behaviors in adult and adolescent rats during elevated plus maze (EPM) exploration, including stretch and attend postures, head dips, and rears. We found that adolescents exhibited fewer EPM risk assessment behaviors than adults. We also quantified MO→NAc projections using a fluorescent anterograde tracer, Fluoro-Ruby, in both age groups. Labeled MO→NAc pathways exhibited greater total fluorescence in adults than in adolescents, indicating MO→NAc fibers increase over development. Using a disconnection approach to measure the contribution of the MO-NAc pathway in adults, we found that ipsilateral inactivation of the MO-NAc did not alter risk assessment behavior; however, MO-NAc disconnection reduced the number of stretch-and-attend postures. Together, this work suggests that the development of MO-NAc pathways can contribute to age-dependent differences in risk assessment.

Sequential delay and probability discounting tasks in mice reveal anchoring effects partially attributable to decision noise

Delay discounting and probability discounting decision making tasks in rodent models have high translational potential. However, it is unclear whether the discounted value of the large reward option is the main contributor to variability in animals' choices in either task, which may limit translation to humans. Male and female mice underwent sessions of delay and probability discounting in sequence to assess how choice behavior adapts over experience with each task. To control for "anchoring" (persistent choices based on the initial delay or probability), mice experienced "Worsening" schedules where the large reward was offered under initially favorable conditions that became less favorable during testing, followed by "Improving" schedules where the large reward was offered under initially unfavorable conditions that improved over a session. During delay discounting, both male and female mice showed elimination of anchoring effects over training. In probability discounting, both sexes of mice continued to show some anchoring even after months of training. One possibility is that "noisy", exploratory choices could contribute to these persistent anchoring effects, rather than constant fluctuations in value discounting. We fit choice behavior in individual animals using models that included both a value-based discounting parameter and a decision noise parameter that captured variability in choices deviating from value maximization. Changes in anchoring behavior over time were tracked by changes in both the value and decision noise parameters in delay discounting, but by the decision noise parameter in probability discounting. Exploratory decision making was also reflected in choice response times that tracked the degree of conflict caused by both uncertainty and temporal cost, but was not linked with differences in locomotor activity reflecting chamber exploration. Thus, variable discounting behavior in mice can result from changes in exploration of the decision options rather than changes in reward valuation.

Regulation of sex differences in risk-based decision making by gonadal hormones: Insights from rodent models

Men and women differ in their ability to evaluate options that vary in their rewards and the risks that are associated with these outcomes. Most studies have shown that women are more risk averse than men and that gonadal hormones significantly contribute to this sex difference. Gonadal hormones can influence risk-based decision making (i.e., risk taking) by modulating the neurobiological substrates underlying this cognitive process. Indeed, estradiol, progesterone and testosterone modulate activity in the prefrontal cortex, amygdala and nucleus accumbens associated with reward and risk-related information. The use of animal models of decision making has advanced our understanding of the intersection between the behavioral, neural and hormonal mechanisms underlying sex differences in risk taking. This review will outline the current state of this literature, identify the current gaps in knowledge and suggest the neurobiological mechanisms by which hormones regulate risky decision making. Collectively, this knowledge can be used to understand the potential consequences of significant hormonal changes, whether endogenously or exogenously induced, on risk-based decision making as well as the neuroendocrinological basis of neuropsychiatric diseases that are characterized by impaired risk taking, such as substance use disorder and schizophrenia.

Prefrontal-hippocampal interactions supporting the extinction of emotional memories: the retrieval stopping model

Neuroimaging has revealed robust interactions between the prefrontal cortex and the hippocampus when people stop memory retrieval. Efforts to stop retrieval can arise when people encounter reminders to unpleasant thoughts they prefer not to think about. Retrieval stopping suppresses hippocampal and amygdala activity, especially when cues elicit aversive memory intrusions, via a broad inhibitory control capacity enabling prepotent response suppression. Repeated retrieval stopping reduces intrusions of unpleasant memories and diminishes their affective tone, outcomes resembling those achieved by the extinction of conditioned emotional responses. Despite this resemblance, the role of inhibitory fronto-hippocampal interactions and retrieval stopping broadly in extinction has received little attention. Here we integrate human and animal research on extinction and retrieval stopping. We argue that reconceptualising extinction to integrate mnemonic inhibitory control with learning would yield a greater understanding of extinction's relevance to mental health. We hypothesize that fear extinction spontaneously engages retrieval stopping across species, and that controlled suppression of hippocampal and amygdala activity by the prefrontal cortex reduces fearful thoughts. Moreover, we argue that retrieval stopping recruits extinction circuitry to achieve affect regulation, linking extinction to how humans cope with intrusive thoughts. We discuss novel hypotheses derived from this theoretical synthesis.

Probabilistic Reinforcement Learning and Anhedonia

Despite the prominence of anhedonic symptoms associated with diverse neuropsychiatric conditions, there are currently no approved therapeutics designed to attenuate the loss of responsivity to previously rewarding stimuli. However, the search for improved treatment options for anhedonia has been reinvigorated by a recent reconceptualization of the very construct of anhedonia, including within the Research Domain Criteria (RDoC) initiative. This chapter will focus on the RDoC Positive Valence Systems construct of reward learning generally and sub-construct of probabilistic reinforcement learning specifically. The general framework emphasizes objective measurement of a subject's responsivity to reward via reinforcement learning under asymmetrical probabilistic contingencies as a means to quantify reward learning. Indeed, blunted reward responsiveness and reward learning are central features of anhedonia and have been repeatedly described in major depression. Moreover, these probabilistic reinforcement techniques can also reveal neurobiological mechanisms to aid development of innovative treatment approaches. In this chapter, we describe how investigating reward learning can improve our understanding of anhedonia via the four RDoC-recommended tasks that have been used to probe sensitivity to probabilistic reinforcement contingencies and how such task performance is disrupted in various neuropsychiatric conditions. We also illustrate how reverse translational approaches of probabilistic reinforcement assays in laboratory animals can inform understanding of pharmacological and physiological mechanisms. Next, we briefly summarize the neurobiology of probabilistic reinforcement learning, with a focus on the prefrontal cortex, anterior cingulate cortex, striatum, and amygdala. Finally, we discuss treatment implications and future directions in this burgeoning area.

Foraging with the frontal cortex: A cross-species evaluation of reward-guided behavior

Efficient foraging is essential to survival and depends on frontal cortex in mammals. Because of its role in psychiatric disorders, frontal cortex and its contributions to reward procurement have been studied extensively in both rodents and non-human primates. How frontal cortex of these animal models compares is a source of intense debate. Here we argue that translating findings from rodents to non-human primates requires an appreciation of both the niche in which each animal forages as well as the similarities in frontal cortex anatomy and function. Consequently, we highlight similarities and differences in behavior and anatomy, before focusing on points of convergence in how parts of frontal cortex contribute to distinct aspects of foraging in rats and macaques, more specifically. In doing so, our aim is to emphasize where translation of frontal cortex function between species is clearer, where there is divergence, and where future work should focus. We finish by highlighting aspects of foraging for which have received less attention but we believe are critical to uncovering how frontal cortex promotes survival in each species.

Early life adversity promotes resilience to opioid addiction-related phenotypes in male rats and sex-specific transcriptional changes

Experiencing some early life adversity can have an "inoculating" effect that promotes resilience in adulthood. However, the mechanisms underlying stress inoculation are unknown, and animal models are lacking. Here we used the limited bedding and nesting (LBN) model of adversity to evaluate stress inoculation of addiction-related phenotypes. In LBN, pups from postnatal days 2 to 9 and their dams were exposed to a low-resource environment. In adulthood, they were tested for addiction-like phenotypes and compared to rats raised in standard housing conditions. High levels of impulsivity are associated with substance abuse, but in males, LBN reduced impulsive choice compared to controls. LBN males also self-administered less morphine and had a lower breakpoint on a progressive ratio reinforcement schedule than controls. These effects of LBN on addiction-related behaviors were not found in females. Because the nucleus accumbens (NAc) mediates these behaviors, we tested whether LBN altered NAc physiology in drug-naïve and morphine-exposed rats. LBN reduced the frequency of spontaneous excitatory postsynaptic currents in males, but a similar effect was not observed in females. Only in males did LBN prevent a morphine-induced increase in the AMPA/NMDA ratio. RNA sequencing was performed to delineate the molecular signature in the NAc associated with LBN-derived phenotypes. LBN produced sex-specific changes in transcription, including in genes related to glutamate transmission. Collectively, these studies reveal that LBN causes a male-specific stress inoculation effect against addiction-related phenotypes. Identifying factors that promote resilience to addiction may reveal novel treatment options for patients.

Shifts in Medial Orbitofrontal Cortex Activity from Adolescence to Adulthood

Adolescents are characterized by a propensity for risky and impulsive behaviors, likely due to immature frontostriatal circuits. The medial orbitofrontal cortex (MO) is linked to risk and reward prediction during decision-making. Identifying age-dependent differences in MO activity and its inputs to downstream regions can elucidate the neural substrates that permit the transition from high-risk adolescent behaviors to increased risk assessment in adulthood. Action selection biased by information gathered by the MO is likely carried out by efferents into the nucleus accumbens (NAc), which guides reward-directed behaviors. Despite the large age dependency of risk-based decision-making, there is nothing known about adolescent MO activity. Here, we recorded action potentials of MO neurons from anesthetized adult and adolescent rats in vivo. On average, adolescent MO neurons fire faster and within narrower ranges than adults, and adolescents have more active MO neurons than adults. Using antidromic stimulation of axon terminals to identify MO neurons that project to NAc (MO→NAc), we found that adolescent MO→NAc neurons have a narrower range of firing frequencies than non-NAc-projecting MO neurons and adult MO→NAc neurons. These age-dependent differences in MO and MO→NAc populations may result from the fine-tuning of circuits between adolescence and adulthood that promote specific age-dependent behaviors.

The role of the medial prefrontal cortex in cognition, ageing and dementia

Humans require a plethora of higher cognitive skills to perform executive functions, such as reasoning, planning, language and social interactions, which are regulated predominantly by the prefrontal cortex. The prefrontal cortex comprises the lateral, medial and orbitofrontal regions. In higher primates, the lateral prefrontal cortex is further separated into the respective dorsal and ventral subregions. However, all these regions have variably been implicated in several fronto-subcortical circuits. Dysfunction of these circuits has been highlighted in vascular and other neurocognitive disorders. Recent advances suggest the medial prefrontal cortex plays an important regulatory role in numerous cognitive functions, including attention, inhibitory control, habit formation and working, spatial or long-term memory. The medial prefrontal cortex appears highly interconnected with subcortical regions (thalamus, amygdala and hippocampus) and exerts top-down executive control over various cognitive domains and stimuli. Much of our knowledge comes from rodent models using precise lesions and electrophysiology readouts from specific medial prefrontal cortex locations. Although, anatomical disparities of the rodent medial prefrontal cortex compared to the primate homologue are apparent, current rodent models have effectively implicated the medial prefrontal cortex as a neural substrate of cognitive decline within ageing and dementia. Human brain connectivity-based neuroimaging has demonstrated that large-scale medial prefrontal cortex networks, such as the default mode network, are equally important for cognition. However, there is little consensus on how medial prefrontal cortex functional connectivity specifically changes during brain pathological states. In context with previous work in rodents and non-human primates, we attempt to convey a consensus on the current understanding of the role of predominantly the medial prefrontal cortex and its functional connectivity measured by resting-state functional MRI in ageing associated disorders, including prodromal dementia states, Alzheimer's disease, post-ischaemic stroke, Parkinsonism and frontotemporal dementia. Previous cross-sectional studies suggest that medial prefrontal cortex functional connectivity abnormalities are consistently found in the default mode network across both ageing and neurocognitive disorders such as Alzheimer's disease and vascular cognitive impairment. Distinct disease-specific patterns of medial prefrontal cortex functional connectivity alterations within specific large-scale networks appear to consistently feature in the default mode network, whilst detrimental connectivity alterations are associated with cognitive impairments independently from structural pathological aberrations, such as grey matter atrophy. These disease-specific patterns of medial prefrontal cortex functional connectivity also precede structural pathological changes and may be driven by ageing-related vascular mechanisms. The default mode network supports utility as a potential biomarker and therapeutic target for dementia-associated conditions. Yet, these associations still require validation in longitudinal studies using larger sample sizes.

Medial orbitofrontal cortex dopamine D1/D2 receptors differentially modulate distinct forms of probabilistic decision-making

Efficient decision-making involves weighing the costs and benefits associated with different actions and outcomes to maximize long-term utility. The medial orbitofrontal cortex (mOFC) has been implicated in guiding choice in situations involving reward uncertainty, as inactivation in rats alters choice involving probabilistic rewards. The mOFC receives considerable dopaminergic input, yet how dopamine (DA) modulates mOFC function has been virtually unexplored. Here, we assessed how mOFC D₁ and D₂ receptors modulate two forms of reward seeking mediated by this region, probabilistic reversal learning and probabilistic discounting. Separate groups of well-trained rats received intra-mOFC microinfusions of selective D₁ or D₂ antagonists or agonists prior to task performance. mOFC D₁ and D₂ blockade had opposing effects on performance during probabilistic reversal learning and probabilistic discounting. D₁ blockade impaired, while D₂ blockade increased the number of reversals completed, both mediated by changes in errors and negative feedback sensitivity apparent during the initial discrimination of the task, which suggests changes in probabilistic reinforcement learning rather than flexibility. Similarly, D₁ blockade reduced, while D₂ blockade increased preference for larger/risky rewards. Excess D₁ stimulation had no effect on either task, while excessive D₂ stimulation impaired probabilistic reversal performance, and reduced both profitable risky choice and overall task engagement. These findings highlight a previously uncharacterized role for mOFC DA, showing that D₁ and D₂ receptors play dissociable and opposing roles in different forms of reward-related action selection. Elucidating how DA biases behavior in these situations will expand our understanding of the mechanisms regulating optimal and aberrant decision-making.

Advances in understanding meso-cortico-limbic-striatal systems mediating risky reward seeking

The risk of an aversive consequence occurring as the result of a reward-seeking action can have a profound effect on subsequent behavior. Such aversive events can be described as punishers, as they decrease the probability that the same action will be produced again in the future and increase the exploration of less risky alternatives. Punishment can involve the omission of an expected rewarding event ("negative" punishment) or the addition of an unpleasant event ("positive" punishment). Although many individuals adaptively navigate situations associated with the risk of negative or positive punishment, those suffering from substance use disorders or behavioral addictions tend to be less able to curtail addictive behaviors despite the aversive consequences associated with them. Here, we discuss the psychological processes underpinning reward seeking despite the risk of negative and positive punishment and consider how behavioral assays in animals have been employed to provide insights into the neural mechanisms underlying addictive disorders. We then review the critical contributions of dopamine signaling to punishment learning and risky reward seeking, and address the roles of interconnected ventral striatal, cortical, and amygdala regions to these processes. We conclude by discussing the ample opportunities for future study to clarify critical gaps in the literature, particularly as related to delineating neural contributions to distinct phases of the risky decision-making process.

Which came first: Cannabis use or deficits in impulse control?

Impulse control deficits are often found to co-occur with substance use disorders (SUDs). On the one hand, it is well known that chronic intake of drugs of abuse remodels the brain with significant consequences for a range of cognitive behaviors. On the other hand, individual variation in impulse control may contribute to differences in susceptibility to SUDs. Both of these relationships have been described, thus leading to a "chicken or the egg" debate which remains to be fully resolved. Does impulsivity precede drug use or does it manifest as a function of problematic drug usage? The link between impulsivity and SUDs has been most strongly established for cocaine and alcohol use disorders using both preclinical models and clinical data. Much less is known about the potential link between impulsivity and cannabis use disorder (CUD) or the directionality of this relationship. The initiation of cannabis use occurs most often during adolescence prior to the brain's maturation, which is recognized as a critical period of development. The long-term effects of chronic cannabis use on the brain and behavior have started to be explored. In this review we will summarize these observations, especially as they pertain to the relationship between impulsivity and CUD, from both a psychological and biological perspective. We will discuss impulsivity as a multi-dimensional construct and attempt to reconcile the results obtained across modalities. Finally, we will discuss possible avenues for future research with emerging longitudinal data.

Reconsidering the Border between the Visual and Posterior Parietal Cortex of Mice

The posterior parietal cortex (PPC) contributes to multisensory and sensory-motor integration, as well as spatial navigation. Based on primate studies, the PPC is composed of several subdivisions with differing connection patterns, including areas that exhibit retinotopy. In mice the composition of the PPC is still under debate. We propose a revised anatomical delineation in which we classify the higher order visual areas rostrolateral area (RL), anteromedial area (AM), and Medio-Medial-Anterior cortex (MMA) as subregions of the mouse PPC. Retrograde and anterograde tracing revealed connectivity, characteristic for primate PPC, with sensory, retrosplenial, orbitofrontal, cingulate and motor cortex, as well as with several thalamic nuclei and the superior colliculus in the mouse. Regarding cortical input, RL receives major input from the somatosensory barrel field, while AM receives more input from the trunk, whereas MMA receives strong inputs from retrosplenial, cingulate, and orbitofrontal cortices. These input differences suggest that each posterior PPC subregion may have a distinct function. Summarized, we put forward a refined cortical map, including a mouse PPC that contains at least 6 subregions, RL, AM, MMA and PtP, MPta, LPta/A. These anatomical results set the stage for a more detailed understanding about the role that the PPC and its subdivisions play in multisensory integration-based behavior in mice.

Evaluation of cognitive effort in rats is not critically dependent on ventrolateral orbitofrontal cortex

Organisms must frequently evaluate the amount of effort to invest in pursuing future rewards. Despite explicit awareness of the potential benefits of cognitive work, individuals vary in their willingness to attempt cognitively demanding tasks, regardless of intellectual ability. Such differences may suggest that the degree to which cognitive effort degrades perceived outcome value is a subjective, rather than objective, process, similar to risk and delay discounting. Although numerous studies suggest the orbitofrontal cortex (OFC) is important for allowing subjective value estimates to be updated and/or used in cost/benefit decision-making, the causal role of the OFC in valuations of mental effort has received scant investigation. We therefore trained 24 female Long-Evans rats on the rodent cognitive effort task (rCET) and assessed performance following temporary bilateral inactivation of the ventrolateral OFC (vlOFC). In the rCET, rats decide at trial outset whether to perform an easy or hard attentional challenge, namely to localize a brief visual stimulus to one of five possible locations. The difficulty of the challenge is determined by the stimulus duration (1.0 vs. 0.2s for easy vs. hard trials respectively), and success on hard trials results in double the sugar pellet rewards. Somewhat surprisingly, inactivations of the vlOFC did not affect rats' willingness or ability to exert cognitive effort for larger rewards, despite increasing omissions and motor impulsivity on-task. When considered with previous work, it appears the vlOFC plays a minimal role in cognitive effort allocation specifically, and in valuations of effort more generally.

Prefrontal regulation of behavioural control: Evidence from learning theory and translational approaches in rodents

Everyday activities require adaptive decision-making and control over our actions to achieve our goals. Sub-regions within the cortex are widely reported to regulate these choices. Here we review rodent studies from two disparate fields of instrumental action control - goal-directed and habitual responding, and impulsive and compulsive behaviour. Our aim was to compare findings across the spectrum, from precision associative learning to translational studies of action control. The evidence suggests that each cortical sub-region performs different roles depending on task requirements and, within tasks, clear dissociations exist between regions. Rather than synthesizing a single role or function for a given region, we should consider regions to be capable of many different functions. Further investigation of cortico-cortical connections and the pattern of input and output circuitry within each region may be needed to identify unique process-specific pathways. Despite differences in the scope and purpose of these two fields, integrating evidence across tasks provides a broader context for testing hypotheses about the role of cortical regions in adaptive actions and decision-making.

Pair housing, but not using a controlled reinforcer frequency procedure, attenuates the modulatory effect of probability presentation order on amphetamine-induced changes in risky choice

Probability discounting is often measured with independent schedules. Independent schedules have several limitations, such as confounding preference for one alternative with frequency of reward presentation and generating ceiling/floor effects at certain probabilities. To address this potential caveat, a controlled reinforcer frequency schedule can be used, in which the manipulandum that leads to reinforcement is pseudo-randomly determined before each trial. This schedule ensures subjects receive equal presentations of the small and large magnitude reinforcers across each block of trials. A total of 24 pair-housed and 11 individually housed female Sprague Dawley rats were tested in a controlled reinforcer frequency procedure. For half of the rats, the odds against (OA) receiving the large magnitude reinforcer increased across the session (ascending schedule); the OA decreased across the session for half of the rats (descending schedule). Following training, rats received treatments of amphetamine (AMPH; 0, 0.25, 0.5, 1.0 mg/kg; s.c.). For pair-housed rats, AMPH (0.5 mg/kg) increased risky choice, regardless of probability presentation order, whereas a higher dose of AMPH (1.0 mg/kg) decreased discriminability of reinforcer magnitude for rats trained on the descending schedule only. For individually housed rats, probability presentation order modulated the effects of AMPH on probability discounting, as AMPH (0.25 and 0.5 mg/kg) increased risky choice in rats trained on the ascending schedule but not on the descending schedule. These results show that pair-housing animals, but not using a controlled reinforcer frequency procedure, attenuates the modulatory effects of probability presentation order on drug effects on risky choice.

Involvement of the rodent prelimbic and medial orbitofrontal cortices in goal-directed action: A brief review

Goal-directed action refers to selecting behaviors based on the expectation that they will be reinforced with desirable outcomes. It is typically conceptualized as opposing habit-based behaviors, which are instead supported by stimulus-response associations and insensitive to consequences. The prelimbic prefrontal cortex (PL) is positioned along the medial wall of the rodent prefrontal cortex. It is indispensable for action-outcome-driven (goal-directed) behavior, consolidating action-outcome relationships and linking contextual information with instrumental behavior. In this brief review, we will discuss the growing list of molecular factors involved in PL function. Ventral to the PL is the medial orbitofrontal cortex (mOFC). We will also summarize emerging evidence from rodents (complementing existing literature describing humans) that it too is involved in action-outcome conditioning. We describe experiments using procedures that quantify responding based on reward value, the likelihood of reinforcement, or effort requirements, touching also on experiments assessing food consumption more generally. We synthesize these findings with the argument that the mOFC is essential to goal-directed action when outcome value information is not immediately observable and must be recalled and inferred.

Selective impact of lateral orbitofrontal cortex inactivation on reinstatement of alcohol seeking in male Long-Evans rats

The orbitofrontal cortex (OFC) plays a fundamental role in motivated behavior and decision-making. In humans, OFC structure and function is significantly disrupted in drug using and dependent individuals, including those exhibiting chronic alcohol use and alcoholism. In animal models, the OFC has been shown to significantly influence the seeking of non-alcohol drugs of abuse. However direct investigations of the OFC during alcohol seeking and use have been more limited. In the studies reported here, we inactivated lateral (lOFC) or medial OFC (mOFC) subregions in rats during multiple stages of alcohol seeking. After one month of intermittent access to homecage 20% ethanol (EtOH), rats were trained to self-administer EtOH under an FR3 schedule and implanted with cannulae directed to lOFC or mOFC. We inactivated OFC subregions with baclofen/muscimol during EtOH self-administration, extinction, cue-induced reinstatement, and progressive ratio testing to broadly characterize the influence of these subregions on alcohol seeking. There were no significant effects of mOFC or lOFC inactivation during FR3 self-administration, extinction, or progressive ratio self-administration. However, lOFC, and not mOFC, inactivation significantly decreased cue-induced reinstatement of EtOH seeking. These findings contribute new information to the specific impact of OFC manipulation on operant alcohol seeking, support previous studies investigating the role of OFC in seeking and consumption of alcohol and other drugs of abuse, and indicate a specific role for lOFC vs. mOFC in reinstatement.

Common neurobiological and psychological underpinnings of gambling and substance-use disorders

Both psychological and neurobiological studies in gambling disorder have increased in the past 10-15 years. This review examines the current state of the literature, with a focus on recent magnetic resonance imaging (MRI) studies in gambling disorder. The review compares and contrasts findings across gambling and substance-use disorders. Additionally, features with arguably particular relevance to gambling disorder (e.g., "near-miss" processing) are described, as well as their relationship to choice behaviors. More broadly, the review informs on how these studies advance our understanding of brain-behavior relationships relating to decision-making and key features of addictive disorders.

Using a dependent schedule to measure risky choice in male rats: Effects of d-amphetamine, methylphenidate, and methamphetamine

Risky choice is the tendency to choose a large, uncertain reward over a small, certain reward, and is typically measured with probability discounting, in which the probability of obtaining the large reinforcer decreases across blocks of trials. One caveat to traditional procedures is that independent schedules are used, in which subjects can show exclusive preference for one alternative relative to the other. For example, some rats show exclusive preference for the small, certain reinforcer as soon as delivery of the large reinforcer becomes probabilistic. Therefore, determining if a drug increases risk aversion (i.e., decreases responding for the probabilistic alternative) is difficult (due to floor effects). The overall goal of this experiment was to use a concurrent-chains procedure that incorporated a dependent schedule during the initial link, thus preventing animals from showing exclusive preference for one alternative relative to the other. To determine how pharmacological manipulations alter performance in this task, male Sprague-Dawley rats (n = 8) received injections of amphetamine (0, 0.25, 0.5, 1.0 mg/kg), methylphenidate (0, 0.3, 1.0, 3.0 mg/kg), and methamphetamine (0, 0.5, 1.0, 2.0 mg/kg). Amphetamine (0.25 mg/kg) and methylphenidate (3.0 mg/kg) selectively increased risky choice, whereas higher doses of amphetamine (0.5 and 1.0 kg/mg) and each dose of methamphetamine impaired stimulus control (i.e., flattened the discounting function). These results show that dependent schedules can be used to measure risk-taking behavior and that psychostimulants promote suboptimal choice when this schedule is used. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The Abused Inhalant Toluene Impairs Medial Prefrontal Cortex Activity and Risk/Reward Decision-Making during a Probabilistic Discounting Task

Inhalant (e.g., toluene) misuse is linked to behavioral and cognitive deficits in humans, yet preclinical studies of the effect of inhalants on higher-order cognition are limited. We addressed this gap in the literature by examining the effect of toluene vapor exposure on risk/reward decision-making in male and female Sprague-Dawley rats using a probabilistic discounting task. In this task, rodents chose a risky/large reward or a safe/small reward, with the odds of risky reinforcement descending or ascending throughout the test session. We observed a dose-dependent, sex-independent deficit in behavioral flexibility during probabilistic discounting caused by acute toluene exposure. Rats exposed to toluene vapor during adolescence and tested as adults performed comparably to air-treated controls and were susceptible to the effects of an acute toluene challenge. These behavioral flexibility deficits observed suggests dysfunctional medial prefrontal cortex (mPFC) activity. To address this hypothesis, we virally expressed the genetically encoded calcium sensor GCaMP6f in glutamatergic mPFC neurons and monitored calcium transients in real-time using in vivo fiber photometry. mPFC activity peaked before either lever press during free-choice trials in toluene- and air-treated animals. During forced-choice trials, GCaMP6f transients shifted from pre-risky to pre-safe choice, an effect mitigated by acute toluene exposure. mPFC activity decreased during rewarded trials, with larger decreases following risky/large wins compared with safe/small wins. Toluene-treated animals also had decreased mPFC activity during rewarded trials, but there was no distinction between risky/large wins and safe/small wins. These results provide physiological evidence for mPFC-dependent behavioral deficits caused by toluene.SIGNIFICANCE STATEMENT Inhalants (e.g., toluene) are an understudied class of drugs of abuse that cause devastating behavioral and cognitive deficits in humans. Understanding the neurobiological interactions of toluene vapor using animal models is important for developing effective treatment strategies for inhalant addicts. Here we find that toluene vapor reduces behavioral flexibility in rodents making risk/reward-based decisions. The medial prefrontal cortex (mPFC) drives behavioral flexibility during this type of decision-making and we show that toluene reduces the ability of mPFC neurons to track optimal choices as reward probabilities change. Toluene also reduces these neurons' ability to distinguish between small and large rewards. A combination of these factors likely leads to the impaired performance in probabilistic discounting following acute toluene exposure.

Differential effects of d-amphetamine and atomoxetine on risk-based decision making of Lewis and Fischer 344 rats

Individuals with attention-deficit/hyperactivity disorder tend to make risker choices during probabilistic-discounting procedures. Thus, how common attention-deficit/hyperactivity disorder medications affect probabilistic discounting is of interest. In general, d-amphetamine increases risk-taking while atomoxetine has produced mixed effects in rats. Results from previous studies may result from genetic factors. Lewis and F344 rats have neurochemical differences that may be relevant to probabilistic discounting and how drugs affect such behavior. In this study, we evaluated dose-dependent effects of d-amphetamine and atomoxetine on probabilistic discounting of Lewis and F344. Male Lewis and F344 chose between one food pellet delivered 100% of the time and three food pellets delivered following decreasing probabilities of delivery (i.e. 100%, 66.7%, 33.3%, 16.5%, and 8.25%). Saline, d-amphetamine (0.1-1.8 mg/kg), and atomoxetine (0.1-7.8 mg/kg) were tested acutely. Lewis and F344 did not differ in choice at baseline. d-Amphetamine increased risky choice for both rat strains at low-to-moderate doses, although it did so at a lower dose (0.1 and 0.3 mg/kg) for F344 as compared to Lewis (0.3 mg/kg only). At high doses (1.0 and 1.8 mg/kg), d-amphetamine disrupted choice, increased frequencies of omitted trials, and reduced reinforcer sensitivity. Although atomoxetine increased frequencies of omitted trials at high doses (5.6 and 7.8 mg/kg), it had no effect on probabilistic discounting for either rat strain. Although Lewis and F344 differ in various types of impulsivity (i.e. motor, choice), with Lewis being the more impulsive of the two, the present results suggest that Lewis and F344 do not differ in risk-based decision-making. Effects of d-amphetamine on probabilistic discounting may be biology-dependent and differ from effects of atomoxetine.

The use of laser capture microdissection to identify specific pathways and mechanisms involved in impulsive choice in rats

Background

Microinjections, lesions, viral-mediated gene transfer, or designer receptors exclusively activated by designer drugs (DREADDs) can identify brain signaling pathways and their pharmacology in research animals. Genetically modified animals are used for more precise assessment of neural circuits. However, only a few of the gene-based pathway modifications are available for use in outbred rat strains.

New method

Behaviorally characterized Sprague-Dawley rats undergo tract tracing through microinjection of fluorospheres, followed by laser capture microdissection (LCM) and qPCR for detecting mRNA of pathway-associated gene products. Correlations between mRNA expression and behavior identify specific involvement of pharmacologically relevant molecules within cells of interest. Here, we examined this methodology in an impulsive choice paradigm and targeted projections from the orbital and medial prefrontal cortex.

Results

In this proof of concept study, we demonstrate relationships between measures of impulsive choice with distinct neurotransmitter receptor expression in cell populations from four different signaling pathways.

Comparisons with existing methods

Combining behavior, tract tracing, LCM, and gene expression profiling provides more cellular selectivity than localized lesions and DREADDs, and greater pharmacological specificity than microinjections and viral-mediated gene transfer due to targeting identified neurons. Furthermore, the assessment of inter-individual pathways provides insight into the complex nature of underlying mechanisms involved in typical and atypical behavior.

Conclusions

The novel combination of behavior, tract tracing, LCM, and single gene or potential whole genome transcriptome analysis allows for a more targeted understanding of the interconnection of neural circuitry with behavior, and holds promise to identify more specific drug targets that are relevant to behavioral phenotypes.

Optogenetic and chemogenetic approaches to manipulate attention, impulsivity and behavioural flexibility in rodents

Studies manipulating neural activity acutely with optogenetic or chemogenetic intervention in behaving rodents have increased considerably in recent years. More often, these circuit-level neural manipulations are tested within an existing framework of behavioural testing that strives to model complex executive functions or symptomologies relevant to multidimensional psychiatric disorders in humans, such as attentional control deficits, impulsivity or behavioural (in)flexibility. This methods perspective argues in favour of carefully implementing these acute circuit-based approaches to better understand and model cognitive symptomologies or their similar isomorphic animal behaviours, which often arise and persist in overlapping brain circuitries. First, we offer some practical considerations for combining long-term, behavioural paradigms with optogenetic or chemogenetic interventions. Next, we examine how cell-type or projection-specific manipulations to the ascending neuromodulatory systems, local brain region or descending cortical glutamatergic projections influence aspects of cognitive control. For this, we primarily focus on the influence exerted on attentional and motor impulsivity performance in the (3-choice or) 5-choice serial reaction time task, and impulsive, risky or inflexible choice biases during alternative preference, reward discounting or reversal learning tasks.

Toward an animal model of borderline personality disorder

BACKGROUND

Borderline personality disorder (BPD) is a pervasive psychiatric disorder characterized by emotion dysregulation, impulsivity, impaired self-perceptions, and interpersonal relationships and currently affects 1-3% of the US population as reported by Torgersen et al. (Arch Gen Psychiatry 58:590-596, Torgersen et al. 2001), Lenzenweger et al. (Biol Psychiatry 62:553-564, Lenzenweger et al. 2007), and Tomko et al. (J Personal Disord 28:734-750, Tomko et al. 2014). One major obstacle to our understanding of the neural underpinnings of BPD is a lack of valid animal models that translate the key known features of the disorder to a system that is amenable to study.

OBJECTIVE

To summarize the etiology, major symptoms, and symptom triggers of BPD and then propose a blueprint for building an animal model of BPD by choosing key components of the disorder that can be implemented in rodents.

RESULTS

We identify the role of early life stress and subsequent mild stress in adulthood as contributing etiological factors and the potential use of altered communication between frontal cortices and the amygdala in extinction and habituation, increased impulsivity, dysregulation of the hypothalamic pituitary axis (HPA), and increased neuroinflammation as biological markers of BPD. Building upon these features of BPD, we propose a two-hit animal model that uses maternal abandonment to alter maturation of the HPA axis and mild secondary adult stress to evoke behavioral symptoms such as increased impulsivity and impaired extinction, habituation, and social interactions.

CONCLUSION

Through exploration of the etiology, symptom presentation, and altered neurological function, we propose an animal model of BPD. We believe that a number of existing animal paradigms that model other mental health disorders should be combined in a unique way to reflect the etiology, symptom presentation, and altered neurological function that is evident in BPD. These model, when compared with available human data, will inform research and treatment in humans for better understanding of systems from the micro-molecular level to more global physiology underlying BPD.

Distinct cortical-amygdala projections drive reward value encoding and retrieval

The value of an anticipated rewarding event is a crucial component of the decision to engage in its pursuit. But little is known of the networks responsible for encoding and retrieving this value. By using biosensors and pharmacological manipulations, we found that basolateral amygdala (BLA) glutamatergic activity tracks and mediates encoding and retrieval of the state-dependent incentive value of a palatable food reward. Projection-specific, bidirectional chemogenetic and optogenetic manipulations revealed that the orbitofrontal cortex (OFC) supports the BLA in these processes. Critically, the function of ventrolateral and medial OFC→BLA projections is doubly dissociable. Whereas lateral OFC→BLA projections are necessary and sufficient for encoding of the positive value of a reward, medial OFC→BLA projections are necessary and sufficient for retrieving this value from memory. These data reveal a new circuit for adaptive reward valuation and pursuit and provide insight into the dysfunction in these processes that characterizes myriad psychiatric diseases.

Impulsive Action and Impulsive Choice Are Differentially Associated With Gene Expression Variations of the GABAA Receptor Alfa 1 Subunit and the CB1 Receptor in the Lateral and Medial Orbitofrontal Cortices

The orbitofrontal cortex (OFC) is a key brain region for decision-making, action control and impulsivity. Quite notably, previous research has identified a double dissociation regarding the role of this cortical territory in impulsive choice. While medial orbitofrontal lesions increase preference for a large but delayed reward, lateral orbitofrontal lesions have the opposite effect. However, there are no data regarding this anatomical dissociation in impulsive action. The neurochemical basis of impulsivity is still being elucidated, however, in recent years a role for the endocannabinoids and the related glutamatergic and GABAergic neurotransmitter systems has been suggested. Here, we submitted male Wistar rats to a delay-discounting task (DDT) or a two-choice serial reaction time task (2-CSRTT) and classified them as high impulsive or low impulsive in either task using cluster analysis. We then examined the gene expression of several elements of the endocannabinoid system or different subunits of certain glutamatergic or GABAergic ionotropic receptors (AMPA, NMDA, or GABA_A) in the lateral or medial divisions of their orbitofrontal cortices. Our results confirm, at the gene expression level, the dissociation in the participation of the medial, and lateral divisions of the orbitofrontal cortex in impulsivity. While in the 2-CSRTT (inhibitory control) we found that high impulsive animals exhibited lower gene expression levels of the α1 GABA_A receptor subunit in the lateral OFC, no such differences were evident in the medial OFC. When we analyzed DDT performance, we found that high impulsive animals displayed lower levels of CB₁ gene expression in the medial but not in the lateral OFC. We propose that GABAergic dynamics in the lateral OFC might contribute to the inhibitory control mechanisms that are altered in impulsive behavior while endocannabinoid receptor gene transcription in the medial OFC may subserve the delay-discounting processes that participate in certain types of impulsiveness.

Brain and Cognition for Addiction Medicine: From Prevention to Recovery Neural Substrates for Treatment of Psychostimulant-Induced Cognitive Deficits

Addiction to psychostimulants like cocaine, methamphetamine, and nicotine poses a continuing medical and social challenge both in the United States and all over the world. Despite a desire to quit drug use, return to drug use after a period of abstinence is a common problem among individuals dependent on psychostimulants. Recovery for psychostimulant drug-dependent individuals is particularly challenging because psychostimulant drugs induce significant changes in brain regions associated with cognitive functions leading to cognitive deficits. These cognitive deficits include impairments in learning/memory, poor decision making, and impaired control of behavioral output. Importantly, these drug-induced cognitive deficits often impact adherence to addiction treatment programs and predispose abstinent addicts to drug use relapse. Additionally, these cognitive deficits impact effective social and professional rehabilitation of abstinent addicts. The goal of this paper is to review neural substrates based on animal studies that could be pharmacologically targeted to reverse psychostimulant-induced cognitive deficits such as impulsivity and impairment in learning and memory. Further, the review will discuss neural substrates that could be used to facilitate extinction learning and thus reduce emotional and behavioral responses to drug-associated cues. Moreover, the review will discuss some non-pharmacological approaches that could be used either alone or in combination with pharmacological compounds to treat the above-mentioned cognitive deficits. Psychostimulant addiction treatment, which includes treatment for cognitive deficits, will help promote abstinence and allow for better rehabilitation and integration of abstinent individuals into society.

The role of the orbitofrontal cortex in alcohol use, abuse, and dependence

One of the major functions of the orbitofrontal cortex (OFC) is to promote flexible motivated behavior. It is no surprise, therefore, that recent work has demonstrated a prominent impact of chronic drug use on the OFC and a potential role for OFC disruption in drug abuse and addiction. Among drugs of abuse, the use of alcohol is particularly salient with respect to OFC function. Although a number of studies in humans have implicated OFC dysregulation in alcohol use disorders, animal models investigating the association between OFC and alcohol use are only beginning to be developed, and there is still a great deal to be revealed. The goal of this review is to consider what is currently known regarding the role of the OFC in alcohol use and dependence. I will first provide a brief, general overview of current views of OFC function and its contributions to drug seeking and addiction. I will then discuss research to date related to the OFC and alcohol use, both in human clinical populations and in non-human models. Finally I will consider issues and strategies to guide future study that may identify this brain region as a key player in the transition from moderated to problematic alcohol use and dependence.

Effects of GluN2B-selective antagonists on delay and probability discounting in male rats: Modulation by delay/probability presentation order

The contribution of the GluN2B subunit of the NMDA receptor to impulsivity has recently been examined. Ro 63-1908, a highly selective antagonist for the GluN2B, decreases impulsive choice. Because the order in which delays are presented modulates drug effects in discounting procedures, one goal of the current study was to determine the effects of Ro 63-1908 in delay discounting procedures in which the delays to obtaining the large reinforcer either increase or decrease across the session. We also determined if Ro 63-1908 differentially alters risky choice in probability discounting procedures that use ascending/descending schedules. Male rats were trained in either delay (n = 24) or probability (n = 24) discounting in which the delay to/odds against reinforcement were presented in either ascending or descending order (n = 12 each schedule). Following training, rats received the GluN2B antagonists Ro 63-1908 (0-1.0 mg/kg) and CP-101,606 (0-3.0 mg/kg). In delay discounting, Ro 63-1908 (1.0 mg/kg), but not CP-101,606, decreased choice for the large reinforcer, but only when the delays decreased across the session. In probability discounting, Ro 63-1908 (0.3 mg/kg)/CP-101,606 (1.0 mg/kg) increased choice for the large reinforcer when the probability of obtaining this alternative decreased across the session, but Ro 63-1908 (1.0 mg/kg)/CP-101,606 (3.0 mg/kg) decreased choice when the probabilities increased. These results show that the GluN2B is a mediator of impulsive/risky choice, but the effects of GluN2B antagonists are dependent on the order in which delays/probabilities are presented. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Functional Heterogeneity within Rat Orbitofrontal Cortex in Reward Learning and Decision Making

Rat orbitofrontal cortex (OFC) is located in the dorsal bank of the rhinal sulcus, and is divided into the medial orbital area, ventral orbital area, ventrolateral orbital area, lateral orbital area, dorsolateral orbital area, and agranular insular areas. Over the past 20 years, there has been a marked increase in the number of publications focused on the functions of rat OFC. While collectively this extensive body of work has provided great insight into the functions of OFC, leading to theoretical and computational models of its functions, one issue that has emerged relates to what is defined as OFC because targeting of this region can be quite variable between studies of appetitive behavior, even within the same species. Also apparent is that there is an oversampling and undersampling of certain subregions of rat OFC for study, and this will be demonstrated here. The intent of the Viewpoint is to summarize studies in rat OFC, given the diversity of what groups refer to as "OFC," and to integrate these with the findings of recent anatomical studies. The primary aim is to help discern functions in reward learning and decision-making, clearing the course for future empirical work.

Deciphering Decision Making: Variation in Animal Models of Effort- and Uncertainty-Based Choice Reveals Distinct Neural Circuitries Underlying Core Cognitive Processes

Maladaptive decision-making is increasingly recognized to play a significant role in numerous psychiatric disorders, such that therapeutics capable of ameliorating core impairments in judgment may be beneficial in a range of patient populations. The field of "decision neuroscience" is therefore in its ascendancy, with researchers from diverse fields bringing their expertise to bear on this complex and fascinating problem. In addition to the advances in neuroimaging and computational neuroscience that contribute enormously to this area, an increase in the complexity and sophistication of behavioral paradigms designed for nonhuman laboratory animals has also had a significant impact on researchers' ability to test the causal nature of hypotheses pertaining to the neural circuitry underlying the choice process. Multiple such decision-making assays have been developed to investigate the neural and neurochemical bases of different types of cost/benefit decisions. However, what may seem like relatively trivial variation in behavioral methodologies can actually result in recruitment of distinct cognitive mechanisms, and alter the neurobiological processes that regulate choice. Here we focus on two areas of particular interest, namely, decisions that involve an assessment of uncertainty or effort, and compare some of the most prominent behavioral paradigms that have been used to investigate these processes in laboratory rodents. We illustrate how an appreciation of the diversity in the nature of these tasks can lead to important insights into the circumstances under which different neural regions make critical contributions to decision making.

Complementary contributions of basolateral amygdala and orbitofrontal cortex to value learning under uncertainty

We make choices based on the values of expected outcomes, informed by previous experience in similar settings. When the outcomes of our decisions consistently violate expectations, new learning is needed to maximize rewards. Yet not every surprising event indicates a meaningful change in the environment. Even when conditions are stable overall, outcomes of a single experience can still be unpredictable due to small fluctuations (i.e., expected uncertainty) in reward or costs. In the present work, we investigate causal contributions of the basolateral amygdala (BLA) and orbitofrontal cortex (OFC) in rats to learning under expected outcome uncertainty in a novel delay-based task that incorporates both predictable fluctuations and directional shifts in outcome values. We demonstrate that OFC is required to accurately represent the distribution of wait times to stabilize choice preferences despite trial-by-trial fluctuations in outcomes, whereas BLA is necessary for the facilitation of learning in response to surprising events.

DOI:http://dx.doi.org/10.7554/eLife.27483.001

Nobody likes waiting – we opt for online shopping to avoid standing in lines, grow impatient in traffic, and often prefer restaurants that serve food quickly. When making decisions, humans and other animals try to maximize the benefits by weighing up the costs and rewards associated with a situation. Many regions in the brain help us choose the best options based on quality and size of rewards, and required waiting times. Even before we make decisions, the activity in these brain regions predicts what we will choose.

Sometimes, however, unexpected changes can lead to longer waiting times and our preferences suddenly become less desirable. The brain can detect such changes by comparing the outcomes we anticipate to those we experience. When the outcomes are surprising, specific areas in the brain such as the amygdala and the orbitofrontal cortex help us learn to make better choices. However, as surprising events can occur purely by chance, we need to be able to ignore irrelevant surprises and only learn from meaningful ones. Until now, it was not clear whether the amygdala and orbitofrontal cortex play specific roles in successfully learning under such conditions.

Stolyarova and Izquierdo trained rats to select between two images and rewarded them with sugar pellets after different delays. If rats chose one of these images they received the rewards after a predictable delay that was about 10 seconds, while choosing the other one produced variable delays – sometimes the time intervals were either very short or very long. Then, the waiting times for one of the alternatives changed unexpectedly. Rats with healthy brains quickly learned to choose the option with the shorter waiting time.

Stolyarova and Izquierdo repeated the experiments with rats that had damage in a part of the amygdala. These rats learned more slowly, particularly when the variable option changed for the better. Rats with damage to the orbitofrontal cortex failed to learn at all. Stolyarova and Izquierdo then examined the rats’ behavior during delays. Rats with damage to the orbitofrontal cortex could not distinguish between meaningful and irrelevant surprises and always looked for the food pellet (i.e. anticipated a reward) at the average delay interval.

These findings highlight two brain regions that help us distinguish meaningful surprises from irrelevant ones. A next step will be to examine how the amygdala and orbitofrontal cortex interact during learning and see if changes to the activity of these brain regions may affect responses. Advanced methods to non-invasively manipulate brain activity in humans may help people who find it hard to cope with changes; or individuals suffering from substance use disorders, who often struggle to give up drugs that provide them immediate and predictable rewards.

DOI:http://dx.doi.org/10.7554/eLife.27483.002

Learning From One's Mistakes: A Dual Role for the Rostromedial Tegmental Nucleus in the Encoding and Expression of Punished Reward Seeking

BACKGROUND

Psychiatric disorders such as addiction and mania are marked by persistent reward seeking despite highly negative or aversive outcomes, but the neural mechanisms underlying this aberrant decision making are unknown. The recently identified rostromedial tegmental nucleus (RMTg) encodes a wide variety of aversive stimuli and sends robust inhibitory projections to midbrain dopamine neurons, leading to the hypothesis that the RMTg provides a brake to reward signaling in response to aversive costs.

METHODS

To test the role of the RMTg in punished reward seeking, adult male Sprague Dawley rats were tested in several cost-benefit decision tasks after excitotoxic lesions of the RMTg or temporally specific optogenetic inhibition of RMTg efferents in the ventral tegmental area.

RESULTS

RMTg lesions drastically impaired the ability of foot shock to suppress operant responding for food. Optogenetic inhibition showed that this resistance to punishment was due in part to RMTg activity at the precise moment of shock delivery and was mediated by projections to the ventral tegmental area, which is consistent with an aversive "teaching signal" role for the RMTg during encoding of the aversive event. We observed a similar resistance to punishment when the RMTg was selectively inhibited immediately prior to the operant lever press, which is consistent with a second distinct role for the RMTg during action selection. These effects were not attributable to RMTg effects on learning rate, locomotion, shock sensitivity, or perseveration.

CONCLUSIONS

The RMTg has two strong and dissociable roles during both encoding and recall of aversive consequences of behavior.

Fractionating impulsivity: neuropsychiatric implications

The ability to make decisions and act quickly without hesitation can be advantageous in many settings. However, when persistently expressed, impulsive decisions and actions are considered risky, maladaptive and symptomatic of such diverse brain disorders as attention-deficit hyperactivity disorder, drug addiction and affective disorders. Over the past decade, rapid progress has been made in the identification of discrete neural networks that underlie different forms of impulsivity - from impaired response inhibition and risky decision making to a profound intolerance of delayed rewards. Herein, we review what is currently known about the neural and psychological mechanisms of impulsivity, and discuss the relevance and application of these new insights to various neuropsychiatric disorders.

Prefrontal Dopamine D1 and D2 Receptors Regulate Dissociable Aspects of Decision Making via Distinct Ventral Striatal and Amygdalar Circuits

Mesocortical dopamine (DA) regulates a variety of cognitive functions via actions on D₁ and/or D₂ receptors. For example, risk/reward decision making is modulated differentially by these two receptors within the prefrontal cortex (PFC), with D₂ receptors enabling flexible decision making and D₁ receptors promoting persistence in choice biases. However, it is unclear how DA mediates opposing patterns of behavior by acting on different receptors within the same terminal region. We explored the possibility that DA may act on separate networks of PFC neurons that are modulated by D₁ or D₂ receptors and in turn interface with divergent downstream structures such as the basolateral amygdala (BLA) or nucleus accumbens (NAc). Decision making was assessed using a probabilistic discounting task in which well trained male rats chose between small/certain or large/risky rewards, with the odds of obtaining the larger reward changing systematically within a session. Selective disruption of D₁ or D₂ modulation of separate PFC output pathways was achieved using unilateral intra-PFC infusions of DA antagonists combined with contralateral inactivation of the BLA or NAc. Disrupting D₂ (but not D₁) modulation of PFC→BLA circuitry impaired adjustments in decision biases in response to changes in reward probabilities. In contrast, disrupting D₁ modulation of PFC→NAc networks reduced risky choice, attenuating reward sensitivity and increasing sensitivity to reward omissions. These findings reveal that mesocortical DA can facilitate dissociable components of reward seeking and action selection by acting on different functional networks of PFC neurons that can be distinguished by the subcortical projection targets with which they interface.SIGNIFICANCE STATEMENT Prefrontal cortical dopamine regulates a variety of executive functions governed by the frontal lobes via actions on D₁ and D₂ receptors. These receptors can in some instances mediate different patterns of behavior, but the mechanisms underlying these dissociable actions are unclear. Using a selective disconnection approach, we reveal that D₁ and D₂ receptors can facilitate diverse aspects of decision making by acting on separate networks of prefrontal neurons that interface with distinct striatal or amygdalar targets. These findings reveal an additional level of complexity in how mesocortical DA regulates different forms of cognition via actions on different receptors, highlighting how it may act upon distinct cortical microcircuits to drive different patterns of behavior.

Gambling disorder: an integrative review of animal and human studies

Gambling disorder (GD), previously called pathological gambling and classified as an impulse control disorder in DSM-III and DSM-IV, has recently been reclassified as an addictive disorder in the DSM-5. It is widely recognized as an important public health problem associated with substantial personal and social costs, high rates of psychiatric comorbidity, poor physical health, and elevated suicide rates. A number of risk factors have been identified, including some genetic polymorphisms. Animal models have been developed in order to study the underlying neural basis of GD. Here, we discuss recent advances in our understanding of the risk factors, disease course, and pathophysiology. A focus on a phenotype-based dissection of the disorder is included in which known neural correlates from animal and human studies are reviewed. Finally, current treatment approaches are discussed, as well as future directions for GD research.

Dissociable effects of systemic and orbitofrontal administration of adrenoceptor antagonists on yohimbine-induced motor impulsivity

The α₂-adrenoceptor antagonist, yohimbine, is commonly used as a pharmacological stressor. Its behavioural effects are typically attributed to elevated noradrenaline release via blockade of central, inhibitory autoreceptors. We have previously reported that yohimbine increases motor impulsivity in rats on the five-choice serial reaction time task (5CSRTT), a cognitive behavioural assessment which measures motor impulsivity and visuospatial attention. Furthermore, this effect depended on cyclic adenomonophosphate (cAMP) signalling via cAMP response element binding (CREB) protein in the orbitofrontal cortex (OFC). However, the role of specific adrenoceptors in this effect is not well-characterised. We therefore investigated whether the pro-impulsive effects of systemic yohimbine could be reproduced by direct administration into the OFC, or attenuated by intra-OFC or systemic administration of prazosin and propranolol-antagonists at the α₁- and β-adrenoceptor, respectively. Male Long-Evans rats were trained on the 5CSRTT and implanted with guide cannulae aimed at the OFC. Systemically administered α₁- or β-adrenoceptor antagonists attenuated yohimbine-induced increases in premature responding. In contrast, local infusion of yohimbine into the OFC reduced such impulsive responding, while blockade of α₁- or β-adrenoceptors within the OFC had no effect on either basal or yohimbine-stimulated motor impulsivity. Direct administration of selective antagonists at the α₁-, α₂- or β-adrenoceptor into the OFC therefore produce clearly dissociable effects from systemic administration. Collectively, these data suggest that the pro-impulsivity effect of yohimbine can be modulated by adrenergic signalling in brain areas outside of the OFC, in addition to non-adrenergic signalling pathways within the OFC.

Lesions of the Orbitofrontal but Not Medial Prefrontal Cortex Affect Cognitive Judgment Bias in Rats

Neuroimaging studies in humans have recently shown that the prefrontal cortex (PFC) and orbitofrontal cortex (OFC) mediate bias in the judgment of forthcoming events. In the present study, we sought to determine whether cognitive judgment bias (CJB) is also dependent on these prefrontal regions in non-human animals. For this, we trained a cohort of rats in the ambiguous-cue interpretation (ACI) paradigm, subjected them to excitotoxic lesions in the medial PFC (mPFC) and OFC, and tested the effects of neuronal loss within these regions on CJB. Comparison of the lesions' behavioral effects in the ACI paradigm revealed that neuronal loss within the OFC but not within the mPFC influences the interpretation of ambiguous cues by animals. Our findings demonstrate the specific involvement of the OFC in CJB in rats.