51
|
Vicario-Feliciano R, Murray EA, Averbeck BB. Ventral striatum lesions do not affect reinforcement learning with deterministic outcomes on slow time scales. Behav Neurosci 2017; 131:385-91. [PMID: 28805428 DOI: 10.1037/bne0000211] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
A large body of work has implicated the ventral striatum (VS) in aspects of reinforcement learning (RL). However, less work has directly examined the effects of lesions in the VS, or other forms of inactivation, on 2-armed bandit RL tasks. We have recently found that lesions in the VS in macaque monkeys affect learning with stochastic schedules but have minimal effects with deterministic schedules. The reasons for this are not currently clear. Because our previous work used short intertrial intervals, one possibility is that the animals were using working memory to bridge stimulus-reward associations from 1 trial to the next. In the present study, we examined learning of 60 pairs of objects, in which the animals received only 1 trial per day with each pair. The large number of object pairs and the long interval (approximately 24 hr) between trials with a given pair minimized the chances that the animals could use working memory to bridge trials. We found that monkeys with VS lesions were unimpaired relative to controls, which suggests that animals with VS lesions can still learn to select rewarded objects even when they cannot make use of working memory. (PsycINFO Database Record
Collapse
Affiliation(s)
- Raquel Vicario-Feliciano
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health
| | - Elisabeth A Murray
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health
| | - Bruno B Averbeck
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health
| |
Collapse
|
52
|
Jenni NL, Larkin JD, Floresco SB. Prefrontal Dopamine D 1 and D 2 Receptors Regulate Dissociable Aspects of Decision Making via Distinct Ventral Striatal and Amygdalar Circuits. J Neurosci 2017; 37:6200-6213. [PMID: 28546312 PMCID: PMC6705698 DOI: 10.1523/jneurosci.0030-17.2017] [Citation(s) in RCA: 64] [Impact Index Per Article: 9.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2017] [Revised: 05/15/2017] [Accepted: 05/18/2017] [Indexed: 01/04/2023] Open
Abstract
Mesocortical dopamine (DA) regulates a variety of cognitive functions via actions on D1 and/or D2 receptors. For example, risk/reward decision making is modulated differentially by these two receptors within the prefrontal cortex (PFC), with D2 receptors enabling flexible decision making and D1 receptors promoting persistence in choice biases. However, it is unclear how DA mediates opposing patterns of behavior by acting on different receptors within the same terminal region. We explored the possibility that DA may act on separate networks of PFC neurons that are modulated by D1 or D2 receptors and in turn interface with divergent downstream structures such as the basolateral amygdala (BLA) or nucleus accumbens (NAc). Decision making was assessed using a probabilistic discounting task in which well trained male rats chose between small/certain or large/risky rewards, with the odds of obtaining the larger reward changing systematically within a session. Selective disruption of D1 or D2 modulation of separate PFC output pathways was achieved using unilateral intra-PFC infusions of DA antagonists combined with contralateral inactivation of the BLA or NAc. Disrupting D2 (but not D1) modulation of PFC→BLA circuitry impaired adjustments in decision biases in response to changes in reward probabilities. In contrast, disrupting D1 modulation of PFC→NAc networks reduced risky choice, attenuating reward sensitivity and increasing sensitivity to reward omissions. These findings reveal that mesocortical DA can facilitate dissociable components of reward seeking and action selection by acting on different functional networks of PFC neurons that can be distinguished by the subcortical projection targets with which they interface.SIGNIFICANCE STATEMENT Prefrontal cortical dopamine regulates a variety of executive functions governed by the frontal lobes via actions on D1 and D2 receptors. These receptors can in some instances mediate different patterns of behavior, but the mechanisms underlying these dissociable actions are unclear. Using a selective disconnection approach, we reveal that D1 and D2 receptors can facilitate diverse aspects of decision making by acting on separate networks of prefrontal neurons that interface with distinct striatal or amygdalar targets. These findings reveal an additional level of complexity in how mesocortical DA regulates different forms of cognition via actions on different receptors, highlighting how it may act upon distinct cortical microcircuits to drive different patterns of behavior.
Collapse
Affiliation(s)
- Nicole L Jenni
- Department of Psychology and Djavad Mowafaghian Centre for Brain Health, University of British Columbia, Vancouver, British Columbia V6T 1Z4, Canada
| | - Joshua D Larkin
- Department of Psychology and Djavad Mowafaghian Centre for Brain Health, University of British Columbia, Vancouver, British Columbia V6T 1Z4, Canada
| | - Stan B Floresco
- Department of Psychology and Djavad Mowafaghian Centre for Brain Health, University of British Columbia, Vancouver, British Columbia V6T 1Z4, Canada
| |
Collapse
|
53
|
Abstract
Learning the values of actions versus stimuli may depend on separable neural circuits. In the current study, we evaluated the performance of rhesus macaques with ventral striatum (VS) lesions on a two-arm bandit task that had randomly interleaved blocks of stimulus-based and action-based reinforcement learning (RL). Compared with controls, monkeys with VS lesions had deficits in learning to select rewarding images but not rewarding actions. We used a RL model to quantify learning and choice consistency and found that, in stimulus-based RL, the VS lesion monkeys were more influenced by negative feedback and had lower choice consistency than controls. Using a Bayesian model to parse the groups' learning strategies, we also found that VS lesion monkeys defaulted to an action-based choice strategy. Therefore, the VS is involved specifically in learning the value of stimuli, not actions.SIGNIFICANCE STATEMENT Reinforcement learning models of the ventral striatum (VS) often assume that it maintains an estimate of state value. This suggests that it plays a general role in learning whether rewards are assigned based on a chosen action or stimulus. In the present experiment, we examined the effects of VS lesions on monkeys' ability to learn that choosing a particular action or stimulus was more likely to lead to reward. We found that VS lesions caused a specific deficit in the monkeys' ability to discriminate between images with different values, whereas their ability to discriminate between actions with different values remained intact. Our results therefore suggest that the VS plays a specific role in learning to select rewarded stimuli.
Collapse
|
54
|
CACNA1C gene regulates behavioral strategies in operant rule learning. PLoS Biol 2017; 15:e2000936. [PMID: 28604818 PMCID: PMC5467799 DOI: 10.1371/journal.pbio.2000936] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2016] [Accepted: 05/15/2017] [Indexed: 11/28/2022] Open
Abstract
Behavioral experiments are usually designed to tap into a specific cognitive function, but animals may solve a given task through a variety of different and individual behavioral strategies, some of them not foreseen by the experimenter. Animal learning may therefore be seen more as the process of selecting among, and adapting, potential behavioral policies, rather than mere strengthening of associative links. Calcium influx through high-voltage-gated Ca2+ channels is central to synaptic plasticity, and altered expression of Cav1.2 channels and the CACNA1C gene have been associated with severe learning deficits and psychiatric disorders. Given this, we were interested in how specifically a selective functional ablation of the Cacna1c gene would modulate the learning process. Using a detailed, individual-level analysis of learning on an operant cue discrimination task in terms of behavioral strategies, combined with Bayesian selection among computational models estimated from the empirical data, we show that a Cacna1c knockout does not impair learning in general but has a much more specific effect: the majority of Cacna1c knockout mice still managed to increase reward feedback across trials but did so by adapting an outcome-based strategy, while the majority of matched controls adopted the experimentally intended cue-association rule. Our results thus point to a quite specific role of a single gene in learning and highlight that much more mechanistic insight could be gained by examining response patterns in terms of a larger repertoire of potential behavioral strategies. The results may also have clinical implications for treating psychiatric disorders. To deal with an uncertain and complex world, animals have developed a large repertoire of behavioral heuristics and default strategies that spring into action in unknown situations. Building on this a priori repertoire, animals may find various ways to succeed on a given behavioral task. Therefore, determining the exact behavioral strategy followed during a task may be essential for understanding the cognitive processes involved. Using computational models to analyze behavior, we examined how a genetic variation in a gene that encodes a calcium channel and has been associated with learning deficits influences the way in which animals acted on a task in which a reward was associated with a specific behavior. We found that a knockout of the relevant gene does not lead to a general learning impairment but rather led animals to adopt a behavioral strategy different from the one employed by the control animals. Specifically, knockout animals managed to increase their reward returns by basing their responses more on the previous reward location rather than on reward-indicating stimuli, like the controls did. These findings may prove useful for behavioral therapy in the context of psychiatric disorders associated with this specific gene variation.
Collapse
|
55
|
Contributions of basolateral amygdala and nucleus accumbens subregions to mediating motivational conflict during punished reward-seeking. Neurobiol Learn Mem 2017; 140:92-105. [DOI: 10.1016/j.nlm.2017.02.017] [Citation(s) in RCA: 43] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2016] [Revised: 02/16/2017] [Accepted: 02/22/2017] [Indexed: 12/20/2022]
|
56
|
Dickson PE, Cairns J, Goldowitz D, Mittleman G. Cerebellar contribution to higher and lower order rule learning and cognitive flexibility in mice. Neuroscience 2017; 345:99-109. [PMID: 27012612 PMCID: PMC5031514 DOI: 10.1016/j.neuroscience.2016.03.040] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2015] [Revised: 03/14/2016] [Accepted: 03/15/2016] [Indexed: 12/21/2022]
Abstract
Cognitive flexibility has traditionally been considered a frontal lobe function. However, converging evidence suggests involvement of a larger brain circuit which includes the cerebellum. Reciprocal pathways connecting the cerebellum to the prefrontal cortex provide a biological substrate through which the cerebellum may modulate higher cognitive functions, and it has been observed that cognitive inflexibility and cerebellar pathology co-occur in psychiatric disorders (e.g., autism, schizophrenia, addiction). However, the degree to which the cerebellum contributes to distinct forms of cognitive flexibility and rule learning is unknown. We tested lurcher↔wildtype aggregation chimeras which lose 0-100% of cerebellar Purkinje cells during development on a touchscreen-mediated attentional set-shifting task to assess the contribution of the cerebellum to higher and lower order rule learning and cognitive flexibility. Purkinje cells, the sole output of the cerebellar cortex, ranged from 0 to 108,390 in tested mice. Reversal learning and extradimensional set-shifting were impaired in mice with⩾95% Purkinje cell loss. Cognitive deficits were unrelated to motor deficits in ataxic mice. Acquisition of a simple visual discrimination and an attentional-set were unrelated to Purkinje cells. A positive relationship was observed between Purkinje cells and errors when exemplars from a novel, non-relevant dimension were introduced. Collectively, these data suggest that the cerebellum contributes to higher order cognitive flexibility, lower order cognitive flexibility, and attention to novel stimuli, but not the acquisition of higher and lower order rules. These data indicate that the cerebellar pathology observed in psychiatric disorders may underlie deficits involving cognitive flexibility and attention to novel stimuli.
Collapse
Affiliation(s)
- P E Dickson
- The Jackson Laboratory, 600 Main Street, Bar Harbor, ME 04609, United States
| | - J Cairns
- Centre for Molecular Medicine and Therapeutics, Department of Medical Genetics, University of British Columbia, Vancouver, BC, Canada
| | - D Goldowitz
- Centre for Molecular Medicine and Therapeutics, Department of Medical Genetics, University of British Columbia, Vancouver, BC, Canada
| | - G Mittleman
- Department of Psychology, University of Memphis, Memphis, TN 38152, United States.
| |
Collapse
|
57
|
Ketamine decreases sensitivity of male rats to misleading negative feedback in a probabilistic reversal-learning task. Psychopharmacology (Berl) 2017; 234:613-620. [PMID: 27933365 PMCID: PMC5263208 DOI: 10.1007/s00213-016-4497-1] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/04/2016] [Accepted: 11/26/2016] [Indexed: 12/15/2022]
Abstract
RATIONALE Depression is characterized by an excessive attribution of value to negative feedback. This imbalance in feedback sensitivity can be measured using the probabilistic reversal-learning (PRL) task. This task was initially designed for clinical research, but introduction of its rodent version provides a new and much needed translational paradigm to evaluate potential novel antidepressants. OBJECTIVES In the present study, we aimed at evaluating the effects of a compound showing clear antidepressant properties-ketamine (KET)-on the sensitivity of rats to positive and negative feedback in the PRL paradigm. METHODS We trained healthy rats in an operant version of the PRL task. For successful completion of the task, subjects had to learn to ignore infrequent and misleading feedback, arising from the probabilistic (80:20) nature of the discrimination. Subsequently, we evaluated the effect of KET (5, 10, and 20 mg/kg) on feedback sensitivity 1, 24, and 48 h after administration. RESULTS We report that acute administration of the highest dose of KET (20 mg/kg) rapidly and persistently decreases the proportion of lose-shift responses made by rats after receiving negative feedback. CONCLUSION Present results suggest that KET decreases negative feedback sensitivity and that changes in this basic neurocognitive function might be one of the factors responsible for its antidepressant action.
Collapse
|
58
|
Costa VD, Dal Monte O, Lucas DR, Murray EA, Averbeck BB. Amygdala and Ventral Striatum Make Distinct Contributions to Reinforcement Learning. Neuron 2016; 92:505-517. [PMID: 27720488 DOI: 10.1016/j.neuron.2016.09.025] [Citation(s) in RCA: 90] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2016] [Revised: 07/28/2016] [Accepted: 08/31/2016] [Indexed: 11/25/2022]
Abstract
Reinforcement learning (RL) theories posit that dopaminergic signals are integrated within the striatum to associate choices with outcomes. Often overlooked is that the amygdala also receives dopaminergic input and is involved in Pavlovian processes that influence choice behavior. To determine the relative contributions of the ventral striatum (VS) and amygdala to appetitive RL, we tested rhesus macaques with VS or amygdala lesions on deterministic and stochastic versions of a two-arm bandit reversal learning task. When learning was characterized with an RL model relative to controls, amygdala lesions caused general decreases in learning from positive feedback and choice consistency. By comparison, VS lesions only affected learning in the stochastic task. Moreover, the VS lesions hastened the monkeys' choice reaction times, which emphasized a speed-accuracy trade-off that accounted for errors in deterministic learning. These results update standard accounts of RL by emphasizing distinct contributions of the amygdala and VS to RL.
Collapse
Affiliation(s)
- Vincent D Costa
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892-4415, USA.
| | - Olga Dal Monte
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892-4415, USA
| | - Daniel R Lucas
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892-4415, USA
| | - Elisabeth A Murray
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892-4415, USA
| | - Bruno B Averbeck
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892-4415, USA
| |
Collapse
|
59
|
Multifaceted Contributions by Different Regions of the Orbitofrontal and Medial Prefrontal Cortex to Probabilistic Reversal Learning. J Neurosci 2016; 36:1996-2006. [PMID: 26865622 DOI: 10.1523/jneurosci.3366-15.2016] [Citation(s) in RCA: 109] [Impact Index Per Article: 13.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
UNLABELLED Different subregions of the prefrontal cortex (PFC) contribute to the ability to respond flexibly to changes in reward contingencies, with the medial versus orbitofrontal cortex (OFC) subregions contributing differentially to processes such as set-shifting and reversal learning. To date, the manner in which these regions may facilitate reversal learning in situations involving reward uncertainty remains relatively unexplored. We investigated the involvement of five distinct regions of the rat OFC (lateral and medial) and medial PFC (prelimbic, infralimbic, and anterior cingulate) on probabilistic reversal learning wherein "correct" versus "incorrect" responses were rewarded on 80% and 20% of trials, respectively. Contingencies were reversed repeatedly within a session. In well trained rats, inactivation of the medial or lateral OFC induced dissociable impairments in performance (indexed by fewer reversals completed) when outcomes were probabilistic, but not when they were assured. Medial OFC inactivation impaired probabilistic learning during the first discrimination, increased perseverative responding and reduced sensitivity to positive and negative feedback, suggestive of a deficit in incorporating information about previous action outcomes to guide subsequent behavior. Lateral OFC inactivation preferentially impaired performance during reversal phases. In contrast, prelimbic inactivation caused an apparent improvement in performance by increasing the number of reversals completed. This was associated with enhanced sensitivity to recently rewarded actions and reduced sensitivity to negative feedback. Infralimbic inactivation had no effect, whereas the anterior cingulate appeared to play a permissive role in this form of reversal learning. These results clarify the dissociable contributions of different regions of the frontal lobes to probabilistic learning. SIGNIFICANCE STATEMENT The ability to adjust behavior in response to changes involving uncertain or probabilistic reward contingencies is an essential survival skill that is impaired in a variety of psychiatric disorders. It is well established that different forms of cognitive flexibility are mediated by anatomically distinct regions of the frontal lobes when reinforcement contingencies are assured, however, less is known about the contribution of these regions to probabilistic reinforcement learning. Here we show that different regions of the orbitofrontal and medial prefrontal cortex make distinct contributions to probabilistic reversal learning. These findings provide novel information about the complex interplay between frontal lobe regions in mediating these processes and accordingly provide insight into possible pathophysiology that underlies impairments in cognitive flexibility observed in mental illnesses.
Collapse
|
60
|
Takahashi YK, Langdon AJ, Niv Y, Schoenbaum G. Temporal Specificity of Reward Prediction Errors Signaled by Putative Dopamine Neurons in Rat VTA Depends on Ventral Striatum. Neuron 2016; 91:182-93. [PMID: 27292535 DOI: 10.1016/j.neuron.2016.05.015] [Citation(s) in RCA: 67] [Impact Index Per Article: 8.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2015] [Revised: 03/30/2016] [Accepted: 04/27/2016] [Indexed: 10/21/2022]
Abstract
Dopamine neurons signal reward prediction errors. This requires accurate reward predictions. It has been suggested that the ventral striatum provides these predictions. Here we tested this hypothesis by recording from putative dopamine neurons in the VTA of rats performing a task in which prediction errors were induced by shifting reward timing or number. In controls, the neurons exhibited error signals in response to both manipulations. However, dopamine neurons in rats with ipsilateral ventral striatal lesions exhibited errors only to changes in number and failed to respond to changes in timing of reward. These results, supported by computational modeling, indicate that predictions about the temporal specificity and the number of expected reward are dissociable and that dopaminergic prediction-error signals rely on the ventral striatum for the former but not the latter.
Collapse
Affiliation(s)
| | - Angela J Langdon
- Department of Psychology and Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Yael Niv
- Department of Psychology and Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Geoffrey Schoenbaum
- NIDA Intramural Research Program, Baltimore, MD 21224, USA; Departments of Anatomy & Neurobiology and Psychiatry, University of Maryland School of Medicine, Baltimore, MD 21201, USA; Solomon H. Snyder Department of Neuroscience, The Johns Hopkins University, Baltimore, MD 21287, USA.
| |
Collapse
|
61
|
Syed A, Baker PM, Ragozzino ME. Pedunculopontine tegmental nucleus lesions impair probabilistic reversal learning by reducing sensitivity to positive reward feedback. Neurobiol Learn Mem 2016; 131:1-8. [PMID: 26976089 PMCID: PMC4862904 DOI: 10.1016/j.nlm.2016.03.010] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2015] [Revised: 02/25/2016] [Accepted: 03/03/2016] [Indexed: 01/15/2023]
Abstract
Recent findings indicate that pedunculopontine tegmental nucleus (PPTg) neurons encode reward-related information that is context-dependent. This information is critical for behavioral flexibility when reward outcomes change signaling a shift in response patterns should occur. The present experiment investigated whether NMDA lesions of the PPTg affects the acquisition and/or reversal learning of a spatial discrimination using probabilistic reinforcement. Male Long-Evans rats received a bilateral infusion of NMDA (30nmoles/side) or saline into the PPTg. Subsequently, rats were tested in a spatial discrimination test using a probabilistic learning procedure. One spatial location was rewarded with an 80% probability and the other spatial location rewarded with a 20% probability. After reaching acquisition criterion of 10 consecutive correct trials, the spatial location - reward contingencies were reversed in the following test session. Bilateral and unilateral PPTg-lesioned rats acquired the spatial discrimination test comparable to that as sham controls. In contrast, bilateral PPTg lesions, but not unilateral PPTg lesions, impaired reversal learning. The reversal learning deficit occurred because of increased regressions to the previously 'correct' spatial location after initially selecting the new, 'correct' choice. PPTg lesions also reduced the frequency of win-stay behavior early in the reversal learning session, but did not modify the frequency of lose-shift behavior during reversal learning. The present results suggest that the PPTg contributes to behavioral flexibility under conditions in which outcomes are uncertain, e.g. probabilistic reinforcement, by facilitating sensitivity to positive reward outcomes that allows the reliable execution of a new choice pattern.
Collapse
Affiliation(s)
- Anam Syed
- Department of Psychology, University of Illinois at Chicago, 1007 West Harrison Street, Chicago, IL 60607-7137, United States
| | - Phillip M Baker
- Department of Psychology, University of Illinois at Chicago, 1007 West Harrison Street, Chicago, IL 60607-7137, United States; Graduate Program in Neuroscience, University of Illinois at Chicago, Chicago, IL 60607, United States
| | - Michael E Ragozzino
- Department of Psychology, University of Illinois at Chicago, 1007 West Harrison Street, Chicago, IL 60607-7137, United States.
| |
Collapse
|
62
|
Amodeo LR, McMurray MS, Roitman JD. Orbitofrontal cortex reflects changes in response-outcome contingencies during probabilistic reversal learning. Neuroscience 2016; 345:27-37. [PMID: 26996511 PMCID: PMC10187050 DOI: 10.1016/j.neuroscience.2016.03.034] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2015] [Revised: 02/24/2016] [Accepted: 03/14/2016] [Indexed: 12/16/2022]
Abstract
In a continuously changing environment, in which behavioral outcomes are rarely certain, animals must be able to learn to integrate feedback from their choices over time and adapt to changing reward contingencies to maintain flexible behavior. The orbitofrontal region of prefrontal cortex (OFC) has been widely implicated as playing a role in the ability to flexibly control behavior. We used a probabilistic reversal learning task to measure rats' behavioral flexibility and its neural basis in the activity of single neurons in OFC. In this task, one lever, designated as 'correct', was rewarded at a high probability (80%) and a second, spatially distinct lever, designated as 'incorrect', was rewarded at a low probability (20%). Once rats reached a learning criterion for reliably selecting the correct lever, reward contingencies of the two levers were switched, and daily sessions were conducted until rats reliably selected the new correct lever. All rats performed the initial Acquisition and subsequent Reversal successfully, with more sessions needed to learn the Reversal. OFC neurons were recorded during five behavioral sessions spanning Acquisition and Reversal learning. The dominant pattern of neural responding in OFC, identified by principal component analysis of the population of neurons recorded, was modulated by reward outcome across behavioral sessions. Generally, activity was higher following rewarded choices than unrewarded. However, there was a correlation between reduced responses to reward following incorrect choices and the establishment of the preference for the correct lever. These results show how signaling by individual OFC neurons may participate in the flexible adaptation of behavior under changing reward contingencies.
Collapse
Affiliation(s)
- L R Amodeo
- Department of Psychology, University of Illinois at Chicago, 1007 West Harrison Street, Chicago, IL 60607, USA
| | - M S McMurray
- Department of Psychology, Miami University, 90 N Patterson Avenue, Oxford, OH 45056, USA
| | - J D Roitman
- Department of Psychology, University of Illinois at Chicago, 1007 West Harrison Street, Chicago, IL 60607, USA; Laboratory of Integrative Neuroscience, University of Illinois at Chicago, 1007 West Harrison Street, Chicago, IL 60607, USA.
| |
Collapse
|
63
|
Izquierdo A, Brigman JL, Radke AK, Rudebeck PH, Holmes A. The neural basis of reversal learning: An updated perspective. Neuroscience 2016; 345:12-26. [PMID: 26979052 DOI: 10.1016/j.neuroscience.2016.03.021] [Citation(s) in RCA: 351] [Impact Index Per Article: 43.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2015] [Revised: 02/24/2016] [Accepted: 03/07/2016] [Indexed: 01/21/2023]
Abstract
Reversal learning paradigms are among the most widely used tests of cognitive flexibility and have been used as assays, across species, for altered cognitive processes in a host of neuropsychiatric conditions. Based on recent studies in humans, non-human primates, and rodents, the notion that reversal learning tasks primarily measure response inhibition, has been revised. In this review, we describe how cognitive flexibility is measured by reversal learning and discuss new definitions of the construct validity of the task that are serving as a heuristic to guide future research in this field. We also provide an update on the available evidence implicating certain cortical and subcortical brain regions in the mediation of reversal learning, and an overview of the principal neurotransmitter systems involved.
Collapse
Affiliation(s)
- A Izquierdo
- Department of Psychology, The Brain Research Institute, University of California, Los Angeles, Los Angeles, CA, USA.
| | - J L Brigman
- Department of Neurosciences, University of New Mexico School of Medicine, Albuquerque, NM, USA
| | - A K Radke
- Laboratory of Behavioral and Genomic Neuroscience, National Institute on Alcohol Abuse and Alcoholism, Bethesda, MD, USA
| | - P H Rudebeck
- Department of Neuroscience and Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10014, USA
| | - A Holmes
- Laboratory of Behavioral and Genomic Neuroscience, National Institute on Alcohol Abuse and Alcoholism, Bethesda, MD, USA
| |
Collapse
|
64
|
Kumar G, Olley J, Steckler T, Talpos J. Dissociable effects of NR2A and NR2B NMDA receptor antagonism on cognitive flexibility but not pattern separation. Psychopharmacology (Berl) 2015; 232:3991-4003. [PMID: 26184010 DOI: 10.1007/s00213-015-4008-9] [Citation(s) in RCA: 28] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/03/2015] [Accepted: 06/28/2015] [Indexed: 12/29/2022]
Abstract
RATIONALE N-methyl-D-aspartate (NMDA) receptors play crucial roles in learning and memory, but the role of each NMDA receptor subtype in a specific cognitive process is unclear. Non-selective blockers of NMDA receptor are used to model the cognitive impairment in schizophrenia and Alzheimer's disease. Counter-intuitively selective NR2A and 2B NMDA receptor antagonists are thought to have pro-cognitive properties. These seemingly contrasting findings might in part be the result of different compounds and behavioral measures used across studies. OBJECTIVE We compared the effect of NVP-AAM077 (NR2A antagonist), CP 101-606 (NR2B antagonist), and MK-801 (non-selective antagonist) in a series of touch screen tasks that can be used to measure spatial cognition and cognitive flexibility. METHODS NVP-AAM077, CP 101-606, and MK-801 were administered prior to testing, in adult male Lister-hooded rats trained in tasks of location discrimination, paired associate learning (PAL), and trial unique non-match to location (TUNL). RESULTS Results showed that MK-801 impaired performance on all the tasks. In contrast, CP 101-606 only impaired reversal learning in location discrimination and had minimal effect on working memory in TUNL and caused a modest improvement in accuracy in PAL and acquisition of a spatial discrimination. NVP-AAM077 had little effect on performance across tasks, although these data allude to a potential enhancement of acquisition of a spatial location and impairments in spatial reversal learning in a separation-dependent manner. CONCLUSIONS These data demonstrated that non-selective NMDA antagonism will disrupt numerous aspects of cognitive function. However, selective antagonism is capable of impairing or enhancing cognitive function in a task-dependent fashion.
Collapse
Affiliation(s)
- Gaurav Kumar
- Janssen Research & Development, Turnhoutseweg 30, 2340, Beerse, Belgium.
| | - Joseph Olley
- Janssen Research & Development, Turnhoutseweg 30, 2340, Beerse, Belgium
| | - Thomas Steckler
- Janssen Research & Development, Turnhoutseweg 30, 2340, Beerse, Belgium
| | - John Talpos
- Janssen Research & Development, Turnhoutseweg 30, 2340, Beerse, Belgium
| |
Collapse
|
65
|
Abstract
The ability to change strategies in different contexts is a form of behavioral flexibility that is crucial for adaptive behavior. The striatum has been shown to contribute to certain forms of behavioral flexibility such as reversal learning. Here we report on the contribution of striatal cholinergic interneurons-a key element in the striatal neuronal circuit-to strategy set-shifting in which an attentional shift from one stimulus dimension to another is required. We made lesions of rat cholinergic interneurons in dorsomedial or ventral striatum using a specific immunotoxin and investigated the effects on set-shifting paradigms and on reversal learning. In shifting to a set that required attention to a previously irrelevant cue, lesions of dorsomedial striatum significantly increased the number of perseverative errors. In this condition, the number of never-reinforced errors was significantly decreased in both types of lesions. When shifting to a set that required attention to a novel cue, rats with ventral striatum lesions made more perseverative errors. Neither lesion impaired learning of the initial response strategy nor a subsequent switch to a new strategy when response choice was indicated by a previously relevant cue. Reversal learning was not affected. These results suggest that in set-shifting the striatal cholinergic interneurons play a fundamental role, which is dissociable between dorsomedial and ventral striatum depending on behavioral context. We propose a common mechanism in which cholinergic interneurons inhibit neurons representing the old strategy and enhance plasticity underlying exploration of a new rule.
Collapse
|
66
|
Young JW, Markou A. Translational Rodent Paradigms to Investigate Neuromechanisms Underlying Behaviors Relevant to Amotivation and Altered Reward Processing in Schizophrenia. Schizophr Bull 2015; 41:1024-34. [PMID: 26194891 PMCID: PMC4535652 DOI: 10.1093/schbul/sbv093] [Citation(s) in RCA: 40] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
Abstract
Amotivation and reward-processing deficits have long been described in patients with schizophrenia and considered large contributors to patients' inability to integrate well in society. No effective treatments exist for these symptoms, partly because the neuromechanisms mediating such symptoms are poorly understood. Here, we propose a translational neuroscientific approach that can be used to assess reward/motivational deficits related to the negative symptoms of schizophrenia using behavioral paradigms that can also be conducted in experimental animals. By designing and using objective laboratory behavioral tools that are parallel in their parameters in rodents and humans, the neuromechanisms underlying behaviors with relevance to these symptoms of schizophrenia can be investigated. We describe tasks that measure the motivation of rodents to expend physical and cognitive effort to gain rewards, as well as probabilistic learning tasks that assess both reward learning and feedback-based decision making. The latter tasks are relevant because of demonstrated links of performance deficits correlating with negative symptoms in patients with schizophrenia. These tasks utilize operant techniques in order to investigate neural circuits targeting a specific domain across species. These tasks therefore enable the development of insights into altered mechanisms leading to negative symptom-relevant behaviors in patients with schizophrenia. Such findings will then enable the development of targeted treatments for these altered neuromechanisms and behaviors seen in schizophrenia.
Collapse
Affiliation(s)
- Jared W Young
- Department of Psychiatry, University of California San Diego, 9500 Gilman Drive MC 0804, La Jolla, CA; Research Service, VA San Diego Healthcare System, San Diego, CA
| | - Athina Markou
- Department of Psychiatry, University of California San Diego, 9500 Gilman Drive MC 0804, La Jolla, CA
| |
Collapse
|
67
|
Liu Z, Braunlich K, Wehe HS, Seger CA. Neural networks supporting switching, hypothesis testing, and rule application. Neuropsychologia 2015. [PMID: 26197092 DOI: 10.1016/j.neuropsychologia.2015.07.019] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
We identified dynamic changes in recruitment of neural connectivity networks across three phases of a flexible rule learning and set-shifting task similar to the Wisconsin Card Sort Task: switching, rule learning via hypothesis testing, and rule application. During fMRI scanning, subjects viewed pairs of stimuli that differed across four dimensions (letter, color, size, screen location), chose one stimulus, and received feedback. Subjects were informed that the correct choice was determined by a simple unidimensional rule, for example "choose the blue letter". Once each rule had been learned and correctly applied for 4-7 trials, subjects were cued via either negative feedback or visual cues to switch to learning a new rule. Task performance was divided into three phases: Switching (first trial after receiving the switch cue), hypothesis testing (subsequent trials through the last error trial), and rule application (correct responding after the rule was learned). We used both univariate analysis to characterize activity occurring within specific regions of the brain, and a multivariate method, constrained principal component analysis for fMRI (fMRI-CPCA), to investigate how distributed regions coordinate to subserve different processes. As hypothesized, switching was subserved by a limbic network including the ventral striatum, thalamus, and parahippocampal gyrus, in conjunction with cortical salience network regions including the anterior cingulate and frontoinsular cortex. Activity in the ventral striatum was associated with switching regardless of how switching was cued; visually cued shifts were associated with additional visual cortical activity. After switching, as subjects moved into the hypothesis testing phase, a broad fronto-parietal-striatal network (associated with the cognitive control, dorsal attention, and salience networks) increased in activity. This network was sensitive to rule learning speed, with greater extended activity for the slowest learning speed late in the time course of learning. As subjects shifted from hypothesis testing to rule application, activity in this network decreased and activity in the somatomotor and default mode networks increased.
Collapse
Affiliation(s)
- Zhiya Liu
- South China Normal University, School of Psychology, Center for Studies of Psychological Application, Guangzhou, China.
| | - Kurt Braunlich
- Colorado State University, Department of Psychology, Colorado, United States; Colorado State University, Molecular, Cellular and Integrative Neurosciences Program, Colorado, United States
| | - Hillary S Wehe
- Colorado State University, Department of Psychology, Colorado, United States; Colorado State University, Molecular, Cellular and Integrative Neurosciences Program, Colorado, United States
| | - Carol A Seger
- South China Normal University, School of Psychology, Center for Studies of Psychological Application, Guangzhou, China; Colorado State University, Department of Psychology, Colorado, United States; Colorado State University, Molecular, Cellular and Integrative Neurosciences Program, Colorado, United States.
| |
Collapse
|
68
|
Orsini CA, Moorman DE, Young JW, Setlow B, Floresco SB. Neural mechanisms regulating different forms of risk-related decision-making: Insights from animal models. Neurosci Biobehav Rev 2015; 58:147-67. [PMID: 26072028 DOI: 10.1016/j.neubiorev.2015.04.009] [Citation(s) in RCA: 97] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2014] [Revised: 02/13/2015] [Accepted: 04/24/2015] [Indexed: 11/18/2022]
Abstract
Over the past 20 years there has been a growing interest in the neural underpinnings of cost/benefit decision-making. Recent studies with animal models have made considerable advances in our understanding of how different prefrontal, striatal, limbic and monoaminergic circuits interact to promote efficient risk/reward decision-making, and how dysfunction in these circuits underlies aberrant decision-making observed in numerous psychiatric disorders. This review will highlight recent findings from studies exploring these questions using a variety of behavioral assays, as well as molecular, pharmacological, neurophysiological, and translational approaches. We begin with a discussion of how neural systems related to decision subcomponents may interact to generate more complex decisions involving risk and uncertainty. This is followed by an overview of interactions between prefrontal-amygdala-dopamine and habenular circuits in regulating choice between certain and uncertain rewards and how different modes of dopamine transmission may contribute to these processes. These data will be compared with results from other studies investigating the contribution of some of these systems to guiding decision-making related to rewards vs. punishment. Lastly, we provide a brief summary of impairments in risk-related decision-making associated with psychiatric disorders, highlighting recent translational studies in laboratory animals.
Collapse
Affiliation(s)
- Caitlin A Orsini
- Department of Psychiatry and Center for Addiction Research and Education, University of Florida College of Medicine, Gainesville, FL, United States
| | - David E Moorman
- Department of Psychological and Brain Sciences, University of Massachusetts, Amherst, MA, United States
| | - Jared W Young
- Department of Psychiatry, University of California San Diego, United States; VISN-22 Mental Illness, Research, Education and Clinical Center (MIRECC), VA San Diego Healthcare System, San Diego, CA, United States
| | - Barry Setlow
- Department of Psychiatry and Center for Addiction Research and Education, University of Florida College of Medicine, Gainesville, FL, United States
| | - Stan B Floresco
- Department of Psychology and Brain Research Center, University of British Columbia, Vancouver, BC, Canada.
| |
Collapse
|
69
|
Orbitofrontal or accumbens dopamine depletion does not affect risk-based decision making in rats. COGNITIVE AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2015; 15:507-22. [DOI: 10.3758/s13415-015-0353-8] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
70
|
Young JW, Geyer MA. Developing treatments for cognitive deficits in schizophrenia: the challenge of translation. J Psychopharmacol 2015; 29:178-96. [PMID: 25516372 PMCID: PMC4670265 DOI: 10.1177/0269881114555252] [Citation(s) in RCA: 83] [Impact Index Per Article: 9.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Schizophrenia is a life-long debilitating mental disorder affecting tens of millions of people worldwide. The serendipitous discovery of antipsychotics focused pharmaceutical research on developing a better antipsychotic. Our understanding of the disorder has advanced however, with the knowledge that cognitive enhancers are required for patients in order to improve their everyday lives. While antipsychotics treat psychosis, they do not enhance cognition and hence are not antischizophrenics. Developing pro-cognitive therapeutics has been extremely difficult, however, especially when no approved treatment exists. In lieu of stumbling on an efficacious treatment, developing targeted compounds can be facilitated by understanding the neural mechanisms underlying altered cognitive functioning in patients. Equally importantly, these cognitive domains will need to be measured similarly in animals and humans so that novel targets can be tested prior to conducting expensive clinical trials. To date, the limited similarity of testing across species has resulted in a translational bottleneck. In this review, we emphasize that schizophrenia is a disorder characterized by abnormal cognitive behavior. Quantifying these abnormalities using tasks having cross-species validity would enable the quantification of comparable processes in rodents. This approach would increase the likelihood that the neural substrates underlying relevant behaviors will be conserved across species. Hence, we detail cross-species tasks which can be used to test the effects of manipulations relevant to schizophrenia and putative therapeutics. Such tasks offer the hope of providing a bridge between non-clinical and clinical testing that will eventually lead to treatments developed specifically for patients with deficient cognition.
Collapse
Affiliation(s)
- JW Young
- Department of Psychiatry, University of California San Diego, La Jolla, CA, USA
| | - MA Geyer
- Research Service, VA San Diego Healthcare System, San Diego, CA, USA
| |
Collapse
|
71
|
Affiliation(s)
- Stan B. Floresco
- Department of Psychology and Brain Research Center, University of British Columbia, Vancouver, British Columbia, V6T 1Z4 Canada;
| |
Collapse
|