1
|
Madden GJ, Mahmoudi S, Brown K. Pavlovian learning and conditioned reinforcement. J Appl Behav Anal 2023; 56:498-519. [PMID: 37254881 PMCID: PMC10364091 DOI: 10.1002/jaba.1004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2023] [Accepted: 05/09/2023] [Indexed: 06/01/2023]
Abstract
Conditioned reinforcers are widely used in applied behavior analysis. Basic research evidence reveals that Pavlovian learning plays an important role in the acquisition and efficacy of new conditioned-reinforcer functions. Thus, a better understanding of Pavlovian principles holds the promise of improving the efficacy of conditioned reinforcement in applied research and practice. This paper surveys how (and if) Pavlovian principles are presented in behavior-analytic textbooks; imprecisions and knowledge gaps within contemporary Pavlovian empirical findings are highlighted. Thereafter, six practical principles of Pavlovian conditioning are presented along with empirical support and knowledge gaps that should be filled by applied and translational behavior-analytic researchers. Innovative applications of these principles are outlined for research in language acquisition, token reinforcement, and self-control.
Collapse
Affiliation(s)
| | - Saba Mahmoudi
- Department of Psychology, Utah State University, Logan, UT, USA
| | - Katherine Brown
- Department of Psychology, Utah State University, Logan, UT, USA
| |
Collapse
|
2
|
Gyawali U, Martin DA, Sun F, Li Y, Calu D. Dopamine in the dorsal bed nucleus of stria terminalis signals Pavlovian sign-tracking and reward violations. eLife 2023; 12:e81980. [PMID: 37232554 PMCID: PMC10219648 DOI: 10.7554/elife.81980] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Accepted: 05/05/2023] [Indexed: 05/27/2023] Open
Abstract
Midbrain and striatal dopamine signals have been extremely well characterized over the past several decades, yet novel dopamine signals and functions in reward learning and motivation continue to emerge. A similar characterization of real-time sub-second dopamine signals in areas outside of the striatum has been limited. Recent advances in fluorescent sensor technology and fiber photometry permit the measurement of dopamine binding correlates, which can divulge basic functions of dopamine signaling in non-striatal dopamine terminal regions, like the dorsal bed nucleus of the stria terminalis (dBNST). Here, we record GRABDA signals in the dBNST during a Pavlovian lever autoshaping task. We observe greater Pavlovian cue-evoked dBNST GRABDA signals in sign-tracking (ST) compared to goal-tracking/intermediate (GT/INT) rats and the magnitude of cue-evoked dBNST GRABDA signals decreases immediately following reinforcer-specific satiety. When we deliver unexpected rewards or omit expected rewards, we find that dBNST dopamine signals encode bidirectional reward prediction errors in GT/INT rats, but only positive prediction errors in ST rats. Since sign- and goal-tracking approach strategies are associated with distinct drug relapse vulnerabilities, we examined the effects of experimenter-administered fentanyl on dBNST dopamine associative encoding. Systemic fentanyl injections do not disrupt cue discrimination but generally potentiate dBNST dopamine signals. These results reveal multiple dBNST dopamine correlates of learning and motivation that depend on the Pavlovian approach strategy employed.
Collapse
Affiliation(s)
- Utsav Gyawali
- Program in Neuroscience, University of Maryland School of MedicineBaltimoreUnited States
- Department of Anatomy and Neurobiology, University of Maryland School of MedicineBaltimoreUnited States
| | - David A Martin
- Department of Anatomy and Neurobiology, University of Maryland School of MedicineBaltimoreUnited States
| | - Fangmiao Sun
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences; PKU-IDG/McGovern Institute for Brain Research; Peking-Tsinghua Center for Life SciencesBeijingChina
| | - Yulong Li
- State Key Laboratory of Membrane Biology, Peking University School of Life Sciences; PKU-IDG/McGovern Institute for Brain Research; Peking-Tsinghua Center for Life SciencesBeijingChina
| | - Donna Calu
- Program in Neuroscience, University of Maryland School of MedicineBaltimoreUnited States
- Department of Anatomy and Neurobiology, University of Maryland School of MedicineBaltimoreUnited States
| |
Collapse
|
3
|
Armstrong A, Rosenthal H, Stout N, Richard JM. Reinstatement of Pavlovian responses to alcohol cues by stress. Psychopharmacology (Berl) 2023; 240:531-545. [PMID: 36227353 PMCID: PMC9931652 DOI: 10.1007/s00213-022-06255-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Accepted: 10/02/2022] [Indexed: 11/29/2022]
Abstract
RATIONALE Stress may contribute to relapse to alcohol use in part by enhancing reactivity to cues previously paired with alcohol. Yet, standard models of stress-induced reinstatement generally use contingent presentations of alcohol-paired cues to reinforce instrumental behaviors, making it difficult to isolate the ability of cues to invigorate alcohol-seeking. OBJECTIVE Here we sought to test the impact of stress on behavioral responses to alcohol-paired cues, using a model of stress-induced reinstatement of Pavlovian conditioned approach, inspired by Nadia Chaudhri's work on context-induced reinstatement. METHODS Long Evans rats were trained to associate one auditory cue with delivery of alcohol or sucrose and an alternative auditory cue with no reward. Following extinction training, rats were exposed to a stressor prior to being re-exposed to the cues under extinction conditions. We assessed the effects of yohimbine, intermittent footshock and olfactory cues paired with social defeat on responses to alcohol-paired cues and the effects of yohimbine on responses to sucrose-paired cues. RESULTS The pharmacological stressor, yohimbine, enhanced alcohol seeking in a Pavlovian setting, but not in a cue-selective manner. Intermittent footshock and social defeat cues did not enhance alcohol seeking in this paradigm. CONCLUSIONS While yohimbine elicited reinstatement of reward-seeking in a Pavlovian setting, these effects may be unrelated to activation of stress systems or to interactions with specific cues.
Collapse
Affiliation(s)
- Anne Armstrong
- Department of Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Hailey Rosenthal
- Department of Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Nakura Stout
- Department of Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Jocelyn M Richard
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, 55415, USA.
- Medical Discovery Team On Addiction, University of Minnesota, Minneapolis, MN, 55415, USA.
| |
Collapse
|
4
|
Inactivation of the Basolateral Amygdala to Insular Cortex Pathway Makes Sign-Tracking Sensitive to Outcome Devaluation. eNeuro 2022; 9:ENEURO.0156-22.2022. [PMID: 36127135 PMCID: PMC9522321 DOI: 10.1523/eneuro.0156-22.2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Revised: 08/26/2022] [Accepted: 09/11/2022] [Indexed: 12/15/2022] Open
Abstract
Goal-tracking (GT) rats are sensitive to Pavlovian outcome devaluation while sign-tracking (ST) rats are devaluation insensitive. During outcome devaluation, GT rats flexibly modify responding to cues based on the current value of the associated outcome. However, ST rats rigidly respond to cues regardless of the current outcome value. Prior work demonstrated disconnection of the basolateral amygdala (BLA) and anterior insular cortex (aIC) decreased both GT and ST behaviors. Given the role of these regions in appetitive motivation and behavioral flexibility, we predicted that disrupting BLA to aIC pathway during outcome devaluation would reduce flexibility in GT rats and reduce rigid appetitive motivation in ST rats. We inhibited the BLA to aIC pathway by infusing inhibitory DREADDs (hM4Di-mcherry) or control (mCherry) virus into the BLA and implanted cannulae into the aIC to inhibit BLA terminals using intracranial injections of clozapine N-oxide (CNO). After training, we used a within-subject satiety-induced outcome devaluation procedure in which we sated rats on training pellets (devalued condition) or homecage chow (valued condition). All rats received bilateral CNO infusions into the aIC before brief nonreinforced test sessions. Contrary to our hypothesis, BLA-IC inhibition did not interfere with devaluation sensitivity in GT rats but did make ST behaviors sensitive to devaluation. Intermediate rats showed the opposite effect, showing rigid responding to cues with BLA-aIC pathway inactivation. Together, these results demonstrate BLA-IC projections mediate tracking-specific Pavlovian devaluation sensitivity and highlights the importance of considering individual differences in Pavlovian approach when evaluating circuitry contributions to behavioral flexibility.
Collapse
|
5
|
Keefer SE, Gyawali U, Calu DJ. Choose your path: Divergent basolateral amygdala efferents differentially mediate incentive motivation, flexibility and decision-making. Behav Brain Res 2021; 409:113306. [PMID: 33887310 PMCID: PMC8189324 DOI: 10.1016/j.bbr.2021.113306] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2020] [Revised: 04/09/2021] [Accepted: 04/12/2021] [Indexed: 10/21/2022]
Abstract
To survive in a complex environment, individuals form associations between environmental stimuli and rewards to organize and optimize reward seeking behaviors. The basolateral amygdala (BLA) uses these learned associations to inform decision-making processes. In this review, we describe functional projections between BLA and its cortical and striatal targets that promote learning and motivational processes central to decision-making. Specifically, we compare and contrast divergent projections from the BLA to the orbitofrontal (OFC) and to the nucleus accumbens (NAc) and examine the roles of these pathways in associative learning, value-guided decision-making, choice behaviors, as well as cue and context-driven drug seeking. Finally, we consider how these projections are involved in disorders of motivation, with a focus on Substance Use Disorder.
Collapse
Affiliation(s)
- Sara E Keefer
- Department of Anatomy and Neurobiology, University of Maryland, School of Medicine, Baltimore, MD, United States
| | - Utsav Gyawali
- Department of Anatomy and Neurobiology, University of Maryland, School of Medicine, Baltimore, MD, United States; Program in Neuroscience, University of Maryland, School of Medicine, Baltimore, MD, United States
| | - Donna J Calu
- Department of Anatomy and Neurobiology, University of Maryland, School of Medicine, Baltimore, MD, United States; Program in Neuroscience, University of Maryland, School of Medicine, Baltimore, MD, United States.
| |
Collapse
|
6
|
Lay BPP, Khoo SYS. Associative processes in addiction relapse models: A review of their Pavlovian and instrumental mechanisms, history, and terminology. NEUROANATOMY AND BEHAVIOUR 2021. [DOI: 10.35430/nab.2021.e18] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
Animal models of relapse to drug-seeking have borrowed heavily from associative learning approaches. In studies of relapse-like behaviour, animals learn to self-administer drugs then receive a period of extinction during which they learn to inhibit the operant response. Several triggers can produce a recovery of responding which form the basis of a variety of models. These include the passage of time (spontaneous recovery), drug availability (rapid reacquisition), extinction of an alternative response (resurgence), context change (renewal), drug priming, stress, and cues (reinstatement). In most cases, the behavioural processes driving extinction and recovery in operant drug self-administration studies are similar to those in the Pavlovian and behavioural literature, such as context effects. However, reinstatement in addiction studies have several differences with Pavlovian reinstatement, which have emerged over several decades, in experimental procedures, associative mechanisms, and terminology. Interestingly, in cue-induced reinstatement, drug-paired cues that are present during acquisition are omitted during lever extinction. The unextinguished drug-paired cue may limit the model’s translational relevance to cue exposure therapy and renders its underlying associative mechanisms ambiguous. We review major behavioural theories that explain recovery phenomena, with a particular focus on cue-induced reinstatement because it is a widely used model in addiction. We argue that cue-induced reinstatement may be explained by a combination of behavioural processes, including reacquisition of conditioned reinforcement and Pavlovian to Instrumental Transfer. While there are important differences between addiction studies and the behavioural literature in terminology and procedures, it is clear that understanding associative learning processes is essential for studying relapse.
Collapse
Affiliation(s)
- Belinda Po Pyn Lay
- Center for Studies in Behavioral Neurobiology/Groupe de Recherche en Neurobiologie Comportementale, Department of Psychology, Concordia University, Montreal, Canada
| | - Shaun Yon-Seng Khoo
- Department of Pharmacology and Physiology, Faculty of Medicine, Université de Montréal, Montreal, Quebec, Canada
| |
Collapse
|
7
|
Panayi MC, Killcross S. The Role of the Rodent Lateral Orbitofrontal Cortex in Simple Pavlovian Cue-Outcome Learning Depends on Training Experience. Cereb Cortex Commun 2021; 2:tgab010. [PMID: 34296155 PMCID: PMC8152875 DOI: 10.1093/texcom/tgab010] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2020] [Revised: 01/29/2021] [Accepted: 02/01/2021] [Indexed: 11/30/2022] Open
Abstract
The orbitofrontal cortex (OFC) is a critical structure in the flexible control of value-based behaviors. OFC dysfunction is typically only detected when task or environmental contingencies change, against a backdrop of apparently intact initial acquisition and behavior. While intact acquisition following OFC lesions in simple Pavlovian cue-outcome conditioning is often predicted by models of OFC function, this predicted null effect has not been thoroughly investigated. Here, we test the effects of lesions and temporary muscimol inactivation of the rodent lateral OFC on the acquisition of a simple single cue-outcome relationship. Surprisingly, pretraining lesions significantly enhanced acquisition after overtraining, whereas post-training lesions and inactivation significantly impaired acquisition. This impaired acquisition to the cue reflects a disruption of behavioral control and not learning since the cue could also act as an effective blocking stimulus in an associative blocking procedure. These findings suggest that even simple cue-outcome representations acquired in the absence of OFC function are impoverished. Therefore, while OFC function is often associated with flexible behavioral control in complex environments, it is also involved in very simple Pavlovian acquisition where complex cue-outcome relationships are irrelevant to task performance.
Collapse
Affiliation(s)
- Marios C Panayi
- School of Psychology, UNSW Sydney, Sydney, NSW 2052, Australia
- National Institute on Drug Abuse Intramural Research Program, Cellular Neurobiology Research Branch, Behavioral Neurophysiology Research Section, 251 Bayview Blvd., Baltimore, MD 21224, USA
| | - Simon Killcross
- School of Psychology, UNSW Sydney, Sydney, NSW 2052, Australia
| |
Collapse
|
8
|
Behavioral determinants in the expression of the Kamin blocking effect: Implications for associative learning theory. Neurosci Biobehav Rev 2021; 124:16-34. [PMID: 33497781 DOI: 10.1016/j.neubiorev.2021.01.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2020] [Revised: 01/02/2021] [Accepted: 01/11/2021] [Indexed: 10/22/2022]
Abstract
Associative learning makes important contributions to our behavior and decisions. The Kamin blocking effect is an associative learning phenomenon that plays a central role in understanding of the psychological principles underlying associative learning. However, several recent failures to replicate the blocking effect suggest that the conditions necessary for blocking are poorly understood. To understand the conditions necessary for blocking, here we review studies into the expression of blocking in subjects that either approach and interact with the conditioned cue (sign trackers) or approach and interact with the reward location (goal trackers) during appetitive classical conditioning. Psychological theory and the neurophysiological correlates of appetitive classical conditioning make opposing predictions regarding the expression of blocking in sign and goal trackers. We reconcile these opposing predictions in a qualitative model using two parallel learning processes. Such models offer a better framework for understanding the psychological associative structures acquired during learning, their interactions contributing to the conditioned response, and how they affect subsequent learning and the expression of the Kamin blocking effect.
Collapse
|
9
|
Fraser KM, Janak PH. Occasion setters attain incentive motivational value: implications for contextual influences on reward-seeking. ACTA ACUST UNITED AC 2019; 26:291-298. [PMID: 31308248 PMCID: PMC6636542 DOI: 10.1101/lm.049320.119] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2019] [Accepted: 06/07/2019] [Indexed: 01/03/2023]
Abstract
The context in which reward-paired cues are encountered can resolve ambiguity and set the occasion for appropriate reward-seeking. The psychological processes by which contexts regulate reward-seeking remain unclear as contexts are diffuse and difficult to isolate from other stimuli. To overcome this, we modeled a context as a phasic and discrete event—an occasion setter (OS)—which allowed for control over its presentation and influence on cue-driven reward-seeking. This allowed us to directly assess how OSs regulate the predictive and motivational significance of Pavlovian cues. Male rats (n = 50) were trained in a Pavlovian procedure where the presentation of an ambiguous conditioned stimulus (CS) was reinforced only if preceded by an occasion setting cue. We assessed the motivational value of the OS and CS alone or in combination using tests of conditioned reinforcement. Rats enhanced conditioned approach to the reward port during the CS when it was preceded by the OS. When allowed the opportunity, rats responded more to obtain presentations of the CS in combination with the OS than the CS alone. Critically, rats also worked to obtain presentations of the OS alone more than the CS alone, and this was resistant to manipulations of the value of the OS. We conclude that occasion setting can act via incentive motivational mechanisms and that, apart from resolving predictive information about ambiguous reward-paired cues, OSs themselves generate states of appetitive motivation that can facilitate reward-seeking.
Collapse
Affiliation(s)
- Kurt M Fraser
- Department of Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, Maryland 21218, USA
| | - Patricia H Janak
- Department of Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, Maryland 21218, USA.,The Solomon H. Snyder Department of Neuroscience, Johns Hopkins School of Medicine, Baltimore, Maryland 21205, USA.,Kavli Neuroscience Discovery Institute, Johns Hopkins School of Medicine, Baltimore, Maryland 21205, USA
| |
Collapse
|
10
|
Gu S, Wang F, Cao C, Wu E, Tang YY, Huang JH. An Integrative Way for Studying Neural Basis of Basic Emotions With fMRI. Front Neurosci 2019; 13:628. [PMID: 31275107 PMCID: PMC6593191 DOI: 10.3389/fnins.2019.00628] [Citation(s) in RCA: 38] [Impact Index Per Article: 7.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2018] [Accepted: 05/31/2019] [Indexed: 01/18/2023] Open
Abstract
How emotions are represented in the nervous system is a crucial unsolved problem in the affective neuroscience. Many studies are striving to find the localization of basic emotions in the brain but failed. Thus, many psychologists suspect the specific neural loci for basic emotions, but instead, some proposed that there are specific neural structures for the core affects, such as arousal and hedonic value. The reason for this widespread difference might be that basic emotions used previously can be further divided into more “basic” emotions. Here we review brain imaging data and neuropsychological data, and try to address this question with an integrative model. In this model, we argue that basic emotions are not contrary to the dimensional studies of emotions (core affects). We propose that basic emotion should locate on the axis in the dimensions of emotion, and only represent one typical core affect (arousal or valence). Therefore, we propose four basic emotions: joy-on positive axis of hedonic dimension, sadness-on negative axis of hedonic dimension, fear, and anger-on the top of vertical dimensions. This new model about basic emotions and construction model of emotions is promising to improve and reformulate neurobiological models of basic emotions.
Collapse
Affiliation(s)
- Simeng Gu
- Institute of Brain and Psychological Science, Sichuan Normal University, Chengdu, China.,Department of Psychology, Jiangsu University, Zhenjiang, China
| | - Fushun Wang
- Institute of Brain and Psychological Science, Sichuan Normal University, Chengdu, China.,Department of Pharmacology, Nanjing University of Chinese Medicine, Nanjing, China.,Department of Neurosurgery, Baylor Scott & White Health, Temple, TX, United States
| | - Caiyun Cao
- Department of Pharmacology, Nanjing University of Chinese Medicine, Nanjing, China
| | - Erxi Wu
- Department of Neurosurgery, Baylor Scott & White Health, Temple, TX, United States.,Department of Surgery, Texas A&M University College of Medicine, Temple, TX, United States.,Department of Pharmaceutical Sciences, Texas A&M University College of Pharmacy, College Station, TX, United States.,LIVESTRONG Cancer Institutes, Dell Medical School, The University of Texas at Austin, Austin, TX, United States
| | - Yi-Yuan Tang
- Department of Psychological Sciences, Texas Tech University, Lubbock, TX, United States
| | - Jason H Huang
- Department of Neurosurgery, Baylor Scott & White Health, Temple, TX, United States.,Department of Surgery, Texas A&M University College of Medicine, Temple, TX, United States.,Department of Pharmaceutical Sciences, Texas A&M University College of Pharmacy, College Station, TX, United States.,LIVESTRONG Cancer Institutes, Dell Medical School, The University of Texas at Austin, Austin, TX, United States
| |
Collapse
|
11
|
Sharpe MJ, Batchelor HM, Schoenbaum G. Preconditioned cues have no value. eLife 2017; 6:28362. [PMID: 28925358 PMCID: PMC5619948 DOI: 10.7554/elife.28362] [Citation(s) in RCA: 30] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2017] [Accepted: 09/18/2017] [Indexed: 01/15/2023] Open
Abstract
Sensory preconditioning has been used to implicate midbrain dopamine in model-based learning, contradicting the view that dopamine transients reflect model-free value. However, it has been suggested that model-free value might accrue directly to the preconditioned cue through mediated learning. Here, building on previous work (Sadacca et al., 2016), we address this question by testing whether a preconditioned cue will support conditioned reinforcement in rats. We found that while both directly conditioned and second-order conditioned cues supported robust conditioned reinforcement, a preconditioned cue did not. These data show that the preconditioned cue in our procedure does not directly accrue model-free value and further suggest that the cue may not necessarily access value even indirectly in a model-based manner. If so, then phasic response of dopamine neurons to cues in this setting cannot be described as signaling errors in predicting value.
Collapse
Affiliation(s)
- Melissa J Sharpe
- NIDA Intramural Research Program, Baltimore, United States.,Princeton Neuroscience Institute, Princeton University, Princeton, United States.,School of Psychology, University of New South Wales, Sydney, Australia
| | | | - Geoffrey Schoenbaum
- NIDA Intramural Research Program, Baltimore, United States.,Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, United States.,Department of Psychiatry, University of Maryland School of Medicine, Baltimore, United States.,Solomon H. Snyder Department of Neuroscience, The Johns Hopkins University, Baltimore, United States
| |
Collapse
|
12
|
Effects of sex and remifentanil dose on rats' acquisition of responding for a remifentanil-conditioned reinforcer. Behav Pharmacol 2016; 27:137-47. [PMID: 26580131 DOI: 10.1097/fbp.0000000000000205] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
Opioid-conditioned reinforcement is thought to exacerbate opioid abuse and dependence. Sex/gender can influence opioid abuse behaviors, but the effects of sex/gender on opioid-conditioned reinforcement, specifically, are unclear. In this study, we compared new-response acquisition with opioid-conditioned reinforcement in male and female rats. First, separate groups received response-independent remifentanil injections (0.0-32.0 μg/kg, intravenous) and presentations of a light-noise stimulus. In the experimental groups, injections and stimulus presentations always co-occurred [paired Pavlovian conditioning (PAV)]; in the control groups, the two occurred independently of each other (random PAV). Next, in the instrumental acquisition (ACQ) sessions, two novel nose-poke manipulanda were introduced. All animals (regardless of sex, dose, and PAV type) could respond in the active nose-poke, which produced the stimulus alone, or in the inactive nose-poke. Both males and females dose-dependently acquired nose-poke responding (active>inactive) after paired PAV, but not after random PAV. Therefore, the stimulus was a conditioned reinforcer. We identified three sex differences. First, only females acquired responding after paired PAV with 32.0 μg/kg remifentanil. Second, using a progressive ratio schedule for ACQ, both sexes acquired responding, but females made significantly more active responses. Third, when a single session of PAV was conducted, only males acquired responding. Thus, rats' sex interacts with pharmacological and environmental factors to determine opioid-conditioned reinforcement.
Collapse
|
13
|
Dopamine D2/3- and μ-opioid receptor antagonists reduce cue-induced responding and reward impulsivity in humans. Transl Psychiatry 2016; 6:e850. [PMID: 27378550 PMCID: PMC4969763 DOI: 10.1038/tp.2016.113] [Citation(s) in RCA: 48] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/03/2016] [Revised: 04/27/2016] [Accepted: 04/29/2016] [Indexed: 01/05/2023] Open
Abstract
Increased responding to drug-associated stimuli (cue reactivity) and an inability to tolerate delayed gratification (reward impulsivity) have been implicated in the development and maintenance of drug addiction. Whereas data from animal studies suggest that both the dopamine and opioid system are involved in these two reward-related processes, their role in humans is less clear. Moreover, dopaminergic and opioidergic drugs have not been directly compared with regard to these functions, even though a deeper understanding of the underlying mechanisms might inform the development of specific treatments for elevated cue reactivity and reward impulsivity. In a randomized, double-blind, between-subject design we administered the selective dopamine D2/D3 receptor antagonist amisulpride (400 mg, n=41), the unspecific opioid receptor antagonist naltrexone (50 mg, n=40) or placebo (n=40) to healthy humans and measured cue-induced responding with a Pavlovian-instrumental transfer task and reward impulsivity with a delay discounting task. Mood was assessed using a visual analogue scale. Compared with placebo, amisulpride significantly suppressed cue-induced responding and reward impulsivity. The effects of naltrexone were similar, although less pronounced. Both amisulpride and naltrexone decreased average mood ratings compared with placebo. Our results demonstrate that a selective blockade of dopamine D2/D3 receptors reduces cue-induced responding and reward impulsivity in healthy humans. Antagonizing μ-opioid receptors has similar effects for cue-induced responding and to a lesser extent for reward impulsivity.
Collapse
|
14
|
Ochoa JG, Stolyarova A, Kaur A, Hart EE, Bugarin A, Izquierdo A. Post-training depletions of basolateral amygdala serotonin fail to disrupt discrimination, retention, or reversal learning. Front Neurosci 2015; 9:155. [PMID: 26029036 PMCID: PMC4426727 DOI: 10.3389/fnins.2015.00155] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2015] [Accepted: 04/15/2015] [Indexed: 02/01/2023] Open
Abstract
In goal-directed pursuits, the basolateral amygdala (BLA) is critical in learning about changes in the value of rewards. BLA-lesioned rats show enhanced reversal learning, a task employed to measure the flexibility of response to changes in reward. Similarly, there is a trend for enhanced discrimination learning, suggesting that BLA may modulate formation of stimulus-reward associations. There is a parallel literature on the importance of serotonin (5HT) in new stimulus-reward and reversal learning. Recent postulations implicate 5HT in learning from punishment. Whereas, dopaminergic involvement is critical in behavioral activation and reinforcement, 5HT may be most critical for aversive processing and behavioral inhibition, complementary cognitive processes. Given these findings, a 5HT-mediated mechanism in BLA may mediate the facilitated learning observed previously. The present study investigated the effects of selective 5HT lesions in BLA using 5,7-dihydroxytryptamine (5,7-DHT) vs. infusions of saline (Sham) on discrimination, retention, and deterministic reversal learning. Rats were required to reach an 85% correct pairwise discrimination and single reversal criterion prior to surgery. Postoperatively, rats were then tested on the (1) retention of the pretreatment discrimination pair, (2) discrimination of a novel pair, and (3) reversal learning performance. We found statistically comparable preoperative learning rates between groups, intact postoperative retention, and unaltered novel discrimination and reversal learning in 5,7-DHT rats. These findings suggest that 5HT in BLA is not required for formation and flexible adjustment of new stimulus-reward associations when the strategy to efficiently solve the task has already been learned. Given the complementary role of orbitofrontal cortex in reward learning and its interconnectivity with BLA, these findings add to the list of dissociable mechanisms for BLA and orbitofrontal cortex in reward learning.
Collapse
Affiliation(s)
- Jesus G Ochoa
- Department of Psychology, University of California, Los Angeles Los Angeles, CA, USA ; Brain Research Institute, University of California, Los Angeles Los Angeles, CA, USA
| | - Alexandra Stolyarova
- Department of Psychology, University of California, Los Angeles Los Angeles, CA, USA ; Brain Research Institute, University of California, Los Angeles Los Angeles, CA, USA
| | - Amandeep Kaur
- Department of Psychology, University of California, Los Angeles Los Angeles, CA, USA ; Brain Research Institute, University of California, Los Angeles Los Angeles, CA, USA
| | - Evan E Hart
- Department of Psychology, University of California, Los Angeles Los Angeles, CA, USA ; Brain Research Institute, University of California, Los Angeles Los Angeles, CA, USA
| | - Amador Bugarin
- Department of Psychology, University of California, Los Angeles Los Angeles, CA, USA ; Brain Research Institute, University of California, Los Angeles Los Angeles, CA, USA
| | - Alicia Izquierdo
- Department of Psychology, University of California, Los Angeles Los Angeles, CA, USA ; Brain Research Institute, University of California, Los Angeles Los Angeles, CA, USA
| |
Collapse
|
15
|
Bertz JW, Chen J, Woods JH. Effects of pramipexole on the acquisition of responding with opioid-conditioned reinforcement in the rat. Psychopharmacology (Berl) 2015; 232:209-21. [PMID: 24985891 PMCID: PMC4282623 DOI: 10.1007/s00213-014-3659-2] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/25/2014] [Accepted: 06/02/2014] [Indexed: 12/13/2022]
Abstract
RATIONALE Dopamine D3 receptor-preferring ligands may be able to modify the conditioned reinforcing effects of drug-associated stimuli. In evaluating the effects of these compounds, it is important to clarify the extent to which responding depends on (1) conditioned reinforcement vs. other behavioral mechanisms and (2) dopamine D3 vs. D2 receptor activity. OBJECTIVES To use behaviorally stringent new-response acquisition procedures to characterize the effects of the D3-preferring agonist, pramipexole, on the conditioned reinforcing effects of a stimulus paired with the opioid agonist, remifentanil. METHODS First, in Pavlovian conditioning (PAV) sessions, rats received response-independent IV injections of remifentanil and presentations of a light-noise stimulus. In separate groups, injections and stimuli either always co-occurred ("paired PAV") or occurred with no consistent relationship ("random PAV" control). Next, in instrumental acquisition (ACQ) sessions, all animals could respond in two nose-poke manipulanda: an active nose-poke, which produced the stimulus alone, or an inactive nose-poke. Pramipexole was injected SC prior to ACQ sessions with or without pretreatments of the D3-preferring antagonist, SB-277011A, or the D2-preferring antagonist, L-741,626. RESULTS After paired PAV, but not random PAV, rats acquired nose-poke responding during ACQ (i.e., active > inactive). Pramipexole dose-dependently increased active responding without changing inactive responding. Pramipexole-induced increases in responding were blocked by pretreatment with L-741,626, but not SB-277011A. CONCLUSIONS Pramipexole specifically enhanced remifentanil-conditioned reinforcement: active responding was selectively increased only after the stimulus was paired with remifentanil. Although pramipexole is D3-preferring, the antagonist effects obtained presently suggest an important role for the D2 receptor in opioid-conditioned reinforcement.
Collapse
Affiliation(s)
- Jeremiah W. Bertz
- University of Michigan Medical School, Departments of Pharmacology, Ann Arbor, MI, USA
| | - Jianyong Chen
- University of Michigan Medical School, Departments of Internal Medicine, Ann Arbor, MI, USA
| | - James H. Woods
- University of Michigan Medical School, Departments of Pharmacology, Ann Arbor, MI, USA
| |
Collapse
|
16
|
Fernando ABP, Murray JE, Milton AL. The amygdala: securing pleasure and avoiding pain. Front Behav Neurosci 2013; 7:190. [PMID: 24367307 PMCID: PMC3854486 DOI: 10.3389/fnbeh.2013.00190] [Citation(s) in RCA: 53] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2013] [Accepted: 11/18/2013] [Indexed: 12/02/2022] Open
Abstract
The amygdala has traditionally been associated with fear, mediating the impact of negative emotions on memory. However, this view does not fully encapsulate the function of the amygdala, nor the impact that processing in this structure has on the motivational limbic corticostriatal circuitry of which it is an important structure. Here we discuss the interactions between different amygdala nuclei with cortical and striatal regions involved in motivation; interconnections and parallel circuitries that have become increasingly understood in recent years. We review the evidence that the amygdala stores memories that allow initially motivationally neutral stimuli to become associated through pavlovian conditioning with motivationally relevant outcomes which, importantly, can be either appetitive (e.g. food) or aversive (e.g. electric shock). We also consider how different psychological processes supported by the amygdala such as conditioned reinforcement and punishment, conditioned motivation and suppression, and conditioned approach and avoidance behavior, are not only psychologically but also neurobiologically dissociable, being mediated by distinct yet overlapping neural circuits within the limbic corticostriatal circuitry. Clearly the role of the amygdala goes beyond encoding aversive stimuli to also encode the appetitive, requiring an appreciation of the amygdala's mediation of both appetitive and fearful behavior through diverse psychological processes.
Collapse
Affiliation(s)
- Anushka B P Fernando
- Department of Psychology, Behavioural and Clinical Neuroscience Institute, University of Cambridge Cambridge, UK
| | - Jennifer E Murray
- Department of Psychology, Behavioural and Clinical Neuroscience Institute, University of Cambridge Cambridge, UK
| | - Amy L Milton
- Department of Psychology, Behavioural and Clinical Neuroscience Institute, University of Cambridge Cambridge, UK
| |
Collapse
|
17
|
Saunders BT, Robinson TE. Individual variation in resisting temptation: implications for addiction. Neurosci Biobehav Rev 2013; 37:1955-75. [PMID: 23438893 PMCID: PMC3732519 DOI: 10.1016/j.neubiorev.2013.02.008] [Citation(s) in RCA: 114] [Impact Index Per Article: 10.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2012] [Revised: 01/28/2013] [Accepted: 02/12/2013] [Indexed: 11/29/2022]
Abstract
When exposed to the sights, sounds, smells and/or places that have been associated with rewards, such as food or drugs, some individuals have difficulty resisting the temptation to seek out and consume them. Others have less difficulty restraining themselves. Thus, Pavlovian reward cues may motivate maladaptive patterns of behavior to a greater extent in some individuals than in others. We are just beginning to understand the factors underlying individual differences in the extent to which reward cues acquire powerful motivational properties, and therefore, the ability to act as incentive stimuli. Here we review converging evidence from studies in both human and non-human animals suggesting that a subset of individuals are more "cue reactive", in that certain reward cues are more likely to attract these individuals to them and motivate actions to get them. We suggest that those individuals for whom Pavlovian reward cues become especially powerful incentives may be more vulnerable to impulse control disorders, such as binge eating and addiction.
Collapse
Affiliation(s)
| | - Terry E. Robinson
- Department of Psychology (Biopsychology Program), University of Michigan
| |
Collapse
|
18
|
Acquisition of responding with a remifentanil-associated conditioned reinforcer in the rat. Psychopharmacology (Berl) 2013; 229:235-43. [PMID: 23609770 PMCID: PMC3757104 DOI: 10.1007/s00213-013-3102-0] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/02/2012] [Accepted: 03/29/2013] [Indexed: 10/26/2022]
Abstract
RATIONALE Drug-associated environmental stimuli may serve as conditioned reinforcers to enhance drug self-administration behaviors in humans and laboratory animals. However, it can be difficult to distinguish experimentally the conditioned reinforcing effects of a stimulus from other behavioral processes that can change rates of responding. OBJECTIVES To characterize the conditioned reinforcing effects of a stimulus paired with the μ-opioid agonist, remifentanil, using a new-response acquisition procedure in the rat. METHODS First, in Pavlovian conditioning (PAV) sessions, rats received response-independent IV injections of remifentanil and presentations of a light-noise compound stimulus. In paired PAV groups, injections and stimulus presentations always co-occurred. In random PAV control groups, injections and stimulus presentations occurred with no consistent relationship. Second, in instrumental acquisition (ACQ) sessions, all animals could respond in an active nose-poke that produced the stimulus alone or in an inactive nose-poke that had no scheduled consequences. RESULTS During ACQ, rats made significantly more active nose-pokes than inactive nose-pokes after paired PAV, but not after random PAV. Between groups, rats also made more active nose-pokes after paired PAV than after random PAV. After paired PAV, increased active responding was obtained under different schedules of reinforcement, persisted across multiple ACQ sessions, and depended on the number of PAV sessions conducted. CONCLUSIONS The remifentanil-paired stimulus served as a conditioned reinforcer for nose-poking: responding depended on both the contingency between the stimulus and remifentanil and the contingency between the nose-poke and the stimulus. Generally, new-response acquisition procedures may provide valid, flexible models for studying opioid-based conditioned reinforcement.
Collapse
|
19
|
Pujara M, Koenigs M. Mechanisms of reward circuit dysfunction in psychiatric illness: prefrontal-striatal interactions. Neuroscientist 2013; 20:82-95. [PMID: 23924665 DOI: 10.1177/1073858413499407] [Citation(s) in RCA: 34] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
The brain's "reward circuit" has been widely implicated in the pathophysiology of mental illness. Although there has been significant progress in identifying the functional characteristics of individual nodes within the circuit and linking dysfunction of these brain areas to various forms of psychopathology, there remains a substantial gap in understanding how the nodes of the circuit interact with one another, and how the growing neurobiological knowledge may be applied to improve psychiatric patient care. In this article, we summarize what is currently known about the functions and interactions of two key nodes of this circuit-the ventral striatum and the ventromedial prefrontal/orbital frontal cortex-in relation to mental illness.
Collapse
Affiliation(s)
- Maia Pujara
- 1Department of Psychiatry, University of Wisconsin-Madison, Madison, WI, USA
| | | |
Collapse
|
20
|
Nonkes LJP, Homberg JR. Perseverative instrumental and Pavlovian responding to conditioned stimuli in serotonin transporter knockout rats. Neurobiol Learn Mem 2012; 100:48-55. [PMID: 23261854 DOI: 10.1016/j.nlm.2012.12.004] [Citation(s) in RCA: 11] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2012] [Revised: 12/01/2012] [Accepted: 12/06/2012] [Indexed: 01/08/2023]
Abstract
Environmental stimuli can influence behavior via the process of Pavlovian conditioning. Recent genetic research suggests that some individuals are more sensitive to environmental stimuli for behavioral guidance than others. One important mediator of this effect is serotonin transporter (5-HTT) genetic variance, which increases sensitivity to Pavlovian conditioned stimuli through changes in the build-up of corticolimbic circuits. As these stimuli can have reinforcing effects on instrumental responding, we here investigated their effects on instrumental behavior in 5-HTT knockout rats and their wild-type counterparts by means of the signal attenuation paradigm. In this paradigm animals acquired a Pavlovian association between a stimulus and food reward, and subsequently they had to lever press in order to gain access to this food reward-associated stimulus. Thereafter, half of the animals underwent extinction training during which extinction of the primary Pavlovian association was induced via non-reinforced stimulus presentations, whereas the other half did not receive this training. During a final test session all animals were tested for instrumental responding for the non-reinforced Pavlovian conditioned stimulus, as well as instrumental and Pavlovian responding to the stimulus after an initial lever-press. No genotype differences were observed during the training and extinction sessions. However, during the test session 5-HTT knockout rats that had not received prior extinction training displayed excessive instrumental responding. This was specifically observed during presentation of the stimulus (induced by the first lever press) and was accompanied by an increased number of feeder visits after termination of the stimulus presentation. An additionally performed c-Fos immunohistochemistry study revealed that the behaviors in these animals were associated with abnormal c-Fos immunoreactivity in the orbitofrontal cortex and basolateral amygdala, regions important for the acquisition and maintenance of Pavlovian conditioned stimuli. These findings complement earlier findings showing that 5-HTT knockout animals' behavior is heavily influenced by environmental stimuli and indicate that this extends to the instrumental domain.
Collapse
Affiliation(s)
- Lourens J P Nonkes
- Donders Institute for Brain, Cognition, and Behaviour, Centre for Neuroscience, Department of Cognitive Neuroscience, Radboud University Nijmegen Medical Centre, The Netherlands
| | | |
Collapse
|
21
|
Schoenbaum G, Takahashi Y, Liu TL, McDannald MA. Does the orbitofrontal cortex signal value? Ann N Y Acad Sci 2012; 1239:87-99. [PMID: 22145878 DOI: 10.1111/j.1749-6632.2011.06210.x] [Citation(s) in RCA: 169] [Impact Index Per Article: 14.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
The orbitofrontal cortex (OFC) has long been implicated in associative learning. Early work by Mishkin and Rolls showed that the OFC was critical for rapid changes in learned behavior, a role that was reflected in the encoding of associative information by orbitofrontal neurons. Over the years, new data-particularly neurophysiological data-have increasingly emphasized the OFC in signaling actual value. These signals have been reported to vary according to internal preferences and judgments and to even be completely independent of the sensory qualities of predictive cues, the actual rewards, and the responses required to obtain them. At the same time, increasingly sophisticated behavioral studies have shown that the OFC is often unnecessary for simple value-based behavior and instead seems critical when information about specific outcomes must be used to guide behavior and learning. Here, we review these data and suggest a theory that potentially reconciles these two ideas, value versus specific outcomes, and bodies of work on the OFC.
Collapse
Affiliation(s)
- Geoffrey Schoenbaum
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, MD 21201, USA.
| | | | | | | |
Collapse
|
22
|
Morrison SE, Salzman CD. Representations of appetitive and aversive information in the primate orbitofrontal cortex. Ann N Y Acad Sci 2012; 1239:59-70. [PMID: 22145876 DOI: 10.1111/j.1749-6632.2011.06255.x] [Citation(s) in RCA: 41] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
Abstract
Individuals weigh information about both rewarding and aversive stimuli to make adaptive decisions. Most studies of the orbitofrontal cortex (OFC), an area where appetitive and aversive neural subsystems might interact, have focused only on reward. Using a classical conditioning task where novel stimuli are paired with a reward or an aversive air puff, we discovered that two groups of orbitofrontal neurons respond preferentially to conditioned stimuli associated with rewarding and aversive outcomes; however, information about appetitive and aversive stimuli converges on individual neurons from both populations. Therefore, neurons in the OFC might participate in appetitive and aversive networks that track the motivational significance of stimuli even when they vary in valence and sensory modality. Further, we show that these networks, which also extend to the amygdala, exhibit different rates of change during reversal learning. Thus, although both networks represent appetitive and aversive associations, their distinct temporal dynamics might indicate different roles in learning processes.
Collapse
Affiliation(s)
- Sara E Morrison
- Department of Neuroscience, Columbia University, New York, New York 10032, USA
| | | |
Collapse
|
23
|
Lucantonio F, Stalnaker TA, Shaham Y, Niv Y, Schoenbaum G. The impact of orbitofrontal dysfunction on cocaine addiction. Nat Neurosci 2012; 15:358-66. [PMID: 22267164 PMCID: PMC3701259 DOI: 10.1038/nn.3014] [Citation(s) in RCA: 149] [Impact Index Per Article: 12.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
Cocaine addiction is characterized by poor judgment and maladaptive decision-making. Here we review evidence implicating the orbitofrontal cortex in such behavior. This evidence suggests that cocaine-induced changes in orbitofrontal cortex disrupt the representation of states and transition functions that form the basis of flexible and adaptive 'model-based' behavioral control. By impairing this function, cocaine exposure leads to an overemphasis on less flexible, maladaptive 'model-free' control systems. We propose that such an effect accounts for the complex pattern of maladaptive behaviors associated with cocaine addiction.
Collapse
Affiliation(s)
- Federica Lucantonio
- Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, Maryland, USA
| | | | | | | | | |
Collapse
|
24
|
Ostrander S, Cazares VA, Kim C, Cheung S, Gonzalez I, Izquierdo A. Orbitofrontal cortex and basolateral amygdala lesions result in suboptimal and dissociable reward choices on cue-guided effort in rats. Behav Neurosci 2011; 125:350-9. [PMID: 21639604 DOI: 10.1037/a0023574] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The orbitofrontal cortex (OFC) and basolateral nucleus of the amygdala (BLA) are important neural regions in responding adaptively to changes in the incentive value of reward. Recent evidence suggests these structures may be differentially engaged in effort and cue-guided choice behavior. In 2 T-maze experiments, we examined the effects of bilateral lesions of either BLA or OFC on (1) effortful choices in which rats could climb a barrier for a high reward or select a low reward with no effort and (2) effortful choices when a visual cue signaled changes in reward magnitude. In both experiments, BLA rats displayed transient work aversion, choosing the effortless low reward option. OFC rats were work averse only in the no cue conditions, displaying a pattern of attenuated recovery from the cue conditions signaling reward unavailability in the effortful arm. Control measures rule out an inability to discriminate the cue in either lesion group.
Collapse
Affiliation(s)
- Serena Ostrander
- Laboratory of Cognitive Neuroscience, Department of Psychology, California State University, Los Angeles, 5151 State University Drive, Los Angeles, CA 90032, USA
| | | | | | | | | | | |
Collapse
|
25
|
Duka T, Crombag HS, Stephens DN. Experimental medicine in drug addiction: towards behavioral, cognitive and neurobiological biomarkers. J Psychopharmacol 2011; 25:1235-55. [PMID: 21169391 DOI: 10.1177/0269881110388324] [Citation(s) in RCA: 25] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Several theoretical frameworks have been developed to understand putative processes and mechanisms involved in addiction. Whilst these 'theories of addiction' disagree about importance and/or nature of a number of key psychological processes (e.g. the necessity of craving and/or the involvement of drug-value representations), a number of commonalities exist. For instance, it is widely accepted that Pavlovian associations between cues and environmental contexts and the drug effects acquired over the course of addiction play a critical role, especially in relapse vulnerability in detoxified addicts. Additionally, all theories of addiction (explicitly or implicitly) propose that chronic drug exposure produces persistent neuroplastic changes in neurobiological circuitries underlying critical emotional, cognitive and motivational processes, although disagreement exists as to the precise nature of these neurobiological changes and/or their psychological consequences. The present review, rather than limiting itself to any particular theoretical stance, considers various candidate psychological, neurobiological and/or behavioral processes in addiction and outlines conceptual and procedural approaches for the experimental medicine laboratory. The review discusses (1) extinction, renewal and (re)consolidation of learned associations between cues and drugs, (2) the drug reward value, (3) motivational states contributing to drug seeking and (4) reflective (top-down) and sensory (bottom-up) driven decision-making. In evaluating these psychological and/or behavioral processes and their relationship to addiction we make reference to putative underlying brain structures identified by basic animal studies and/or imaging studies with humans.
Collapse
Affiliation(s)
- Theodora Duka
- Behavioral and Clinical Neuroscience Research Group, School of Psychology, University of Sussex, Brighton, UK.
| | | | | |
Collapse
|
26
|
Agustín-Pavón C, Parkinson J, Man MS, Roberts AC. Contribution of the amygdala, but not orbitofrontal or medial prefrontal cortices, to the expression of flavour preferences in marmoset monkeys. Eur J Neurosci 2011; 34:1006-17. [PMID: 21848920 DOI: 10.1111/j.1460-9568.2011.07813.x] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
The development of food preferences contributes to a balanced diet, and involves both innate and learnt factors. By associating flavour cues with the reinforcing properties of the food (i.e. postingestive nutrient cues and innately preferred tastes, such as sweetness), animals acquire individual preferences. How the brain codes and guides selection when the subject has to choose between different palatable foods is little understood. To investigate this issue, we trained common marmoset monkeys (Callithrix jacchus) to respond to abstract visual patterns on a touch-sensitive computer screen to gain access to four different flavoured juices. After preferences were stable, animals received excitotoxic lesions of either the amygdala, the orbitofrontal cortex or the medial prefrontal cortex. Neither the orbitofrontal nor the medial prefrontal cortex lesions affected pre-surgery-expressed flavour preferences or the expression of preferences for novel flavours post-surgery. In contrast, amygdala lesions caused a shift in the preferences for juices expressed pre-surgery such that, post-surgery, juices were chosen according to their overall carbohydrate (simple sugars) content or 'sweetness'. Subsequent tests revealed that amygdala-lesioned animals only expressed juice preferences if they differed in 'sweetness'. Unlike controls, orbitofrontal cortex-lesioned and medial prefrontal cortex-lesioned animals, they were unable to display preferences between juices matched for 'sweetness' i.e. 5% sucrose solutions aromatised with different essential oils. The most parsimonious explanation is that the amygdala contributes to the expression of food preferences based on learnt cues but not those based on an innate preference for sweetness.
Collapse
Affiliation(s)
- Carmen Agustín-Pavón
- Department of Physiology, Development and Neuroscience, University of Cambridge, Downing Street, Cambridge CB2 3DY, UK
| | | | | | | |
Collapse
|
27
|
O'Connor EC, Stephens DN, Crombag HS. Modeling appetitive Pavlovian-instrumental interactions in mice. CURRENT PROTOCOLS IN NEUROSCIENCE 2011; Chapter 8:Unit 8.25. [PMID: 20938924 DOI: 10.1002/0471142301.ns0825s53] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
In appetitive Pavlovian associative learning, a stimulus (conditioned stimulus, CS) that has been associated with the delivery of a reinforcing event (unconditioned stimulus, US; e.g., food) can subsequently elicit or modulate goal-directed instrumental behaviors. For example, a Pavlovian CS can serve to reinforce (novel) instrumental behavior (conditioned reinforcement or CRf), or it can energize and potentiate ongoing instrumental responses when presented non-contingently (Pavlovian-instrumental transfer or PIT). Notably, these different effects of a Pavlovian CS on instrumental behavior are mediated by dissociable psychological and neurobiological mechanisms. Given the critical role that Pavlovian-instrumental interactions play in regulating motivated behavior and maladaptive manifestations of motivation such as eating disorders and addictions, understanding the underlying psychological and neurobiological mechanisms will be important. This unit describes behavioral protocols that produce robust and reliable PIT and CRf in mice and that open the door for future studies using transgenic approaches into the molecular mechanisms underlying associative learning and motivation.
Collapse
Affiliation(s)
- Eoin C O'Connor
- Behavioral and Clinical Neuroscience Research Group, School of Psychology, The University of Sussex, Brighton, East Sussex, United Kingdom
| | | | | |
Collapse
|
28
|
Suvak MK, Barrett LF. Considering PTSD from the perspective of brain processes: a psychological construction approach. J Trauma Stress 2011; 24:3-24. [PMID: 21298725 PMCID: PMC3141586 DOI: 10.1002/jts.20618] [Citation(s) in RCA: 47] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/01/2023]
Abstract
Posttraumatic stress disorder (PTSD) is a complex psychiatric disorder that involves symptoms from various domains that appear to be produced by the combination of several mechanisms. The authors contend that existing neural accounts fail to provide a viable model that explains the emergence and maintenance of PTSD and the associated heterogeneity in the expression of this disorder (cf. Garfinkel & Liberzon, 2009). They introduce a psychological construction approach as a novel framework to probe the brain basis of PTSD, where distributed networks within the human brain are thought to correspond to the basic psychological ingredients of the mind. The authors posit that it is the combination of these ingredients that produces the heterogeneous symptom clusters in PTSD. Their goal is show that a constructionist approach has significant heuristic value in understanding the emergence and maintenance of PTSD symptoms, and leads to different and perhaps more useful conjectures about the origins and maintenance of the syndrome than the traditional hyperreactive fear account.
Collapse
Affiliation(s)
- Michael K Suvak
- Women's Health Sciences Division, VA National Center for PTSD, VA Boston Healthcare System, USA
| | | |
Collapse
|
29
|
Singh T, McDannald MA, Haney RZ, Cerri DH, Schoenbaum G. Nucleus Accumbens Core and Shell are Necessary for Reinforcer Devaluation Effects on Pavlovian Conditioned Responding. Front Integr Neurosci 2010; 4:126. [PMID: 21088698 PMCID: PMC2981418 DOI: 10.3389/fnint.2010.00126] [Citation(s) in RCA: 34] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2010] [Accepted: 10/01/2010] [Indexed: 12/03/2022] Open
Abstract
The nucleus accumbens (NA) has been hypothesized to be part of a circuit in which cue-evoked information about expected outcomes is mobilized to guide behavior. Here we tested this hypothesis using a Pavlovian reinforcer devaluation task, previously applied to assess outcome-guided behavior after damage to regions such as the orbitofrontal cortex and amygdala that send projections to NA. Rats with sham lesions or neurotoxic lesions of either the core or shell subdivision of NA were trained to associate a 10-s CS+ with delivery of three food pellets. After training, half of the rats in each lesion group received food paired with illness induced by LiCl injections; the remaining rats received food and illness unpaired. Subsequently, responding to the CS+ was assessed in an extinction probe test. Both sham and lesioned rats conditioned to the CS+ and formed a conditioned taste aversion. However only sham rats reduced their conditioned responding as a result of reinforcer devaluation; devalued rats with lesions of either core or shell showed levels of responding that were similar to lesioned, non-devalued rats. This impairment was not due to the loss of motivational salience conferred to the CS+ in lesioned rats as both groups responded similarly for the cue in conditioned reinforcement testing. These data suggest that NA core and shell are part of a circuit necessary for the use of cue-evoked information about expected outcomes to guide behavior.
Collapse
Affiliation(s)
- Teghpal Singh
- Program in Neuroscience, University of Maryland, School of MedicineBaltimore, MD, USA
| | - Michael A. McDannald
- Department of Anatomy and Neurobiology, University of Maryland, School of MedicineBaltimore, MD, USA
| | - Richard Z. Haney
- Department of Anatomy and Neurobiology, University of Maryland, School of MedicineBaltimore, MD, USA
| | - Domenic H. Cerri
- Department of Anatomy and Neurobiology, University of Maryland, School of MedicineBaltimore, MD, USA
| | - Geoffrey Schoenbaum
- Program in Neuroscience, University of Maryland, School of MedicineBaltimore, MD, USA
- Department of Anatomy and Neurobiology, University of Maryland, School of MedicineBaltimore, MD, USA
- Department of Psychiatry, University of Maryland, School of MedicineBaltimore, MD, USA
| |
Collapse
|
30
|
The mGluR5 antagonist MTEP dissociates the acquisition of predictive and incentive motivational properties of reward-paired stimuli in mice. Neuropsychopharmacology 2010; 35:1807-17. [PMID: 20375996 PMCID: PMC3055484 DOI: 10.1038/npp.2010.48] [Citation(s) in RCA: 27] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
An environmental stimulus paired with reward (a conditioned stimulus; CS) can acquire predictive properties that signal reward availability and may also acquire incentive motivational properties that enable the CS to influence appetitive behaviors. The neural mechanisms involved in the acquisition and expression of these CS properties are not fully understood. The metabotropic glutamate receptor, mGluR5, contributes to synaptic plasticity underlying learning and memory processes. We examined the role of mGluR5 in the acquisition and expression of learning that enables a CS to predict reward (goal-tracking) and acquire incentive properties (conditioned reinforcement). Mice were injected with vehicle or the mGluR5 antagonist, MTEP (3 or 10 mg/kg), before each Pavlovian conditioning session in which a stimulus (CS+) was paired with food delivery. Subsequently, in the absence of the primary food reward, we determined whether the CS+ could reinforce a novel instrumental response (conditioned reinforcement) and direct behavior toward the place of reward delivery (goal-tracking). MTEP did not affect performance during the conditioning phase, or the ability of the CS+ to elicit a goal-tracking response. In contrast, 10 mg/kg MTEP given before each conditioning session prevented the subsequent expression of conditioned reinforcement. This dose of MTEP did not affect conditioned reinforcement when administered before the test, in mice that had received vehicle before conditioning sessions. Thus, mGluR5 has a critical role in the acquisition of incentive properties by a CS, but is not required for the expression of incentive learning, or for the CS to acquire predictive properties that signal reward availability.
Collapse
|
31
|
Phillips GD, Salussolia E, Hitchcott PK. Role of the mesoamygdaloid dopamine projection in emotional learning. Psychopharmacology (Berl) 2010; 210:303-16. [PMID: 20401751 DOI: 10.1007/s00213-010-1813-z] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/02/2009] [Accepted: 02/22/2010] [Indexed: 11/26/2022]
Abstract
RATIONALE Amygdala dopamine is crucially involved in the acquisition of Pavlovian associations, as measured via conditioned approach to the location of the unconditioned stimulus (US). However, learning begins before skeletomotor output, so this study assessed whether amygdala dopamine is also involved in earlier 'emotional' learning. OBJECTIVES A variant of the conditioned reinforcement (CR) procedure was validated where training was restricted to curtail the development of selective conditioned approach to the US location, and effects of amygdala dopamine manipulations before training or later CR testing assessed. METHODS Experiment 1a presented a light paired (CS+ group) or unpaired (CS- group) with a US. There were 1, 2 or 10 sessions, 4 trials per session. Then, the US was removed, and two novel levers presented. One lever (CR+) presented the light, and lever pressing was recorded. Experiment 1b also included a tone stimulus. Experiment 2 applied intra-amygdala R(+) 7-OH-DPAT (10 nmol/1.0 microl/side) before two training sessions (Experiment 2a) or a CR session (Experiment 2b). RESULTS For Experiments 1a and 1b, the CS+ group preferred the CR+ lever across all sessions. Conditioned alcove approach during 1 or 2 training sessions or associated CR tests was low and nonspecific. In Experiment 2a, R(+) 7-OH-DPAT before training greatly diminished lever pressing during a subsequent CR test, preferentially on the CR+ lever. For Experiment 2b, R(+) 7-OH-DPAT infusions before the CR test also reduced lever pressing. CONCLUSIONS Manipulations of amygdala dopamine impact the earliest stage of learning in which emotional reactions may be most prevalent.
Collapse
Affiliation(s)
- Gavin D Phillips
- Department of Psychology, University of York, Heslington, York, YO10 5DD, UK.
| | | | | |
Collapse
|
32
|
Milton AL, Everitt BJ. The psychological and neurochemical mechanisms of drug memory reconsolidation: implications for the treatment of addiction. Eur J Neurosci 2010; 31:2308-19. [PMID: 20497475 DOI: 10.1111/j.1460-9568.2010.07249.x] [Citation(s) in RCA: 170] [Impact Index Per Article: 12.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Memory reconsolidation is the process by which memories, destabilised at retrieval, require restabilisation to persist in the brain. It has been demonstrated that even old, well-established memories require reconsolidation following retrieval; therefore, memory reconsolidation could potentially be exploited to disrupt, or even erase, aberrant memories that underlie psychiatric disorders, thereby providing a novel therapeutic target. Drug addiction is one such disorder; it is both chronic and relapsing, and one prominent risk factor for a relapse episode is the presentation of environmental cues that have previously been associated with drugs of abuse. This 'cue-induced relapse' can be accounted for in psychological terms by reinforcing memories of the pavlovian association between the cue and the drug, which can thus influence behaviour through at least three psychologically and neurobiologically dissociable mechanisms: conditioned reinforcement, conditioned approach and conditioned motivation. As each of these psychological processes could contribute to the resumption of drug-seeking following abstinence, it is important to develop treatments that can reduce drug-seeking re-established via influences on each or all of these pavlovian processes, in order to minimise the risk of a subsequent relapse. Investigation of the memory reconsolidation mechanisms of the memories underlying conditioned reinforcement, conditioned approach and conditioned motivation indicate that they depend upon different neurochemical systems, including the glutamatergic and adrenergic systems within limbic corticostriatal circuitry. We also discuss here the subsequent translation to the clinic of this preclinical work.
Collapse
Affiliation(s)
- Amy L Milton
- Department of Experimental Psychology, Behavioural and Clinical Neuroscience Institute, University of Cambridge, Cambridge, UK.
| | | |
Collapse
|
33
|
Savage LM, Ramos RL. Reward expectation alters learning and memory: the impact of the amygdala on appetitive-driven behaviors. Behav Brain Res 2008; 198:1-12. [PMID: 19022299 DOI: 10.1016/j.bbr.2008.10.028] [Citation(s) in RCA: 39] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2008] [Accepted: 10/14/2008] [Indexed: 12/26/2022]
Abstract
The capacity to seek and obtain rewards is essential for survival. Pavlovian conditioning is one mechanism by which organisms develop predictions about rewards and such anticipatory or expectancy states enable successful behavioral adaptations to environmental demands. Reward expectancies have both affective/motivational and discriminative properties that allow for the modulation of instrumental goal-directed behavior. Recent data provide evidence that different cognitive strategies (cue-outcome associations) and neural systems (amygdala) are used when subjects are trained under conditions that allow Pavlovian-induced reward expectancies to guide instrumental behavioral choices. Furthermore, it has been demonstrated that impairments typically observed in a number of brain-damaged models are alleviated or eliminated by embedding unique reward expectancies into learning/memory tasks. These results suggest that Pavlovian-induced reward expectancies can change both behavioral and brain processes.
Collapse
Affiliation(s)
- Lisa M Savage
- Behavioral Neuroscience Program, Department of Psychology, Binghamton University-SUNY, Binghamton, NY 13902, USA.
| | | |
Collapse
|