1
|
Martel AC, Apicella P. Insights into the interaction between time and reward prediction on the activity of striatal tonically active neurons: A pilot study in rhesus monkeys. Physiol Rep 2024; 12:e70037. [PMID: 39245818 PMCID: PMC11381318 DOI: 10.14814/phy2.70037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2024] [Revised: 08/21/2024] [Accepted: 08/26/2024] [Indexed: 09/10/2024] Open
Abstract
Prior studies have documented the role of the striatum and its dopaminergic input in time processing, but the contribution of local striatal cholinergic innervation has not been specifically investigated. To address this issue, we recorded the activity of tonically active neurons (TANs), thought to be cholinergic interneurons in the striatum, in two male macaques performing self-initiated movements after specified intervals in the seconds range have elapsed. The behavioral data showed that movement timing was adjusted according to the temporal requirements. About one-third of all recorded TANs displayed brief depressions in firing in response to the cue that indicates the interval duration, and the strength of these modulations was, in some instances, related to the timing of movement. The rewarding outcome of actions also impacted TAN activity, as reflected by stronger responses to the cue paralleled by weaker responses to reward when monkeys performed correctly timed movements over consecutive trials. It therefore appears that TAN responses may act as a start signal for keeping track of time and reward prediction could be incorporated in this signaling function. We conclude that the role of the striatal cholinergic TAN system in time processing is embedded in predicting rewarding outcomes during timing behavior.
Collapse
Affiliation(s)
- A C Martel
- Institut de Neurosciences de la Timone, UMR 7289, Aix Marseille Université, CNRS, Marseille, France
| | - P Apicella
- Institut de Neurosciences de la Timone, UMR 7289, Aix Marseille Université, CNRS, Marseille, France
| |
Collapse
|
2
|
Brassard SL, Liu H, Dosanjh J, MacKillop J, Balodis I. Neurobiological foundations and clinical relevance of effort-based decision-making. Brain Imaging Behav 2024:10.1007/s11682-024-00890-x. [PMID: 38819540 DOI: 10.1007/s11682-024-00890-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/28/2024] [Indexed: 06/01/2024]
Abstract
Applying effort-based decision-making tasks provides insights into specific variables influencing choice behaviors. The current review summarizes the structural and functional neuroanatomy of effort-based decision-making. Across 39 examined studies, the review highlights the ventromedial prefrontal cortex in forming reward-based predictions, the ventral striatum encoding expected subjective values driven by reward size, the dorsal anterior cingulate cortex for monitoring choices to maximize rewards, and specific motor areas preparing for effort expenditure. Neuromodulation techniques, along with shifting environmental and internal states, are promising novel treatment interventions for altering neural alterations underlying decision-making. Our review further articulates the translational promise of this construct into the development, maintenance and treatment of psychiatric conditions, particularly those characterized by reward-, effort- and valuation-related deficits.
Collapse
Affiliation(s)
- Sarah L Brassard
- Neuroscience Graduate Program, McMaster University, Hamilton, ON, Canada
- Peter Boris Center for Addictions Research, St. Joseph's Healthcare Hamilton, Hamilton, ON, Canada
- Department of Psychiatry and Behavioural Neuroscience, McMaster University, Hamilton, ON, Canada
| | - Hanson Liu
- Peter Boris Center for Addictions Research, St. Joseph's Healthcare Hamilton, Hamilton, ON, Canada
- Department of Psychology, Neuroscience and Behaviour, McMaster University, Hamilton, ON, Canada
| | - Jadyn Dosanjh
- Peter Boris Center for Addictions Research, St. Joseph's Healthcare Hamilton, Hamilton, ON, Canada
- Department of Psychology, Neuroscience and Behaviour, McMaster University, Hamilton, ON, Canada
| | - James MacKillop
- Peter Boris Center for Addictions Research, St. Joseph's Healthcare Hamilton, Hamilton, ON, Canada
- Department of Psychiatry and Behavioural Neuroscience, McMaster University, Hamilton, ON, Canada
- Michael G. DeGroote Centre for Medicinal Cannabis Research, Hamilton, ON, Canada
- Department of Psychology, Neuroscience and Behaviour, McMaster University, Hamilton, ON, Canada
| | - Iris Balodis
- Peter Boris Center for Addictions Research, St. Joseph's Healthcare Hamilton, Hamilton, ON, Canada.
- Department of Psychiatry and Behavioural Neuroscience, McMaster University, Hamilton, ON, Canada.
- Michael G. DeGroote Centre for Medicinal Cannabis Research, Hamilton, ON, Canada.
| |
Collapse
|
3
|
Basile BM, Waters SJ, Murray EA. What does preferential viewing tell us about the neurobiology of recognition memory? Trends Neurosci 2024; 47:326-337. [PMID: 38582659 PMCID: PMC11096050 DOI: 10.1016/j.tins.2024.03.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 02/09/2024] [Accepted: 03/14/2024] [Indexed: 04/08/2024]
Abstract
The two tests most widely used in nonhuman primates to assess the neurobiology of recognition memory produce conflicting results. Preferential viewing tests (e.g., visual paired comparison) produce robust impairments following hippocampal lesions, whereas matching tests (e.g., delayed nonmatching-to-sample) often show complete sparing. Here, we review the data, the proposed explanations for this discrepancy, and then critically evaluate those explanations. The most likely explanation is that preferential viewing tests are not a process-pure assessment of recognition memory, but also test elements of novelty-seeking, habituation, and motivation. These confounds likely explain the conflicting results. Thus, we propose that memory researchers should prefer explicit matching tests and readers interested in the neural substrates of recognition memory should give explicit matching tests greater interpretive weight.
Collapse
Affiliation(s)
| | - Spencer J Waters
- Interdisciplinary Program in Neuroscience, Georgetown University, Washington, DC, USA; Section on the Neurobiology of Learning and Memory, Laboratory of Neuropsychology, National Institute of Mental Health, Bethesda, MD, USA
| | - Elisabeth A Murray
- Section on the Neurobiology of Learning and Memory, Laboratory of Neuropsychology, National Institute of Mental Health, Bethesda, MD, USA.
| |
Collapse
|
4
|
Gupta TA, Sanabria F. Motivated to time: Effects of reinforcer devaluation and opportunity cost on interval timing. Learn Behav 2023; 51:308-320. [PMID: 36781823 DOI: 10.3758/s13420-023-00572-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/22/2023] [Indexed: 02/15/2023]
Abstract
Prior research suggests that interval timing performance is sensitive to reinforcer devaluation effects and to the rate of competing sources of reinforcement. The present study sought to replicate and account for these findings in rats. A self-paced concurrent fixed-interval (FI) random-ratio (RR) schedule of reinforcement was implemented in which the FI requirement varied across training conditions (12, 24, 48 s). The RR requirement-which imposed an opportunity cost to responding on the FI component-was adjusted so that it took about twice the FI requirement, on average, to complete it. Probe reinforcer devaluation (prefeeding) sessions were conducted at the end of each condition. To assess the effect of contextual reinforcement on timing performance, the RR requirement was removed before the end of the experiment. Consistent with prior findings, performance on the FI component tracked schedule requirement and displayed scalar invariance; the removal of the RR component yielded more premature FI responses. For some rats, prefeeding reduced the number of trials initiated without affecting timing performance; for other rats, prefeeding delayed responding on the FI component but had a weaker effect on trial initiation. These results support the notion that timing and motivational processes are separable, suggesting novel explanations for ostensible motivational effects on timing performance.
Collapse
Affiliation(s)
- Tanya A Gupta
- Department of Psychology, Arizona State University, Tempe, AZ, USA.
- Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, CA, USA.
| | | |
Collapse
|
5
|
Speer SPH, Keysers C, Barrios JC, Teurlings CJS, Smidts A, Boksem MAS, Wager TD, Gazzola V. A multivariate brain signature for reward. Neuroimage 2023; 271:119990. [PMID: 36878456 DOI: 10.1016/j.neuroimage.2023.119990] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2022] [Revised: 02/20/2023] [Accepted: 02/25/2023] [Indexed: 03/07/2023] Open
Abstract
The processing of reinforcers and punishers is crucial to adapt to an ever changing environment and its dysregulation is prevalent in mental health and substance use disorders. While many human brain measures related to reward have been based on activity in individual brain regions, recent studies indicate that many affective and motivational processes are encoded in distributed systems that span multiple regions. Consequently, decoding these processes using individual regions yields small effect sizes and limited reliability, whereas predictive models based on distributed patterns yield larger effect sizes and excellent reliability. To create such a predictive model for the processes of rewards and losses, termed the Brain Reward Signature (BRS), we trained a model to predict the signed magnitude of monetary rewards on the Monetary Incentive Delay task (MID; N = 39) and achieved a highly significant decoding performance (92% for decoding rewards versus losses). We subsequently demonstrate the generalizability of our signature on another version of the MID in a different sample (92% decoding accuracy; N = 12) and on a gambling task from a large sample (73% decoding accuracy, N = 1084). We further provided preliminary data to characterize the specificity of the signature by illustrating that the signature map generates estimates that significantly differ between rewarding and negative feedback (92% decoding accuracy) but do not differ for conditions that differ in disgust rather than reward in a novel Disgust-Delay Task (N = 39). Finally, we show that passively viewing positive and negatively valenced facial expressions loads positively on our signature, in line with previous studies on morbid curiosity. We thus created a BRS that can accurately predict brain responses to rewards and losses in active decision making tasks, and that possibly relates to information seeking in passive observational tasks.
Collapse
Affiliation(s)
- Sebastian P H Speer
- Social Brain Lab, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands; Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Christian Keysers
- Social Brain Lab, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands; Brain and Cognition, Department of Psychology, University of Amsterdam, The Netherlands
| | | | - Cas J S Teurlings
- Social Brain Lab, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands
| | - Ale Smidts
- Rotterdam School of Management, Erasmus University, 3062 PA Rotterdam, The Netherlands
| | - Maarten A S Boksem
- Rotterdam School of Management, Erasmus University, 3062 PA Rotterdam, The Netherlands
| | - Tor D Wager
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755, USA
| | - Valeria Gazzola
- Social Brain Lab, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands.
| |
Collapse
|
6
|
Panfil K, Deavours A, Kirkpatrick K. Effects of the estrous cycle on impulsive choice and interval timing in female rats. Horm Behav 2023; 149:105315. [PMID: 36669427 PMCID: PMC9974800 DOI: 10.1016/j.yhbeh.2023.105315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Revised: 01/11/2023] [Accepted: 01/12/2023] [Indexed: 01/20/2023]
Abstract
Research in humans and animals shows differences in impulsive choice, which is a failure to wait for larger, delayed rewards, when comparing males and females. It is possible that fluctuations in sex hormones (estradiol and progesterone) across the reproductive cycle contribute to sex differences in impulsive choice. The current study delivered an impulsive choice task with peak interval trials to female rats while estrous cycles, the rodent reproductive cycle, were tracked over the course of the task. Female rats were more sensitive to changes in delay in the proestrus phase of the estrous cycle and made more larger-later choices when in estrus, particularly when the delay to the smaller reward was short. Estradiol increases dramatically during proestrus while progesterone peaks during estrus, suggesting that estradiol and progesterone may affect impulsive choice through mechanisms such as delay discounting, delay aversion, and/or timing processes. Analyses of timing of the choice task delays showed inconsistent effects of the estrous cycle across delays, suggesting that reward-timing interactions may have complicated how hormone fluctuations affected interval timing. Further research is needed to determine the mechanism underlying increased larger-later choices during the estrus phase, increased delay sensitivity during the proestrus phase, and variability in interval timing across delays and estrous cycle stages.
Collapse
Affiliation(s)
- Kelsey Panfil
- Department of Psychological Sciences, Kansas State University, Manhattan, KS 66506, United States of America.
| | - Aubrey Deavours
- Department of Psychological Sciences, Kansas State University, Manhattan, KS 66506, United States of America; Department of Diagnostic Medicine/Pathobiology, Kansas State University, Manhattan, KS 66506, United States of America
| | - Kimberly Kirkpatrick
- Department of Psychological Sciences, Kansas State University, Manhattan, KS 66506, United States of America
| |
Collapse
|
7
|
Gorka SM, Manzler CA, Jones EE, Smith RJ, Bryan CJ. Reward-related neural dysfunction in youth with a history of suicidal ideation: The importance of temporal predictability. J Psychiatr Res 2023; 158:20-26. [PMID: 36549196 DOI: 10.1016/j.jpsychires.2022.11.036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 09/09/2022] [Accepted: 11/12/2022] [Indexed: 12/13/2022]
Abstract
Abnormal reward processing is an important yet understudied risk factor for suicide. Recent neuroimaging studies have found that suicidality is associated with abnormal reward-related neural reactivity and connectivity across a wide range of brain regions and circuits. The varying, and oftentimes discrepant, findings have hindered progress in elucidating the neurobiological link between reward processing dysfunction and suicide risk. Some of this variability is likely related to different reward-related paradigms that are utilized across studies. The primary aim of the current study was to address these issues by comparing neural reactivity between youth with and without a history of suicidal ideation during direct manipulation of reward parameters. A total of 108 unmedicated youth, ages 17-19, were classified into two groups: 1) history of suicidal ideation (n = 39) and 2) no history of suicidal ideation (n = 69). All participants completed a novel reward anticipation task probing anticipation of predictable (P-reward) and unpredictable (U-reward) monetary reward. Results revealed that compared with controls, youth with a history of suicidal ideation exhibited increased neural activation in the dorsal anterior cingulate cortex (dACC) and right anterior insula (aINS) during anticipation of U-reward. There were no group differences during anticipation of P-reward. These findings suggest that propensity for suicidal ideation may be related to specific abnormalities during anticipation of U-reward, but not P-reward.
Collapse
Affiliation(s)
- Stephanie M Gorka
- Department of Psychiatry and Behavioral Health, The Ohio State University Wexner Medical Center, 370 W. 9th Avenue, Columbus, OH, 43210, USA; Institute for Behavioral Medicine Research, The Ohio State University, 460 Medical Center Drive, Columbus, OH, 43210, USA.
| | - Charles A Manzler
- Department of Psychiatry and Behavioral Health, The Ohio State University Wexner Medical Center, 370 W. 9th Avenue, Columbus, OH, 43210, USA; Institute for Behavioral Medicine Research, The Ohio State University, 460 Medical Center Drive, Columbus, OH, 43210, USA
| | - Emily E Jones
- Department of Psychiatry and Behavioral Health, The Ohio State University Wexner Medical Center, 370 W. 9th Avenue, Columbus, OH, 43210, USA; Institute for Behavioral Medicine Research, The Ohio State University, 460 Medical Center Drive, Columbus, OH, 43210, USA
| | - Reid J Smith
- Department of Psychiatry and Behavioral Health, The Ohio State University Wexner Medical Center, 370 W. 9th Avenue, Columbus, OH, 43210, USA; Institute for Behavioral Medicine Research, The Ohio State University, 460 Medical Center Drive, Columbus, OH, 43210, USA
| | - Craig J Bryan
- Department of Psychiatry and Behavioral Health, The Ohio State University Wexner Medical Center, 370 W. 9th Avenue, Columbus, OH, 43210, USA
| |
Collapse
|
8
|
Altınok S, Vatansever G, Apaydın N, Üstün S, Kale EH, Çelikağ İ, Devrimci-Özgüven H, Baskak B, Çiçek M. Reward Processing Alters the Time Perception Networks in Patients with Major Depressive Disorder. TIMING & TIME PERCEPTION 2023. [DOI: 10.1163/22134468-bja10073] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
Abstract
Behavioral studies revealed that time perception is affected by the presence of a reward. Both the experience of time and the reward processing were shown to be distorted in major depressive disorder (MDD). We aimed to investigate how neural correlates of time perception and reward anticipation interact in patients with MDD. Participants (17 healthy, seven MDD) performed a time perception task during fMRI scanning that requires estimating the speed of a moving rectangle which was briefly occluded. In the control condition, participants attended to the change in color tone of the rectangle. Half of the runs were rewarded with a monetary payment per correctly done trial to evaluate the effect of a reward. The fMRI data were acquired with a 3T scanner and analyzed with repeated-measures analysis of variance (ANOVA) using SPM12. The activations related to the integration of time with reward were different between both groups in the supplementary motor area, intraparietal sulcus, thalamus, frontal eye field and caudate nucleus. Increased coupling between supplementary motor area and caudate/putamen region during timing was found in MDD patients more than in controls. Overall, our findings suggest that functional differences related to the interaction of time perception with reward anticipation in MDD occur via dysfunction of the cortico-striatal-thalamic network.
Collapse
Affiliation(s)
- Simge Altınok
- Department of Interdisciplinary Neuroscience, Ankara University, Ankara, 06230 Turkey
- Brain Research Center, Ankara University, Ankara, 06340 Turkey
| | - Gözde Vatansever
- Department of Interdisciplinary Neuroscience, Ankara University, Ankara, 06230 Turkey
- Brain Research Center, Ankara University, Ankara, 06340 Turkey
- Neuroscience and Neurotechnology Center of Excellence (NÖROM), Ankara, 06560 Turkey
| | - Nihal Apaydın
- Department of Interdisciplinary Neuroscience, Ankara University, Ankara, 06230 Turkey
- Brain Research Center, Ankara University, Ankara, 06340 Turkey
- Neuroscience and Neurotechnology Center of Excellence (NÖROM), Ankara, 06560 Turkey
- Department of Anatomy, School of Medicine, Ankara University, Ankara, 06230 Turkey
| | - Sertaç Üstün
- Department of Interdisciplinary Neuroscience, Ankara University, Ankara, 06230 Turkey
- Brain Research Center, Ankara University, Ankara, 06340 Turkey
- Neuroscience and Neurotechnology Center of Excellence (NÖROM), Ankara, 06560 Turkey
- Department of Physiology, School of Medicine, Ankara University, Ankara, 06230 Turkey
| | - Emre H. Kale
- Brain Research Center, Ankara University, Ankara, 06340 Turkey
| | - İpek Çelikağ
- Brain Research Center, Ankara University, Ankara, 06340 Turkey
| | - Halise Devrimci-Özgüven
- Department of Interdisciplinary Neuroscience, Ankara University, Ankara, 06230 Turkey
- Brain Research Center, Ankara University, Ankara, 06340 Turkey
- Department of Psychiatry, School of Medicine, Ankara University, Ankara, 06590 Turkey
| | - Bora Baskak
- Department of Interdisciplinary Neuroscience, Ankara University, Ankara, 06230 Turkey
- Brain Research Center, Ankara University, Ankara, 06340 Turkey
- Neuroscience and Neurotechnology Center of Excellence (NÖROM), Ankara, 06560 Turkey
- Department of Psychiatry, School of Medicine, Ankara University, Ankara, 06590 Turkey
| | - Metehan Çiçek
- Department of Interdisciplinary Neuroscience, Ankara University, Ankara, 06230 Turkey
- Brain Research Center, Ankara University, Ankara, 06340 Turkey
- Neuroscience and Neurotechnology Center of Excellence (NÖROM), Ankara, 06560 Turkey
- Department of Physiology, School of Medicine, Ankara University, Ankara, 06230 Turkey
| |
Collapse
|
9
|
Tardelli VS, Berro LF, Gerra G, Tadonio L, Bisaga A, Fidalgo TM. Prescription psychostimulants for cocaine use disorder: A review from molecular basis to clinical approach. Addict Biol 2023; 28:e13271. [PMID: 37016755 PMCID: PMC10499006 DOI: 10.1111/adb.13271] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Revised: 12/20/2022] [Accepted: 02/13/2023] [Indexed: 03/05/2023]
Abstract
Cocaine use is a public health concern in many countries worldwide, particularly in the Americas and Oceania. Overdose deaths involving stimulants, such as cocaine, have been increasing markedly in North America, especially with concurrent opioid involvement. To date, no pharmacological treatment is available to treat stimulant (including cocaine) use disorders. Prescription psychostimulants (PPs) could be useful to treat cocaine use disorder (CUD) as they share the pharmacological effects with cocaine, as evidenced by a recent meta-analysis that assessed 38 randomized clinical trials (RCTs). PPs were found to promote sustained abstinence and reduce drug use in patients with CUD. The aim of this paper is to provide a narrative review of the clinical pharmacology of PPs and comment on the current stage of evidence supporting PPs to treat CUD. We also propose a model of care that integrates PPs with evidence-based psychosocial interventions (such as cognitive-behavioural therapy [CBT] and contingency management [CM]), a harm reduction approach and case management with social support.
Collapse
Affiliation(s)
- Vitor S. Tardelli
- Departamento de Psiquiatria, Universidade Federal de Sao Paulo (Unifesp), Sao Paulo, SP, Brazil
- Translational Addiction Research Laboratory, Center for Addiction and Mental Health, Toronto, ON, Canada
| | - Lais F. Berro
- Department of Psychiatry and Human Behavior, University of Mississipi Medical Center, Jackson, MS, USA
| | - Gilberto Gerra
- Mental Health Department, Azienda Unitá Sanitaria Locale, Parma, Italy
| | - Leonardo Tadonio
- Mental Health Department, Azienda Unitá Sanitaria Locale, Parma, Italy
| | - Adam Bisaga
- The Department of Psychiatry, Vagelos College of Physicians and Surgeons, Columbia University and the New York State Psychiatric Institute, New York, NY, USA
| | - Thiago M. Fidalgo
- Departamento de Psiquiatria, Universidade Federal de Sao Paulo (Unifesp), Sao Paulo, SP, Brazil
- Young Leaders Program from the National Academy of Medicine, Brazil
| |
Collapse
|
10
|
Pérez-Calzada M, Zamora-Arevalo O. Effects of reinforcement during the intertrial interval on temporal discrimination: Location version with rats. Front Behav Neurosci 2022; 16:956175. [PMID: 36248027 PMCID: PMC9561882 DOI: 10.3389/fnbeh.2022.956175] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 09/08/2022] [Indexed: 11/13/2022] Open
Abstract
Different studies on temporal control of behavior have focused on making modifications to experimental tasks by introducing disruptors to know how these manipulations modify temporal control. The aim of this study was to produce changes in temporal discrimination in a temporal bisection task by using a disruptor associated with motivation, which consisted in delivering reinforcement during the intertrial interval (RITI). Four Wistar rats and a pair of duration 2s−8s were used. There were two types of sessions: baseline generalization, where the disruptor was not applied, and RITI generalization, where the disruptive manipulation was applied. The analysis of results consisted of comparing psychophysical parameters, Signal Detection Theory indices, and latencies to start trials of baseline sessions and disruption sessions. The results showed a change in the point of subjective equality, a change in the psychophysical function, an increasing trend in the latencies to start trials on RITI disruption, and no change in the Signal Detection Theory indices. The results highlight the importance of incorporating motivational explanations to theories of temporal control in non-human organisms.
Collapse
|
11
|
Radoman M, Lieberman L, Jimmy J, Gorka SM. Shared and unique neural circuitry underlying temporally unpredictable threat and reward processing. Soc Cogn Affect Neurosci 2021; 16:370-382. [PMID: 33449089 PMCID: PMC7990065 DOI: 10.1093/scan/nsab006] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2019] [Revised: 11/20/2020] [Accepted: 01/14/2021] [Indexed: 11/14/2022] Open
Abstract
Temporally unpredictable stimuli influence behavior across species, as previously demonstrated for sequences of simple threats and rewards with fixed or variable onset. Neuroimaging studies have identified a specific frontolimbic circuit that may become engaged during the anticipation of temporally unpredictable threat (U-threat). However, the neural mechanisms underlying processing of temporally unpredictable reward (U-reward) are incompletely understood. It is also unclear whether these processes are mediated by overlapping or distinct neural systems. These knowledge gaps are noteworthy given that disruptions within these neural systems may lead to maladaptive response to uncertainty. Here, using functional magnetic resonance imaging data from a sample of 159 young adults, we showed that anticipation of both U-threat and U-reward elicited activation in the right anterior insula, right ventral anterior nucleus of the thalamus and right inferior frontal gyrus. U-threat also activated the right posterior insula and dorsal anterior cingulate cortex, relative to U-reward. In contrast, U-reward elicited activation in the right fusiform and left middle occipital gyrus, relative to U-threat. Although there is some overlap in the neural circuitry underlying anticipation of U-threat and U-reward, these processes appear to be largely mediated by distinct circuits. Future studies are needed to corroborate and extend these preliminary findings.
Collapse
Affiliation(s)
- Milena Radoman
- Department of Psychiatry, University of Illinois-Chicago, Chicago, IL 60612, USA.,Graduate Program in Neuroscience, University of Illinois-Chicago, Chicago, IL 60612, USA
| | - Lynne Lieberman
- Road Home Program, Rush University Medical Center, Chicago, IL 60612, USA
| | - Jagan Jimmy
- Department of Psychiatry, University of Illinois-Chicago, Chicago, IL 60612, USA
| | - Stephanie M Gorka
- Department of Psychiatry and Behavioral Health, Ohio State University, Columbus, OH 43205, USA
| |
Collapse
|
12
|
Mikhael JG, Lai L, Gershman SJ. Rational inattention and tonic dopamine. PLoS Comput Biol 2021; 17:e1008659. [PMID: 33760806 PMCID: PMC7990190 DOI: 10.1371/journal.pcbi.1008659] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2019] [Accepted: 12/28/2020] [Indexed: 11/27/2022] Open
Abstract
Slow-timescale (tonic) changes in dopamine (DA) contribute to a wide variety of processes in reinforcement learning, interval timing, and other domains. Furthermore, changes in tonic DA exert distinct effects depending on when they occur (e.g., during learning vs. performance) and what task the subject is performing (e.g., operant vs. classical conditioning). Two influential theories of tonic DA-the average reward theory and the Bayesian theory in which DA controls precision-have each been successful at explaining a subset of empirical findings. But how the same DA signal performs two seemingly distinct functions without creating crosstalk is not well understood. Here we reconcile the two theories under the unifying framework of 'rational inattention,' which (1) conceptually links average reward and precision, (2) outlines how DA manipulations affect this relationship, and in so doing, (3) captures new empirical phenomena. In brief, rational inattention asserts that agents can increase their precision in a task (and thus improve their performance) by paying a cognitive cost. Crucially, whether this cost is worth paying depends on average reward availability, reported by DA. The monotonic relationship between average reward and precision means that the DA signal contains the information necessary to retrieve the precision. When this information is needed after the task is performed, as presumed by Bayesian inference, acute manipulations of DA will bias behavior in predictable ways. We show how this framework reconciles a remarkably large collection of experimental findings. In reinforcement learning, the rational inattention framework predicts that learning from positive and negative feedback should be enhanced in high and low DA states, respectively, and that DA should tip the exploration-exploitation balance toward exploitation. In interval timing, this framework predicts that DA should increase the speed of the internal clock and decrease the extent of interference by other temporal stimuli during temporal reproduction (the central tendency effect). Finally, rational inattention makes the new predictions that these effects should be critically dependent on the controllability of rewards, that post-reward delays in intertemporal choice tasks should be underestimated, and that average reward manipulations should affect the speed of the clock-thus capturing empirical findings that are unexplained by either theory alone. Our results suggest that a common computational repertoire may underlie the seemingly heterogeneous roles of DA.
Collapse
Affiliation(s)
- John G. Mikhael
- Program in Neuroscience, Harvard Medical School, Boston, Massachusetts, United States of America
- MD-PhD Program, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Lucy Lai
- Program in Neuroscience, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Samuel J. Gershman
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, Massachusetts, United States of America
| |
Collapse
|
13
|
Balcı F, Freestone D. The Peak Interval Procedure in Rodents: A Tool for Studying the Neurobiological Basis of Interval Timing and Its Alterations in Models of Human Disease. Bio Protoc 2020; 10:e3735. [PMID: 33659396 PMCID: PMC7854006 DOI: 10.21769/bioprotoc.3735] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2020] [Revised: 06/15/2020] [Accepted: 06/17/2020] [Indexed: 11/02/2022] Open
Abstract
Animals keep track of time intervals in the seconds to minutes range with, on average, high accuracy but substantial trial-to-trial variability. The ability to detect the statistical signatures of such timing behavior is an indispensable feature of a good and theoretically-tractable testing procedure. A widely used interval timing procedure is the peak interval (PI) procedure, where animals learn to anticipate rewards that become available after a fixed delay. After learning, they cluster their responses around that reward-availability time. The in-depth analysis of such timed anticipatory responses leads to the understanding of an internal timing mechanism, that is, the processing dynamics and systematic biases of the brain's clock. This protocol explains in detail how the PI procedure can be implemented in rodents, from training through testing to analysis. We showcase both trial-by-trial and trial-averaged analytical methods as a window into these internal processes. This protocol has the advantages of capturing timing behavior in its full-complexity in a fashion that allows for a theoretical treatment of the data.
Collapse
Affiliation(s)
- Fuat Balcı
- Koç University, Department of Psychology, Istanbul, Turkey
| | - David Freestone
- William Paterson University, Department of Psychology, NJ, United States
| |
Collapse
|
14
|
Gupta TA, Daniels CW, Ortiz JB, Stephens M, Overby P, Romero K, Conrad CD, Sanabria F. The differential role of the dorsal hippocampus in initiating and terminating timed responses: A lesion study using the switch-timing task. Behav Brain Res 2019; 376:112184. [PMID: 31473282 DOI: 10.1016/j.bbr.2019.112184] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2019] [Revised: 08/24/2019] [Accepted: 08/28/2019] [Indexed: 11/27/2022]
Abstract
This study investigated the role of the dorsal hippocampus (dHPC) in the temporal entrainment of behavior, while addressing limitations of previous evidence from peak procedure experiments. Rats were first trained on a switch-timing task in which food was obtained from one of two concurrently available levers; one lever was effective after 8 s and the other after 16 s. After performance stabilized, rats underwent either bilateral NMDA lesions of the dHPC or sham lesions. After recovery, switch-timing training resumed. In a subsequent condition, the switch-timing task was modified such that food was available after either 8 or 32 s. Although dHPC lesions had subtle and complex effects on when rats stopped seeking for food at the 8-s lever (departures), it more systematically reduced the time when rats started seeking for food at the 16-s and 32-s lever (switches). No systematic effect of dHPC lesions were observed on the coefficient of quartile variation (normalized dispersion) of latencies to switch. Within the context of the pacemaker-accumulator framework of interval timing, these findings suggest that partially or wholly independent mechanisms control the initiation and termination of timed responses, and that the dHPC is primarily involved in encoding the time to start responding.
Collapse
Affiliation(s)
- Tanya A Gupta
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA.
| | - Carter W Daniels
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA; Columbia University, Department of Psychiatry, 1051 Riverside Drive, New York, NY, 10032, USA.
| | - J Bryce Ortiz
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA; The University of Arizona, College of Medicine - Phoenix, 475 N. 5th Street, Phoenix, AZ, 85004, USA.
| | - McAllister Stephens
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA; The University of Kentucky, Department of Psychology, 106-B Kastle Hall, Lexington, KY 40506-0044.
| | - Paula Overby
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA.
| | - Korinna Romero
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA; Arizona State University, College of Health Solutions, 550 N. 3rd Street, Phoenix, AZ, 85004-0698, USA.
| | - Cheryl D Conrad
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA.
| | - Federico Sanabria
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA.
| |
Collapse
|
15
|
Buriticá J, Alcalá E. Increased generalization in a peak procedure after delayed reinforcement. Behav Processes 2019; 169:103978. [DOI: 10.1016/j.beproc.2019.103978] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2019] [Revised: 09/13/2019] [Accepted: 09/23/2019] [Indexed: 11/30/2022]
|
16
|
Bayesian Behavioral Systems Theory. Behav Processes 2019; 168:103904. [DOI: 10.1016/j.beproc.2019.103904] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2019] [Revised: 06/30/2019] [Accepted: 07/08/2019] [Indexed: 12/29/2022]
|
17
|
Rudzinskas SA, Williams KM, Mong JA, Holder MK. Sex, Drugs, and the Medial Amygdala: A Model of Enhanced Sexual Motivation in the Female Rat. Front Behav Neurosci 2019; 13:203. [PMID: 31551730 PMCID: PMC6746834 DOI: 10.3389/fnbeh.2019.00203] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2019] [Accepted: 08/19/2019] [Indexed: 01/16/2023] Open
Abstract
Methamphetamine (METH) is a psychomotor stimulant that is reported to enhance sexual desire and behavior in both men and women, leading to increases in unplanned pregnancies, sexually-transmitted infections, and even comorbid psychiatric conditions. Here, we discuss our rodent model of increased sexually-motivated behaviors in which the co-administration of METH and the ovarian hormones, estradiol and progesterone, intensify the incentive properties of a sexual stimulus and increases measures of sexually-motivated behavior in the presence of an androgen-specific cue. We then present the neurobiological mechanisms by which this heightened motivational salience is mediated by the actions of METH and ovarian hormones, particularly progestins, in the posterodorsal medial nucleus of the amygdala (MePD), a key integration site for sexually-relevant sensory information with generalized arousal. We finally demonstrate the cellular and molecular mechanisms underlying this facilitation of sexual motivation by METH, including the upregulation, increased phosphorylation, and activation of progestin receptors (PRs) in the MePD by METH in the presence of ovarian hormones. Taken together, this work extends our understanding of the neurobiology of female sexual motivation.
Collapse
Affiliation(s)
- Sarah A Rudzinskas
- Program in Neuroscience, University of Maryland, Baltimore, MD, United States
| | - Katrina M Williams
- Department of Veterans Affairs, Geriatric Research Education and Clinical Center, Baltimore, MD, United States
| | - Jessica A Mong
- Program in Neuroscience, University of Maryland, Baltimore, MD, United States.,Department of Pharmacology, University of Maryland, Baltimore, MD, United States
| | - Mary K Holder
- School of Psychology, Georgia Institute of Technology, Atlanta, GA, United States
| |
Collapse
|
18
|
Zielinski MR, Systrom DM, Rose NR. Fatigue, Sleep, and Autoimmune and Related Disorders. Front Immunol 2019; 10:1827. [PMID: 31447842 PMCID: PMC6691096 DOI: 10.3389/fimmu.2019.01827] [Citation(s) in RCA: 111] [Impact Index Per Article: 22.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2019] [Accepted: 07/18/2019] [Indexed: 12/13/2022] Open
Abstract
Profound and debilitating fatigue is the most common complaint reported among individuals with autoimmune disease, such as systemic lupus erythematosus, multiple sclerosis, type 1 diabetes, celiac disease, chronic fatigue syndrome, and rheumatoid arthritis. Fatigue is multi-faceted and broadly defined, which makes understanding the cause of its manifestations especially difficult in conditions with diverse pathology including autoimmune diseases. In general, fatigue is defined by debilitating periods of exhaustion that interfere with normal activities. The severity and duration of fatigue episodes vary, but fatigue can cause difficulty for even simple tasks like climbing stairs or crossing the room. The exact mechanisms of fatigue are not well-understood, perhaps due to its broad definition. Nevertheless, physiological processes known to play a role in fatigue include oxygen/nutrient supply, metabolism, mood, motivation, and sleepiness-all which are affected by inflammation. Additionally, an important contributing element to fatigue is the central nervous system-a region impacted either directly or indirectly in numerous autoimmune and related disorders. This review describes how inflammation and the central nervous system contribute to fatigue and suggests potential mechanisms involved in fatigue that are likely exhibited in autoimmune and related diseases.
Collapse
Affiliation(s)
- Mark R Zielinski
- Veterans Affairs Boston Healthcare System, Boston, MA, United States.,Department of Psychiatry, Harvard Medical School, Boston, MA, United States
| | - David M Systrom
- Department of Medicine, Harvard Medical School, Boston, MA, United States.,Department of Pulmonary and Critical Care Medicine, Brigham and Women's Hospital, Boston, MA, United States
| | - Noel R Rose
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, United States
| |
Collapse
|
19
|
Smith T, Panfil K, Bailey C, Kirkpatrick K. Cognitive and behavioral training interventions to promote self-control. JOURNAL OF EXPERIMENTAL PSYCHOLOGY. ANIMAL LEARNING AND COGNITION 2019; 45:259-279. [PMID: 31070430 PMCID: PMC6716382 DOI: 10.1037/xan0000208] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
This review article discusses various cognitive and behavioral interventions that have been developed with the goal of promoting self-controlled responding. Self-control can exert a significant impact on human health and impulsive behaviors are associated with a wide range of diseases and disorders, leading to the suggestion that impulsivity is a trans-disease process. The self-control interventions include effort exposure, reward discrimination, reward bundling, interval schedules of reinforcement, impulse control training, and mindfulness training. Most of the interventions have been consistently shown to increase self-control, except for mindfulness training. Some of the successful interventions are long-lasting, whereas others may be transient. Most interventions are domain-specific, targeting specific cognitive and behavioral processes that relate to self-control rather than targeting overall self-control. For example, effort exposure appears to primarily increase effort tolerance, which in turn can improve self-control. Similarly, interval schedules primarily target interval timing, which promotes self-controlled responses. A diagram outlining a proposed set of intervention effects on self-control is introduced to motivate further research in this area. The diagram suggests that the individual target processes of the interventions may potentially summate to produce general self-control, or perhaps even produce synergistic effects. In addition, it is suggested that developing a self-control profile may be advantageous for aligning specific interventions to mitigate specific deficits. Overall, the results indicate that interventions are a promising avenue for promoting self-control and may help to contribute to changing health outcomes associated with a wide variety of diseases and disorders. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
Collapse
|
20
|
Mikhael JG, Gershman SJ. Adapting the flow of time with dopamine. J Neurophysiol 2019; 121:1748-1760. [PMID: 30864882 PMCID: PMC6589719 DOI: 10.1152/jn.00817.2018] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2018] [Revised: 02/04/2019] [Accepted: 02/20/2019] [Indexed: 01/25/2023] Open
Abstract
The modulation of interval timing by dopamine (DA) has been well established over decades of research. The nature of this modulation, however, has remained controversial: Although the pharmacological evidence has largely suggested that time intervals are overestimated with higher DA levels, more recent optogenetic work has shown the opposite effect. In addition, a large body of work has asserted DA's role as a "reward prediction error" (RPE), or a teaching signal that allows the basal ganglia to learn to predict future rewards in reinforcement learning tasks. Whether these two seemingly disparate accounts of DA may be related has remained an open question. By taking a reinforcement learning-based approach to interval timing, we show here that the RPE interpretation of DA naturally extends to its role as a modulator of timekeeping and furthermore that this view reconciles the seemingly conflicting observations. We derive a biologically plausible, DA-dependent plasticity rule that can modulate the rate of timekeeping in either direction and whose effect depends on the timing of the DA signal itself. This bidirectional update rule can account for the results from pharmacology and optogenetics as well as the behavioral effects of reward rate on interval timing and the temporal selectivity of striatal neurons. Hence, by adopting a single RPE interpretation of DA, our results take a step toward unifying computational theories of reinforcement learning and interval timing. NEW & NOTEWORTHY How does dopamine (DA) influence interval timing? A large body of pharmacological evidence has suggested that DA accelerates timekeeping mechanisms. However, recent optogenetic work has shown exactly the opposite effect. In this article, we relate DA's role in timekeeping to its most established role, as a critical component of reinforcement learning. This allows us to derive a neurobiologically plausible framework that reconciles a large body of DA's temporal effects, including pharmacological, behavioral, electrophysiological, and optogenetic.
Collapse
Affiliation(s)
- John G Mikhael
- Program in Neuroscience and MD-PhD Program, Harvard Medical School , Boston, Massachusetts
| | - Samuel J Gershman
- Center for Brain Science and Department of Psychology, Harvard University , Cambridge, Massachusetts
| |
Collapse
|
21
|
Derman RC, Schneider K, Juarez S, Delamater AR. Sign-tracking is an expectancy-mediated behavior that relies on prediction error mechanisms. Learn Mem 2018; 25:550-563. [PMID: 30224558 PMCID: PMC6149955 DOI: 10.1101/lm.047365.118] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2018] [Accepted: 06/29/2018] [Indexed: 01/04/2023]
Abstract
When discrete localizable stimuli are used during appetitive Pavlovian conditioning, "sign-tracking" and "goal-tracking" responses emerge. Sign-tracking is observed when conditioned responding is directed toward the CS, whereas goal-tracking manifests as responding directed to the site of expected reward delivery. These behaviors seem to rely on distinct, though overlapping neural circuitries, and, possibly, distinct psychological processes as well, and are thought to be related to addiction vulnerability. One currently popular view is that sign-tracking reflects an incentive motivational process, whereas goal-tracking reflects the influence of more top-down cognitive processes. To test these ideas, we used illness-induced outcome-devaluation and Kamin blocking procedures to determine whether these behaviors rely on similar or distinct underlying associative mechanisms. In Experiments 1 and 2 we showed that outcome-devaluation reduced sign-tracking responses, demonstrating that sign-tracking is controlled by reward expectancies. We also observed that post-CS goal-tracking in these animals is also devaluation sensitive. To test whether these two types of behaviors rely on similar or different prediction error mechanisms, we next tested whether Kamin blocking effects could be observed across these two classes of behaviors. In Experiment 3 we asked if sign-tracking to a lever CS could block the development of goal-tracking to a tone CS; whereas in Experiment 4, we examined whether goal-tracking to a tone CS could block sign-tracking to a lever CS. In both experiments blocking effects were observed suggesting that both sign- and goal-tracking emerge via a common prediction error mechanism. Collectively, the studies reported here suggest that the psychological mechanisms mediating sign- and goal-tracking are more similar than is commonly acknowledged.
Collapse
Affiliation(s)
- Rifka C Derman
- Department of Neuroscience Graduate Program, University of Michigan, Ann Arbor, Michigan, 48109, USA
| | - Kevin Schneider
- Neuroscience and Cognitive Science, University of Maryland, Maryland, 20742, USA
| | - Shaina Juarez
- Department of Applied Mathematics and Statistics, Stony Brook University, Stony Brook, New York 11794, USA
| | - Andrew R Delamater
- Department of Psychology, Brooklyn College and Graduate Center of the City University of New York, Brooklyn, New York 11210, USA
| |
Collapse
|
22
|
Daniels CW, Overby PF, Sanabria F. Between-session memory degradation accounts for within-session changes in fixed-interval performance. Behav Processes 2018; 153:31-39. [PMID: 29729953 DOI: 10.1016/j.beproc.2018.05.004] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2018] [Revised: 04/15/2018] [Accepted: 05/02/2018] [Indexed: 01/02/2023]
Abstract
A common assumption in the study of fixed-interval (FI) timing is that FI performance is largely stable within sessions, once it is stable between sessions. Within-session changes in FI performance were examined in published data (Daniels and Sanabria, 2017), wherein some rats were trained on a FI 30-s schedule of food reinforcement (FI30) and others on a FI 90-s schedule (FI90). Following stability, FI90 rats were pre-fed for five sessions. Response rates declined as a function of trial, due more to latency lengthening than to run-rate reduction. Latencies were best described by a dynamic gamma-exponential mixture distribution, in which latency lengthening was driven by the growth of the criterion pulse count for a response and not by a reduction in the speed of an endogenous clock. The speed of the clock was selectively sensitive to the length of the FI; the prevalence and length of exponentially-distributed latencies were selectively sensitive to pre-feeding. These findings reveal (a) that parameters governing FI latencies are selectively sensitive to a range of manipulations, (b) a potential degradation of the criterion pulse count between consecutive sessions, and (c) a subsequent recovery of the criterion pulse count within sessions.
Collapse
|
23
|
Interval timing under a behavioral microscope: Dissociating motivational and timing processes in fixed-interval performance. Learn Behav 2018; 45:29-48. [PMID: 27443193 DOI: 10.3758/s13420-016-0234-1] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The distribution of latencies and interresponse times (IRTs) of rats was compared between two fixed-interval (FI) schedules of food reinforcement (FI 30 s and FI 90 s), and between two levels of food deprivation. Computational modeling revealed that latencies and IRTs were well described by mixture probability distributions embodying two-state Markov chains. Analysis of these models revealed that only a subset of latencies is sensitive to the periodicity of reinforcement, and prefeeding only reduces the size of this subset. The distribution of IRTs suggests that behavior in FI schedules is organized in bouts that lengthen and ramp up in frequency with proximity to reinforcement. Prefeeding slowed down the lengthening of bouts and increased the time between bouts. When concatenated, latency and IRT models adequately reproduced sigmoidal FI response functions. These findings suggest that behavior in FI schedules fluctuates in and out of schedule control; an account of such fluctuation suggests that timing and motivation are dissociable components of FI performance. These mixture-distribution models also provide novel insights on the motivational, associative, and timing processes expressed in FI performance. These processes may be obscured, however, when performance in timing tasks is analyzed in terms of mean response rates.
Collapse
|
24
|
Oldham S, Murawski C, Fornito A, Youssef G, Yücel M, Lorenzetti V. The anticipation and outcome phases of reward and loss processing: A neuroimaging meta-analysis of the monetary incentive delay task. Hum Brain Mapp 2018; 39:3398-3418. [PMID: 29696725 PMCID: PMC6055646 DOI: 10.1002/hbm.24184] [Citation(s) in RCA: 261] [Impact Index Per Article: 43.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2017] [Revised: 03/28/2018] [Accepted: 04/09/2018] [Indexed: 12/19/2022] Open
Abstract
The processing of rewards and losses are crucial to everyday functioning. Considerable interest has been attached to investigating the anticipation and outcome phases of reward and loss processing, but results to date have been inconsistent. It is unclear if anticipation and outcome of a reward or loss recruit similar or distinct brain regions. In particular, while the striatum has widely been found to be active when anticipating a reward, whether it activates in response to the anticipation of losses as well remains ambiguous. Furthermore, concerning the orbitofrontal/ventromedial prefrontal regions, activation is often observed during reward receipt. However, it is unclear if this area is active during reward anticipation as well. We ran an Activation Likelihood Estimation meta‐analysis of 50 fMRI studies, which used the Monetary Incentive Delay Task (MIDT), to identify which brain regions are implicated in the anticipation of rewards, anticipation of losses, and the receipt of reward. Anticipating rewards and losses recruits overlapping areas including the striatum, insula, amygdala and thalamus, suggesting that a generalised neural system initiates motivational processes independent of valence. The orbitofrontal/ventromedial prefrontal regions were recruited only during the reward outcome, likely representing the value of the reward received. Our findings help to clarify the neural substrates of the different phases of reward and loss processing, and advance neurobiological models of these processes.
Collapse
Affiliation(s)
- Stuart Oldham
- Brain and Mental Health Research Hub, School of Psychological Sciences and the Monash Institute of Cognitive and Clinical Neurosciences (MICCN), Monash University, Clayton, Victoria, Australia
| | - Carsten Murawski
- Department of Finance, The University of Melbourne, Parkville, Victoria, Australia
| | - Alex Fornito
- Brain and Mental Health Research Hub, School of Psychological Sciences and the Monash Institute of Cognitive and Clinical Neurosciences (MICCN), Monash University, Clayton, Victoria, Australia
| | - George Youssef
- Cognitive Neuroscience Unit, School of Psychology, Deakin University, Geelong, Australia.,Centre for Adolescent Health, Murdoch Children's Research Institute, Parkville, Australia
| | - Murat Yücel
- Brain and Mental Health Research Hub, School of Psychological Sciences and the Monash Institute of Cognitive and Clinical Neurosciences (MICCN), Monash University, Clayton, Victoria, Australia
| | - Valentina Lorenzetti
- Brain and Mental Health Research Hub, School of Psychological Sciences and the Monash Institute of Cognitive and Clinical Neurosciences (MICCN), Monash University, Clayton, Victoria, Australia.,School of Psychology, Faculty of Health Sciences, Australian Catholic University, Fitzroy, Victoria, Australia.,Department of Psychological Sciences, Institute of Psychology Health and Society, University of Liverpool, Liverpool, United Kingdom
| |
Collapse
|
25
|
Delamater AR, Chen B, Nasser H, Elayouby K. Learning what to expect and when to expect it involves dissociable neural systems. Neurobiol Learn Mem 2018; 153:144-152. [PMID: 29477609 DOI: 10.1016/j.nlm.2018.02.026] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2017] [Revised: 01/18/2018] [Accepted: 02/21/2018] [Indexed: 11/16/2022]
Abstract
Two experiments with Long-Evans rats examined the potential independence of learning about different features of food reward, namely, "what" reward is to be expected and "when" it will occur. This was examined by investigating the effects of selective reward devaluation upon responding in an instrumental peak timing task in Experiment 1 and by exploring the effects of pre-training lesions targeting the basolateral amygdala (BLA) upon the selective reward devaluation effect and interval timing in a Pavlovian peak timing task in Experiment 2. In both tasks, two stimuli, each 60 s long, signaled that qualitatively distinct rewards (different flavored food pellets) could occur after 20 s. Responding on non-rewarded probe trials displayed the characteristic peak timing function with mean responding gradually increasing and peaking at approximately 20 s before more gradually declining thereafter. One of the rewards was then independently paired repeatedly with LiCl injections in order to devalue it whereas the other reward was unpaired with these injections. In a final set of test sessions in which both stimuli were presented without rewards, it was observed that responding was selectively reduced in the presence of the stimulus signaling the devalued reward compared to the stimulus signaling the still valued reward. Moreover, the timing function was mostly unaltered by this devaluation manipulation. Experiment 2 showed that pre-training BLA lesions abolished this selective reward devaluation effect, but it had no impact on peak timing functions shown by the two stimuli. It appears from these data that learning about "what" and "when" features of reward may entail separate underlying neural systems.
Collapse
Affiliation(s)
- Andrew R Delamater
- Brooklyn College and Graduate Center, City University of New York, United States.
| | - Brandon Chen
- Brooklyn College and Graduate Center, City University of New York, United States
| | - Helen Nasser
- Brooklyn College and Graduate Center, City University of New York, United States
| | - Karim Elayouby
- Brooklyn College and Graduate Center, City University of New York, United States
| |
Collapse
|
26
|
Maternal immune activation in rats produces temporal perception impairments in adult offspring analogous to those observed in schizophrenia. PLoS One 2017; 12:e0187719. [PMID: 29108010 PMCID: PMC5673217 DOI: 10.1371/journal.pone.0187719] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2017] [Accepted: 10/24/2017] [Indexed: 02/07/2023] Open
Abstract
The neurophysiology underlying temporal perception significantly overlaps with areas of dysfunction identified in schizophrenia. Patients commonly exhibit distorted temporal perception, which likely contributes to functional impairments. Thus, study of temporal perception in animal models of the disease may help to understand both cognitive and neurobiological factors involved in functional impairments in patients. As maternal immune activation (MIA) has been shown to be a significant etiological risk factor in development of schizophrenia and other developmental psychiatric diseases, we tested interval timing in a rat model of MIA that has previously been shown to recapitulate several behavioural and neurophysiological impairments observed in the disease. Rats were tested on a temporal-bisection task, in which temporal duration stimuli were categorized as either “short” or “long” by responding to a corresponding lever. Data from this task were modeled to provide estimates of accuracy and sensitivity of temporal perception. Parameter estimates derived from the model fitting showed that MIA rats significantly overestimated the passage of time compared to controls. These results indicate that the MIA rat paradigm recapitulates timing distortions that are phenotypical of schizophrenia. These findings lend further support to the epidemiological validity of this MIA rat model, supporting its relevance for future research into the role of maternal immune activation in producing neurobiological and behavioural impairments in schizophrenia.
Collapse
|
27
|
Buriticá J, dos Santos CV. Reinforcement value and fixed-interval performance. J Exp Anal Behav 2017; 108:151-170. [DOI: 10.1002/jeab.279] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2016] [Accepted: 08/03/2017] [Indexed: 11/12/2022]
Affiliation(s)
- Jonathan Buriticá
- Centro de Estudios e Investigaciones en Comportamiento; Universidad de Guadalajara
| | | |
Collapse
|
28
|
Morè L, Künnecke B, Yekhlef L, Bruns A, Marte A, Fedele E, Bianchi V, Taverna S, Gatti S, D'Adamo P. Altered fronto-striatal functions in the Gdi1-null mouse model of X-linked Intellectual Disability. Neuroscience 2017; 344:346-359. [PMID: 28057534 PMCID: PMC5315088 DOI: 10.1016/j.neuroscience.2016.12.043] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2016] [Revised: 12/05/2016] [Accepted: 12/23/2016] [Indexed: 01/17/2023]
Abstract
RAB-GDP dissociation inhibitor 1 (GDI1) loss-of-function mutations are responsible for a form of non-specific X-linked Intellectual Disability (XLID) where the only clinical feature is cognitive impairment. GDI1 patients are impaired in specific aspects of executive functions and conditioned response, which are controlled by fronto-striatal circuitries. Previous molecular and behavioral characterization of the Gdi1-null mouse revealed alterations in the total number/distribution of hippocampal and cortical synaptic vesicles as well as hippocampal short-term synaptic plasticity, and memory deficits. In this study, we employed cognitive protocols with high translational validity to human condition that target the functionality of cortico-striatal circuitry such as attention and stimulus selection ability with progressive degree of complexity. We previously showed that Gdi1-null mice are impaired in some hippocampus-dependent forms of associative learning assessed by aversive procedures. Here, using appetitive-conditioning procedures we further investigated associative learning deficits sustained by the fronto-striatal system. We report that Gdi1-null mice are impaired in attention and associative learning processes, which are a key part of the cognitive impairment observed in XLID patients.
Collapse
Affiliation(s)
- Lorenzo Morè
- Molecular Genetics of Intellectual Disability Unit, Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Basil Künnecke
- pRED, Pharma Research & Early Development, NORD Neuroscience, Roche Innovation Center Basel, F. Hoffmann-La Roche AG, Switzerland
| | - Latefa Yekhlef
- Neuroimmunology Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Andreas Bruns
- pRED, Pharma Research & Early Development, NORD Neuroscience, Roche Innovation Center Basel, F. Hoffmann-La Roche AG, Switzerland
| | - Antonella Marte
- Department of Pharmacy, Section of Pharmacology and Toxicology, Center of Excellence for Biomedical Research, University of Genoa, Genoa, Italy
| | - Ernesto Fedele
- Department of Pharmacy, Section of Pharmacology and Toxicology, Center of Excellence for Biomedical Research, University of Genoa, Genoa, Italy
| | - Veronica Bianchi
- Molecular Genetics of Intellectual Disability Unit, Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Stefano Taverna
- Neuroimmunology Unit, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Silvia Gatti
- pRED, Pharma Research & Early Development, NORD Neuroscience, Roche Innovation Center Basel, F. Hoffmann-La Roche AG, Switzerland
| | - Patrizia D'Adamo
- Molecular Genetics of Intellectual Disability Unit, Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy.
| |
Collapse
|
29
|
Akdoğan B, Balcı F. The effects of payoff manipulations on temporal bisection performance. Acta Psychol (Amst) 2016; 170:74-83. [PMID: 27380621 DOI: 10.1016/j.actpsy.2016.06.007] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2015] [Revised: 06/06/2016] [Accepted: 06/15/2016] [Indexed: 01/28/2023] Open
Abstract
There is growing evidence that alterations in reward rates modify timing behavior demonstrating the role of motivational factors in interval timing behavior. This study aimed to investigate the effects of manipulations of rewards and penalties on temporal bisection performance in humans. Participants were trained to classify experienced time intervals as short or long based on the reference durations. Two groups of participants were tested under three different bias conditions in which either the relative reward magnitude or penalty associated with correct or incorrect categorizations of short and long reference durations was manipulated. Participants adapted their choice behavior (i.e., psychometric functions shifted) based on these payoff manipulations in directions predicted by reward maximization. The signal detection theory-based analysis of the data revealed that payoff contingencies affected the response bias parameter (B″) without altering participants' sensitivity (A') to temporal distances. Finally, the response time (RT) analysis showed that short categorization RTs increased, whereas long categorization RTs decreased as a function of stimulus durations. However, overall RTs did not exhibit any modulation in response to payoff manipulations. Taken together, this study provides additional support for the effects of motivational variables on temporal decision-making.
Collapse
|
30
|
Buriticá J, Vilchez Z, Santos CVD. Temporal discrimination and delayed reinforcement. Behav Processes 2016; 130:71-4. [DOI: 10.1016/j.beproc.2016.07.009] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2016] [Revised: 07/11/2016] [Accepted: 07/13/2016] [Indexed: 12/01/2022]
|
31
|
Smith AP, Peterson JR, Kirkpatrick K. Reward Contrast Effects on Impulsive Choice and Timing in Rats. TIMING & TIME PERCEPTION 2016; 4:147-166. [PMID: 27867839 DOI: 10.1163/22134468-00002059] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
Abstract
Despite considerable interest in impulsive choice as a predictor of a variety of maladaptive behaviors, the mechanisms that drive choice behavior are still poorly understood. The present study sought to examine the influence of one understudied variable, reward magnitude contrast, on choice and timing behavior as changes in magnitude commonly occur within choice procedures. In addition, assessments of indirect effects on choice behavior through magnitude-timing interactions were assessed by measuring timing within the choice task. Rats were exposed to choice procedures composed of different pairs of magnitudes of rewards for either the smaller-sooner (SS) or larger-later (LL) option. In Phase 2, the magnitude of reward either increased or decreased by 1 pellet in different groups (LL increase = 1v1→1v2; SS decrease = 2v2 → 1v2; SS increase = 1v2 → 2v2), followed by a return to baseline in Phase 3. Choice behavior was affected by the initial magnitudes experienced in the task, demonstrating a strong anchor effect. The nature of the change in magnitude affected choice behavior as well. Timing behavior was also affected by the reward contrast manipulation albeit to a lesser degree and the timing and choice effects were correlated. The results suggest that models of choice behavior should incorporate reinforcement history, reward contrast elements, and magnitude-timing interactions, but that direct effects of reward contrast on choice should be given more weight than the indirect reward-timing interactions. A better understanding of the factors that contribute to choice behavior could supply key insights into this important individual differences variable.
Collapse
|
32
|
Marshall AT, Kirkpatrick K. Mechanisms of impulsive choice: III. The role of reward processes. Behav Processes 2015; 123:134-48. [PMID: 26506254 DOI: 10.1016/j.beproc.2015.10.013] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2015] [Revised: 10/16/2015] [Accepted: 10/17/2015] [Indexed: 02/07/2023]
Abstract
Two experiments examined the relationship between reward processing and impulsive choice. In Experiment 1, rats chose between a smaller-sooner (SS) reward (1 pellet, 10 s) and a larger-later (LL) reward (1, 2, and 4 pellets, 30 s). The rats then experienced concurrent variable-interval 30-s schedules with variations in reward magnitude to evaluate reward magnitude discrimination. LL choice behavior positively correlated with reward magnitude discrimination. In Experiment 2, rats chose between an SS reward (1 pellet, 10 s) and an LL reward (2 and 4 pellets, 30 s). The rats then received either a reward intervention which consisted of concurrent fixed-ratio schedules associated with different magnitudes to improve their reward magnitude discrimination, or a control task. All rats then experienced a post-intervention impulsive choice task followed by a reward magnitude discrimination task to assess intervention efficacy. The rats that received the intervention exhibited increases in post-intervention LL choice behavior, and made more responses for larger-reward magnitudes in the reward magnitude discrimination task, suggesting that the intervention heightened sensitivities to reward magnitude. The results suggest that reward magnitude discrimination plays a key role in individual differences in impulsive choice, and could be a potential target for further intervention developments.
Collapse
|
33
|
Marshall AT, Kirkpatrick K. Everywhere and everything: The power and ubiquity of time. INTERNATIONAL JOURNAL OF COMPARATIVE PSYCHOLOGY 2015; 28:http://escholarship.org/uc/item/8hg831n3. [PMID: 28392622 PMCID: PMC5382961] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/07/2023] Open
Abstract
Anticipatory timing plays a critical role in many aspects of human and non-human animal behavior. Timing has been consistently observed in the range of milliseconds to hours, and demonstrates a powerful influence on the organization of behavior. Anticipatory timing is acquired early in associative learning and appears to guide association formation in important ways. Importantly, timing participates in regulating goal-directed behaviors in many schedules of reinforcements, and plays a critical role in value-based decision making under concurrent schedules. In addition to playing a key role in fundamental learning processes, timing often dominates when temporal cues are available concurrently with other stimulus dimensions. Such control by the passage of time has even been observed when other cues provide more accurate information and can lead to sub-optimal behaviors. The dominance of temporal cues in governing anticipatory behavior suggests that time may be inherently more salient than many other stimulus dimensions. Discussions of the interface of the timing system with other cognitive processes are provided to demonstrate the powerful and primitive nature of time as a stimulus dimension.
Collapse
|
34
|
Fox AE, Kyonka EGE. Choice and timing in pigeons under differing levels of food deprivation. Behav Processes 2014; 106:82-90. [PMID: 24811449 DOI: 10.1016/j.beproc.2014.04.018] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2013] [Revised: 04/01/2014] [Accepted: 04/28/2014] [Indexed: 10/25/2022]
Abstract
State-dependent valuation learning (SDVL) is a preference for stimuli associated with relative food deprivation over stimuli associated with relative satiety. Pigeons were exposed to experimental conditions designed to investigate SDVL and to test the hypothesis that obtained relative immediacy during training predicts choice during test probes. Energy states were manipulated using a procedure that has previously revealed SDVL in starlings and pigeons. In Experiment 1, pigeons preferred the stimulus associated with deprivation in the first choice probe session, but were indifferent in the second. Changes in choice were consistent with changes in obtained relative immediacy. In Experiment 2, training parameters were altered and SDVL did not occur. Obtained relative immediacy again predicted choice. Results of both experiments provide evidence that obtained relative immediacy may be an important contributing factor to the SDVL phenomenon.
Collapse
Affiliation(s)
- Adam E Fox
- Department of Psychology, West Virginia University, PO Box 6040, Morgantown, WV 26056-6040, USA; Department of Psychology, St. Lawrence University, 23 Romoda Drive, Canton, NY 13617, USA.
| | - Elizabeth G E Kyonka
- Department of Psychology, West Virginia University, PO Box 6040, Morgantown, WV 26056-6040, USA.
| |
Collapse
|
35
|
Matell MS, Kurti AN. Reinforcement probability modulates temporal memory selection and integration processes. Acta Psychol (Amst) 2014; 147:80-91. [PMID: 23896560 DOI: 10.1016/j.actpsy.2013.06.006] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2012] [Revised: 05/28/2013] [Accepted: 06/17/2013] [Indexed: 11/25/2022] Open
Abstract
We have previously shown that rats trained in a mixed-interval peak procedure (tone=4s, light=12s) respond in a scalar manner at a time in between the trained peak times when presented with the stimulus compound (Swanton & Matell, 2011). In our previous work, the two component cues were reinforced with different probabilities (short=20%, long=80%) to equate response rates, and we found that the compound peak time was biased toward the cue with the higher reinforcement probability. Here, we examined the influence that different reinforcement probabilities have on the temporal location and shape of the compound response function. We found that the time of peak responding shifted as a function of the relative reinforcement probability of the component cues, becoming earlier as the relative likelihood of reinforcement associated with the short cue increased. However, as the relative probabilities of the component cues grew dissimilar, the compound peak became non-scalar, suggesting that the temporal control of behavior shifted from a process of integration to one of selection. As our previous work has utilized durations and reinforcement probabilities more discrepant than those used here, these data suggest that the processes underlying the integration/selection decision for time are based on cue value.
Collapse
|
36
|
Vitay J, Hamker FH. Timing and expectation of reward: a neuro-computational model of the afferents to the ventral tegmental area. Front Neurorobot 2014; 8:4. [PMID: 24550821 PMCID: PMC3907710 DOI: 10.3389/fnbot.2014.00004] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2013] [Accepted: 01/15/2014] [Indexed: 12/24/2022] Open
Abstract
Neural activity in dopaminergic areas such as the ventral tegmental area is influenced by timing processes, in particular by the temporal expectation of rewards during Pavlovian conditioning. Receipt of a reward at the expected time allows to compute reward-prediction errors which can drive learning in motor or cognitive structures. Reciprocally, dopamine plays an important role in the timing of external events. Several models of the dopaminergic system exist, but the substrate of temporal learning is rather unclear. In this article, we propose a neuro-computational model of the afferent network to the ventral tegmental area, including the lateral hypothalamus, the pedunculopontine nucleus, the amygdala, the ventromedial prefrontal cortex, the ventral basal ganglia (including the nucleus accumbens and the ventral pallidum), as well as the lateral habenula and the rostromedial tegmental nucleus. Based on a plausible connectivity and realistic learning rules, this neuro-computational model reproduces several experimental observations, such as the progressive cancelation of dopaminergic bursts at reward delivery, the appearance of bursts at the onset of reward-predicting cues or the influence of reward magnitude on activity in the amygdala and ventral tegmental area. While associative learning occurs primarily in the amygdala, learning of the temporal relationship between the cue and the associated reward is implemented as a dopamine-modulated coincidence detection mechanism in the nucleus accumbens.
Collapse
Affiliation(s)
- Julien Vitay
- Department of Computer Science, Chemnitz University of Technology Chemnitz, Germany
| | - Fred H Hamker
- Department of Computer Science, Chemnitz University of Technology Chemnitz, Germany ; Bernstein Center for Computational Neuroscience, Charité University Medicine Berlin, Germany
| |
Collapse
|
37
|
Abstract
The dopamine clock hypothesis suggests that the dopamine level determines the speed of the hypothetical internal clock. However, dopaminergic function has also been implicated for motivation and thus the effect of dopaminergic manipulations on timing behavior might also be independently mediated by altered motivational state. Studies that investigated the effect of motivational manipulations on peak responding are reviewed in this paper. The majority of these studies show that a higher reward magnitude leads to a leftward shift, whereas reward devaluation leads to a rightward shift in the initiation of timed anticipatory behavior, typically in the absence of an effect on the timing of response termination. Similar behavioral effects are also present in a number of studies that investigated the effect of dopamine agonists and dopamine-related genetic factors on peak responding. These results can be readily accounted for by independent modulation of decision-thresholds for the initiation and termination of timed responding.
Collapse
Affiliation(s)
- Fuat Balcı
- Department of Psychology, Koç University, Rumelifeneri yolu, Sarıyer, Istanbul, Turkey
| |
Collapse
|
38
|
Matell MS. Searching for the holy grail: temporally informative firing patterns in the rat. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2014; 829:209-34. [PMID: 25358713 DOI: 10.1007/978-1-4939-1782-2_12] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
This chapter reviews our work from the past decade investigating cortical and striatal firing patterns in rats while they time intervals in the multi-seconds range. We have found that both cortical and striatal firing rates contain information that the rat can use to identify how much time has elapsed both from trial onset and from the onset of an active response state. I describe findings showing that the striatal neurons that are modulated by time are also modulated by overt behaviors, suggesting that time modulates the strength of motor coding in the striatum, rather than being represented as an abstract quantity in isolation. I also describe work showing that there are a variety of temporally informative activity patterns in pre-motor cortex, and argue that the heterogeneity of these patterns can enhance an organism's temporal estimate. Finally, I describe recent behavioral work from my lab in which the simultaneous cueing of multiple durations leads to a scalar temporal expectation at an intermediate time, providing strong support for a monotonic representation of time.
Collapse
|
39
|
Heilbronner SR, Meck WH. Dissociations between interval timing and intertemporal choice following administration of fluoxetine, cocaine, or methamphetamine. Behav Processes 2014; 101:123-34. [PMID: 24135569 PMCID: PMC4081038 DOI: 10.1016/j.beproc.2013.09.013] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2013] [Revised: 09/19/2013] [Accepted: 09/21/2013] [Indexed: 12/26/2022]
Abstract
The goal of our study was to characterize the relationship between intertemporal choice and interval timing, including determining how drugs that modulate brain serotonin and dopamine levels influence these two processes. In Experiment 1, rats were tested on a standard 40-s peak-interval procedure following administration of fluoxetine (3, 5, or 8 mg/kg) or vehicle to assess basic effects on interval timing. In Experiment 2, rats were tested in a novel behavioral paradigm intended to simultaneously examine interval timing and impulsivity. Rats performed a variant of the bi-peak procedure using 10-s and 40-s target durations with an additional "defection" lever that provided the possibility of a small, immediate reward. Timing functions remained relatively intact, and 'patience' across subjects correlated with peak times, indicating a negative relationship between 'patience' and clock speed. We next examined the effects of fluoxetine (5 mg/kg), cocaine (15 mg/kg), or methamphetamine (1 mg/kg) on task performance. Fluoxetine reduced impulsivity as measured by defection time without corresponding changes in clock speed. In contrast, cocaine and methamphetamine both increased impulsivity and clock speed. Thus, variations in timing may mediate intertemporal choice via dopaminergic inputs. However, a separate, serotonergic system can affect intertemporal choice without affecting interval timing directly. This article is part of a Special Issue entitled: Associative and Temporal Learning.
Collapse
Affiliation(s)
- Sarah R Heilbronner
- Department of Pharmacology & Physiology, University of Rochester Medical Center, Rochester, NY 14642, USA
| | - Warren H Meck
- Department of Psychology and Neuroscience, Duke University, Durham, NC 27708, USA.
| |
Collapse
|
40
|
Kirkpatrick K. Interactions of timing and prediction error learning. Behav Processes 2014; 101:135-45. [PMID: 23962670 PMCID: PMC3926915 DOI: 10.1016/j.beproc.2013.08.005] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2013] [Revised: 06/24/2013] [Accepted: 08/06/2013] [Indexed: 11/28/2022]
Abstract
Timing and prediction error learning have historically been treated as independent processes, but growing evidence has indicated that they are not orthogonal. Timing emerges at the earliest time point when conditioned responses are observed, and temporal variables modulate prediction error learning in both simple conditioning and cue competition paradigms. In addition, prediction errors, through changes in reward magnitude or value alter timing of behavior. Thus, there appears to be a bi-directional interaction between timing and prediction error learning. Modern theories have attempted to integrate the two processes with mixed success. A neurocomputational approach to theory development is espoused, which draws on neurobiological evidence to guide and constrain computational model development. Heuristics for future model development are presented with the goal of sparking new approaches to theory development in the timing and prediction error fields.
Collapse
|
41
|
Luzardo A, Ludvig EA, Rivest F. An adaptive drift-diffusion model of interval timing dynamics. Behav Processes 2013; 95:90-9. [PMID: 23428705 DOI: 10.1016/j.beproc.2013.02.003] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2012] [Revised: 02/01/2013] [Accepted: 02/06/2013] [Indexed: 11/26/2022]
Abstract
Animals readily learn the timing between salient events. They can even adapt their timed responding to rapidly changing intervals, sometimes as quickly as a single trial. Recently, drift-diffusion models-widely used to model response times in decision making-have been extended with new learning rules that allow them to accommodate steady-state interval timing, including scalar timing and timescale invariance. These time-adaptive drift-diffusion models (TDDMs) work by accumulating evidence of elapsing time through their drift rate, thereby encoding the to-be-timed interval. One outstanding challenge for these models lies in the dynamics of interval timing-when the to-be-timed intervals are non-stationary. On these schedules, animals often fail to exhibit strict timescale invariance, as expected by the TDDMs and most other timing models. Here, we introduce a simple extension to these TDDMs, where the response threshold is a linear function of the observed event rate. This new model compares favorably against the basic TDDMs and the multiple-time-scale (MTS) habituation model when evaluated against three published datasets on timing dynamics in pigeons. Our results suggest that the threshold for triggering responding in interval timing changes as a function of recent intervals.
Collapse
|
42
|
Balcı F, Wiener M, Çavdaroğlu B, Branch Coslett H. Epistasis effects of dopamine genes on interval timing and reward magnitude in humans. Neuropsychologia 2013; 51:293-308. [DOI: 10.1016/j.neuropsychologia.2012.08.002] [Citation(s) in RCA: 40] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2012] [Revised: 08/03/2012] [Accepted: 08/03/2012] [Indexed: 12/01/2022]
|