1
|
Carter F, Cossette MP, Trujillo-Pisanty I, Pallikaras V, Breton YA, Conover K, Caplan J, Solis P, Voisard J, Yaksich A, Shizgal P. Does phasic dopamine release cause policy updates? Eur J Neurosci 2024; 59:1260-1277. [PMID: 38039083 DOI: 10.1111/ejn.16199] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2023] [Revised: 10/21/2023] [Accepted: 11/01/2023] [Indexed: 12/03/2023]
Abstract
Phasic dopamine activity is believed to both encode reward-prediction errors (RPEs) and to cause the adaptations that these errors engender. If so, a rat working for optogenetic stimulation of dopamine neurons will repeatedly update its policy and/or action values, thus iteratively increasing its work rate. Here, we challenge this view by demonstrating stable, non-maximal work rates in the face of repeated optogenetic stimulation of midbrain dopamine neurons. Furthermore, we show that rats learn to discriminate between world states distinguished only by their history of dopamine activation. Comparison of these results to reinforcement learning simulations suggests that the induced dopamine transients acted more as rewards than RPEs. However, pursuit of dopaminergic stimulation drifted upwards over a time scale of days and weeks, despite its stability within trials. To reconcile the results with prior findings, we consider multiple roles for dopamine signalling.
Collapse
Affiliation(s)
- Francis Carter
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
- Montreal Institute for Learning Algorithms, Université de Montréal, Montreal, Quebec, Canada
| | | | - Ivan Trujillo-Pisanty
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
- Department of Psychology, Langara College, Vancouver, British Columbia, Canada
| | | | | | - Kent Conover
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
| | - Jill Caplan
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
| | - Pavel Solis
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
| | - Jacques Voisard
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
| | - Alexandra Yaksich
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
| | - Peter Shizgal
- Department of Psychology, Concordia University, Montreal, Quebec, Canada
| |
Collapse
|
2
|
Sennesh E, Theriault J, Brooks D, van de Meent JW, Barrett LF, Quigley KS. Interoception as modeling, allostasis as control. Biol Psychol 2021; 167:108242. [PMID: 34942287 DOI: 10.1016/j.biopsycho.2021.108242] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2021] [Revised: 12/13/2021] [Accepted: 12/14/2021] [Indexed: 01/09/2023]
Abstract
The brain regulates the body by anticipating its needs and attempting to meet them before they arise - a process called allostasis. Allostasis requires a model of the changing sensory conditions within the body, a process called interoception. In this paper, we examine how interoception may provide performance feedback for allostasis. We suggest studying allostasis in terms of control theory, reviewing control theory's applications to related issues in physiology, motor control, and decision making. We synthesize these by relating them to the important properties of allostatic regulation as a control problem. We then sketch a novel formalism for how the brain might perform allostatic control of the viscera by analogy to skeletomotor control, including a mathematical view on how interoception acts as performance feedback for allostasis. Finally, we suggest ways to test implications of our hypotheses.
Collapse
Affiliation(s)
- Eli Sennesh
- Northeastern University, Boston, MA , United States.
| | | | - Dana Brooks
- Northeastern University, Boston, MA , United States
| | | | | | | |
Collapse
|
3
|
Budhota A, Chua KSG, Hussain A, Kager S, Cherpin A, Contu S, Vishwanath D, Kuah CWK, Ng CY, Yam LHL, Loh YJ, Rajeswaran DK, Xiang L, Burdet E, Campolo D. Robotic Assisted Upper Limb Training Post Stroke: A Randomized Control Trial Using Combinatory Approach Toward Reducing Workforce Demands. Front Neurol 2021; 12:622014. [PMID: 34149587 PMCID: PMC8206540 DOI: 10.3389/fneur.2021.622014] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2020] [Accepted: 04/23/2021] [Indexed: 01/31/2023] Open
Abstract
Post stroke upper limb rehabilitation is a challenging problem with poor outcomes as 40% of survivors have functionally useless upper limbs. Robot-aided therapy (RAT) is a potential method to alleviate the effort of intensive, task-specific, repetitive upper limb exercises for both patients and therapists. The present study aims to investigate how a time matched combinatory training scheme that incorporates conventional and RAT, using H-Man, compares with conventional training toward reducing workforce demands. In a randomized control trial (NCT02188628, www.clinicaltrials.gov), 44 subacute to chronic stroke survivors with first-ever clinical stroke and predominant arm motor function deficits were recruited and randomized into two groups of 22 subjects: Robotic Therapy (RT) and Conventional Therapy (CT). Both groups received 18 sessions of 90 min; three sessions per week over 6 weeks. In each session, participants of the CT group received 90 min of 1:1 therapist-supervised conventional therapy while participants of the RT group underwent combinatory training which consisted of 60 min of minimally-supervised H-Man therapy followed by 30 min of conventional therapy. The clinical outcomes [Fugl-Meyer (FMA), Action Research Arm Test and, Grip Strength] and the quantitative measures (smoothness, time efficiency, and task error, derived from two robotic assessment tasks) were independently evaluated prior to therapy intervention (week 0), at mid-training (week 3), at the end of training (week 6), and post therapy (week 12 and 24). Significant differences within group were observed at the end of training for all clinical scales compared with baseline [mean and standard deviation of FMA score changes between baseline and week 6; RT: Δ4.41 (3.46) and CT: Δ3.0 (4.0); p < 0.01]. FMA gains were retained 18 weeks post-training [week 24; RT: Δ5.38 (4.67) and week 24 CT: Δ4.50 (5.35); p < 0.01]. The RT group clinical scores improved similarly when compared to CT group with no significant inter-group at all time points although the conventional therapy time was reduced to one third in RT group. There were no training-related adverse side effects. In conclusion, time matched combinatory training incorporating H-Man RAT produced similar outcomes compared to conventional therapy alone. Hence, this study supports a combinatory approach to improve motor function in post-stroke arm paresis. Clinical Trial Registration: www.ClinicalTrials.gov, identifier: NCT02188628.
Collapse
Affiliation(s)
- Aamani Budhota
- Interdisciplinary Graduate School, Nanyang Technological University, Singapore, Singapore.,Robotic Research Center, Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, Singapore
| | - Karen S G Chua
- Centre for Advanced Rehabilitation Therapeutics, Tan Tock Seng Hospital Rehabilitation Centre, Singapore, Singapore
| | - Asif Hussain
- Robotic Research Center, Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, Singapore
| | - Simone Kager
- NUS Graduate School for Integrative Sciences and Engineering, National University of Singapore, Singapore, Singapore
| | - Adèle Cherpin
- Robotic Research Center, Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, Singapore
| | - Sara Contu
- Robotic Research Center, Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, Singapore
| | - Deshmukh Vishwanath
- Centre for Advanced Rehabilitation Therapeutics, Tan Tock Seng Hospital Rehabilitation Centre, Singapore, Singapore
| | - Christopher W K Kuah
- Centre for Advanced Rehabilitation Therapeutics, Tan Tock Seng Hospital Rehabilitation Centre, Singapore, Singapore
| | - Chwee Yin Ng
- Centre for Advanced Rehabilitation Therapeutics, Tan Tock Seng Hospital Rehabilitation Centre, Singapore, Singapore
| | - Lester H L Yam
- Centre for Advanced Rehabilitation Therapeutics, Tan Tock Seng Hospital Rehabilitation Centre, Singapore, Singapore
| | - Yong Joo Loh
- Centre for Advanced Rehabilitation Therapeutics, Tan Tock Seng Hospital Rehabilitation Centre, Singapore, Singapore
| | - Deshan Kumar Rajeswaran
- Centre for Advanced Rehabilitation Therapeutics, Tan Tock Seng Hospital Rehabilitation Centre, Singapore, Singapore
| | - Liming Xiang
- School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore, Singapore
| | - Etienne Burdet
- Department of Bioengineering, Imperial College of Science, Technology and Medicine, London, United Kingdom
| | - Domenico Campolo
- Robotic Research Center, Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, Singapore
| |
Collapse
|
4
|
Hesp C, Smith R, Parr T, Allen M, Friston KJ, Ramstead MJD. Deeply Felt Affect: The Emergence of Valence in Deep Active Inference. Neural Comput 2021; 33:398-446. [PMID: 33253028 PMCID: PMC8594962 DOI: 10.1162/neco_a_01341] [Citation(s) in RCA: 74] [Impact Index Per Article: 24.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2020] [Accepted: 08/17/2020] [Indexed: 01/20/2023]
Abstract
The positive-negative axis of emotional valence has long been recognized as fundamental to adaptive behavior, but its origin and underlying function have largely eluded formal theorizing and computational modeling. Using deep active inference, a hierarchical inference scheme that rests on inverting a model of how sensory data are generated, we develop a principled Bayesian model of emotional valence. This formulation asserts that agents infer their valence state based on the expected precision of their action model-an internal estimate of overall model fitness ("subjective fitness"). This index of subjective fitness can be estimated within any environment and exploits the domain generality of second-order beliefs (beliefs about beliefs). We show how maintaining internal valence representations allows the ensuing affective agent to optimize confidence in action selection preemptively. Valence representations can in turn be optimized by leveraging the (Bayes-optimal) updating term for subjective fitness, which we label affective charge (AC). AC tracks changes in fitness estimates and lends a sign to otherwise unsigned divergences between predictions and outcomes. We simulate the resulting affective inference by subjecting an in silico affective agent to a T-maze paradigm requiring context learning, followed by context reversal. This formulation of affective inference offers a principled account of the link between affect, (mental) action, and implicit metacognition. It characterizes how a deep biological system can infer its affective state and reduce uncertainty about such inferences through internal action (i.e., top-down modulation of priors that underwrite confidence). Thus, we demonstrate the potential of active inference to provide a formal and computationally tractable account of affect. Our demonstration of the face validity and potential utility of this formulation represents the first step within a larger research program. Next, this model can be leveraged to test the hypothesized role of valence by fitting the model to behavioral and neuronal responses.
Collapse
Affiliation(s)
- Casper Hesp
- Department of Psychology and Amsterdam Brain and Cognition Centre, University of Amsterdam, 1098 XH Amsterdam, Netherlands; Institute for Advanced Study, University of Amsterdam, 1012 GC Amsterdam, Netherlands; and Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, U.K.
| | - Ryan Smith
- Laureate Institute for Brain Research, Tulsa, OK 74136, U.S.A.
| | - Thomas Parr
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, U.K.
| | - Micah Allen
- Aarhus Institute of Advanced Studies, Aarhus University, Aarhus 8000, Denmark; Centre of Functionally Integrative Neuroscience, Aarhus University Hospital, Aarhus 8200, Denmark; and Cambridge Psychiatry, Cambridge University, Cambridge CB2 8AH, U.K.
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, U.K.
| | - Maxwell J D Ramstead
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, U.K.; Division of Social and Transcultural Psychiatry, Department of Psychiatry and Culture, Mind, and Brain Program, McGill University, Montreal H3A 0G4, QC, Canada
| |
Collapse
|
5
|
Colombo M, Heinz A. Explanatory integration, computational phenotypes, and dimensional psychiatry: The case of alcohol use disorder. THEORY & PSYCHOLOGY 2019. [DOI: 10.1177/0959354319867392] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
We compare three theoretical frameworks for pursuing explanatory integration in psychiatry: a new dimensional framework grounded in the notion of computational phenotype, a mechanistic framework, and a network of symptoms framework. Considering the phenomenon of alcoholism, we argue that the dimensional framework is the best for effectively integrating computational and mechanistic explanations with phenomenological analyses.
Collapse
|
6
|
Bidirectional regulation of reward, punishment, and arousal by dopamine, the lateral habenula and the rostromedial tegmentum (RMTg). Curr Opin Behav Sci 2019. [DOI: 10.1016/j.cobeha.2018.11.001] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
|
7
|
Persiani M, Franchi AM, Gini G. A working memory model improves cognitive control in agents and robots. COGN SYST RES 2018. [DOI: 10.1016/j.cogsys.2018.04.014] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
|
8
|
Hsu TM, McCutcheon JE, Roitman MF. Parallels and Overlap: The Integration of Homeostatic Signals by Mesolimbic Dopamine Neurons. Front Psychiatry 2018; 9:410. [PMID: 30233430 PMCID: PMC6129766 DOI: 10.3389/fpsyt.2018.00410] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/27/2018] [Accepted: 08/13/2018] [Indexed: 01/08/2023] Open
Abstract
Motivated behaviors are often initiated in response to perturbations of homeostasis. Indeed, animals and humans have fundamental drives to procure (appetitive behaviors) and eventually ingest (consummatory behaviors) substances based on deficits in body fluid (e.g., thirst) and energy balance (e.g., hunger). Consumption, in turn, reinforces motivated behavior and is therefore considered rewarding. Over the years, the constructs of homeostatic (within the purview of the hypothalamus) and reward (within the purview of mesolimbic circuitry) have been used to describe need-based vs. need-free consumption. However, many experiments have demonstrated that mesolimbic circuits and "higher-order" brain regions are also profoundly influenced by changes to physiological state, which in turn generate behaviors that are poised to maintain homeostasis. Mesolimbic pathways, particularly dopamine neurons of the ventral tegmental area (VTA) and their projections to nucleus accumbens (NAc), can be robustly modulated by a variety of energy balance signals, including post-ingestive feedback relaying nutrient content and hormonal signals reflecting hunger and satiety. Moreover, physiological states can also impact VTA-NAc responses to non-nutritive rewards, such as drugs of abuse. Coupled with recent evidence showing hypothalamic structures are modulated in anticipation of replenished need, classic boundaries between circuits that convey perturbations in homeostasis and those that drive motivated behavior are being questioned. In the current review, we examine data that have revealed the importance of mesolimbic dopamine neurons and their downstream pathways as a dynamic neurobiological mechanism that provides an interface between physiological state, perturbations to homeostasis, and reward-seeking behaviors.
Collapse
Affiliation(s)
- Ted M Hsu
- Department of Psychology, University of Illinois at Chicago, Chicago, IL, United States
| | - James E McCutcheon
- Department of Neuroscience, Psychology and Behavior, University of Leicester, Leicester, United Kingdom
| | - Mitchell F Roitman
- Department of Psychology, University of Illinois at Chicago, Chicago, IL, United States
| |
Collapse
|
9
|
DosSantos MF, Moura BDS, DaSilva AF. Reward Circuitry Plasticity in Pain Perception and Modulation. Front Pharmacol 2017; 8:790. [PMID: 29209204 PMCID: PMC5702349 DOI: 10.3389/fphar.2017.00790] [Citation(s) in RCA: 43] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2016] [Accepted: 10/19/2017] [Indexed: 12/30/2022] Open
Abstract
Although pain is a widely known phenomenon and an important clinical symptom that occurs in numerous diseases, its mechanisms are still barely understood. Owing to the scarce information concerning its pathophysiology, particularly what is involved in the transition from an acute state to a chronic condition, pain treatment is frequently unsatisfactory, therefore contributing to the amplification of the chronic pain burden. In fact, pain is an extremely complex experience that demands the recruitment of an intricate set of central nervous system components. This includes cortical and subcortical areas involved in interpretation of the general characteristics of noxious stimuli. It also comprises neural circuits that process the motivational-affective dimension of pain. Hence, the reward circuitry represents a vital element for pain experience and modulation. This review article focuses on the interpretation of the extensive data available connecting the major components of the reward circuitry to pain suffering, including the nucleus accumbens, ventral tegmental area, and the medial prefrontal cortex; with especial attention dedicated to the evaluation of neuroplastic changes affecting these structures found in chronic pain syndromes, such as migraine, trigeminal neuropathic pain, chronic back pain, and fibromyalgia.
Collapse
Affiliation(s)
- Marcos F. DosSantos
- Laboratório de Morfogênese Celular, Instituto de Ciências Biomédicas, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil
| | - Brenda de Souza Moura
- Programa de Pós-Graduação em Radiologia, Faculdade de Medicina, Universidade Federal do Rio de Janeiro, Rio de Janeiro, Brazil
| | - Alexandre F. DaSilva
- Headache and Orofacial Pain Effort, Department of Biologic and Materials Sciences, School of Dentistry, Center for Human Growth and Development, Molecular and Behavioral Neuroscience Institute, University of Michigan, Ann Arbor, MI, United States
| |
Collapse
|
10
|
Wright C, Colombo M, Beard A. HIT and brain reward function: A case of mistaken identity (theory). STUDIES IN HISTORY AND PHILOSOPHY OF BIOLOGICAL AND BIOMEDICAL SCIENCES 2017; 64:28-40. [PMID: 28667934 DOI: 10.1016/j.shpsc.2017.04.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/19/2016] [Revised: 04/03/2017] [Accepted: 04/11/2017] [Indexed: 06/07/2023]
Abstract
This paper employs a case study from the history of neuroscience-brain reward function-to scrutinize the inductive argument for the so-called 'Heuristic Identity Theory' (HIT). The case fails to support HIT, illustrating why other case studies previously thought to provide empirical support for HIT also fold under scrutiny. After distinguishing two different ways of understanding the types of identity claims presupposed by HIT and considering other conceptual problems, we conclude that HIT is not an alternative to the traditional identity theory so much as a relabeling of previously discussed strategies for mechanistic discovery.
Collapse
Affiliation(s)
- Cory Wright
- Department of Philosophy, California State University Long Beach, 1250 Bellflower Boulevard, Long Beach, CA 90840-2408, United States of America.
| | - Matteo Colombo
- Tilburg Center for Logic, Ethics, & Philosophy of Science, Tilburg University, PO Box 90153, 5000 LE, Tilburg, The Netherlands.
| | - Alexander Beard
- Department of Philosophy, California State University Long Beach, 1250 Bellflower Boulevard, Long Beach, CA 90840-2408, United States of America.
| |
Collapse
|
11
|
Striatal dopamine D1 receptor suppression impairs reward-associative learning. Behav Brain Res 2017; 323:100-110. [PMID: 28143767 DOI: 10.1016/j.bbr.2017.01.041] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2016] [Revised: 12/21/2016] [Accepted: 01/25/2017] [Indexed: 12/25/2022]
Abstract
Dopamine (DA) is required for reinforcement learning. Hence, disruptions in DA signaling may contribute to the learning deficits associated with psychiatric disorders. The DA D1 receptor (D1R) has been linked to learning and is a target for cognitive/motivational enhancement in patients with schizophrenia. Separating the striatal D1R contribution to learning vs. motivation, however, has been challenging. We suppressed striatal D1R expression in mice using a D1R-targeting short hairpin RNA (shRNA), delivered locally to the striatum via an adeno-associated virus (AAV). We then assessed reward- and punishment-associative learning using a probabilistic learning task and motivation using a progressive-ratio breakpoint procedure. We confirmed suppression of striatal D1Rs immunohistochemically and by testing locomotor activity after the administration of (+)-doxanthrine, a full D1R agonist, in control mice and those treated with the D1RshRNA. D1RshRNA-treated mice exhibited impaired reward-associative learning, while punishment-associative learning was spared. This deficit was unrelated to general learning impairments or amotivation, because the D1shRNA-treated mice exhibited normal Barnes maze learning and normal motivation in the progressive-ratio breakpoint procedure. Suppression of striatal D1Rs selectively impaired reward-associative learning whereas punishment-associative learning, aversion-motivated learning, and appetitive motivation were spared. Because patients with schizophrenia exhibit similar reward-associative learning deficits, D1R-targeted treatments should be investigated to improve reward learning in these patients.
Collapse
|
12
|
Zsuga J, Biro K, Tajti G, Szilasi ME, Papp C, Juhasz B, Gesztelyi R. 'Proactive' use of cue-context congruence for building reinforcement learning's reward function. BMC Neurosci 2016; 17:70. [PMID: 27793098 PMCID: PMC5086043 DOI: 10.1186/s12868-016-0302-7] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2015] [Accepted: 10/14/2016] [Indexed: 01/29/2023] Open
Abstract
BACKGROUND Reinforcement learning is a fundamental form of learning that may be formalized using the Bellman equation. Accordingly an agent determines the state value as the sum of immediate reward and of the discounted value of future states. Thus the value of state is determined by agent related attributes (action set, policy, discount factor) and the agent's knowledge of the environment embodied by the reward function and hidden environmental factors given by the transition probability. The central objective of reinforcement learning is to solve these two functions outside the agent's control either using, or not using a model. RESULTS In the present paper, using the proactive model of reinforcement learning we offer insight on how the brain creates simplified representations of the environment, and how these representations are organized to support the identification of relevant stimuli and action. Furthermore, we identify neurobiological correlates of our model by suggesting that the reward and policy functions, attributes of the Bellman equitation, are built by the orbitofrontal cortex (OFC) and the anterior cingulate cortex (ACC), respectively. CONCLUSIONS Based on this we propose that the OFC assesses cue-context congruence to activate the most context frame. Furthermore given the bidirectional neuroanatomical link between the OFC and model-free structures, we suggest that model-based input is incorporated into the reward prediction error (RPE) signal, and conversely RPE signal may be used to update the reward-related information of context frames and the policy underlying action selection in the OFC and ACC, respectively. Furthermore clinical implications for cognitive behavioral interventions are discussed.
Collapse
Affiliation(s)
- Judit Zsuga
- Department of Health Systems Management and Quality Management for Health Care, Faculty of Public Health, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary.
| | - Klara Biro
- Department of Health Systems Management and Quality Management for Health Care, Faculty of Public Health, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary
| | - Gabor Tajti
- Department of Health Systems Management and Quality Management for Health Care, Faculty of Public Health, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary
| | - Magdolna Emma Szilasi
- Department of Pharmacology, Faculty of Pharmacy, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary
| | - Csaba Papp
- Department of Health Systems Management and Quality Management for Health Care, Faculty of Public Health, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary
| | - Bela Juhasz
- Department of Pharmacology and Pharmacotherapy, Faculty of Medicine, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary
| | - Rudolf Gesztelyi
- Department of Pharmacology, Faculty of Pharmacy, University of Debrecen, Debrecen, Nagyerdei krt. 98, 4032, Hungary
| |
Collapse
|
13
|
Feng LC, Howell TJ, Bennett PC. How clicker training works: Comparing Reinforcing, Marking, and Bridging Hypotheses. Appl Anim Behav Sci 2016. [DOI: 10.1016/j.applanim.2016.05.012] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
14
|
Negash S, Sheppard NVN, Lambert NM, Fincham FD. Trading Later Rewards for Current Pleasure: Pornography Consumption and Delay Discounting. JOURNAL OF SEX RESEARCH 2016; 53:689-700. [PMID: 26305628 DOI: 10.1080/00224499.2015.1025123] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Internet pornography is a multi-billion-dollar industry that has grown increasingly accessible. Delay discounting involves devaluing larger, later rewards in favor of smaller, more immediate rewards. The constant novelty and primacy of sexual stimuli as particularly strong natural rewards make Internet pornography a unique activator of the brain's reward system, thereby having implications for decision-making processes. Based on theoretical studies of evolutionary psychology and neuroeconomics, two studies tested the hypothesis that consuming Internet pornography would relate to higher rates of delay discounting. Study 1 used a longitudinal design. Participants completed a pornography use questionnaire and a delay discounting task at Time 1 and then again four weeks later. Participants reporting higher initial pornography use demonstrated a higher delay discounting rate at Time 2, controlling for initial delay discounting. Study 2 tested for causality with an experimental design. Participants were randomly assigned to abstain from either their favorite food or pornography for three weeks. Participants who abstained from pornography use demonstrated lower delay discounting than participants who abstained from their favorite food. The finding suggests that Internet pornography is a sexual reward that contributes to delay discounting differently than other natural rewards. Theoretical and clinical implications of these studies are highlighted.
Collapse
Affiliation(s)
- Sesen Negash
- a Couple and Family Therapy Program , Alliant International University , San Diego , California
| | | | | | - Frank D Fincham
- c Family Institute , The Florida State University , Tallahassee , Florida
| |
Collapse
|
15
|
Abstract
UNLABELLED Although the brain relies on auditory information to calibrate vocal behavior, the neural substrates of vocal learning remain unclear. Here we demonstrate that lesions of the dopaminergic inputs to a basal ganglia nucleus in a songbird species (Bengalese finches, Lonchura striata var. domestica) greatly reduced the magnitude of vocal learning driven by disruptive auditory feedback in a negative reinforcement task. These lesions produced no measureable effects on the quality of vocal performance or the amount of song produced. Our results suggest that dopaminergic inputs to the basal ganglia selectively mediate reinforcement-driven vocal plasticity. In contrast, dopaminergic lesions produced no measurable effects on the birds' ability to restore song acoustics to baseline following the cessation of reinforcement training, suggesting that different forms of vocal plasticity may use different neural mechanisms. SIGNIFICANCE STATEMENT During skill learning, the brain relies on sensory feedback to improve motor performance. However, the neural basis of sensorimotor learning is poorly understood. Here, we investigate the role of the neurotransmitter dopamine in regulating vocal learning in the Bengalese finch, a songbird with an extremely precise singing behavior that can nevertheless be reshaped dramatically by auditory feedback. Our findings show that reduction of dopamine inputs to a region of the songbird basal ganglia greatly impairs vocal learning but has no detectable effect on vocal performance. These results suggest a specific role for dopamine in regulating vocal plasticity.
Collapse
|
16
|
Zsuga J, Tajti G, Papp C, Juhasz B, Gesztelyi R. FNDC5/irisin, a molecular target for boosting reward-related learning and motivation. Med Hypotheses 2016; 90:23-8. [PMID: 27063080 DOI: 10.1016/j.mehy.2016.02.020] [Citation(s) in RCA: 35] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2015] [Revised: 02/22/2016] [Accepted: 02/24/2016] [Indexed: 01/10/2023]
Abstract
Interventions focusing on the prevention and treatment of chronic non-communicable diseases are on rise. In the current article, we propose that dysfunction of the mesocortico-limbic reward system contributes to the emergence of the WHO-identified risk behaviors (tobacco use, unhealthy diet, physical inactivity and harmful use of alcohol), behaviors that underlie the evolution of major non-communicable diseases (e.g. cardiovascular diseases, cancer, diabetes and chronic respiratory diseases). Given that dopaminergic neurons of the mesocortico-limbic system are tightly associated with reward-related processes and motivation, their dysfunction may fundamentally influence behavior. While nicotine and alcohol alter dopamine neuron function by influencing some receptors, mesocortico-limbic system dysfunction was associated with elevation of metabolic set-point leading to hedonic over-eating. Although there is some empirical evidence, precise molecular mechanism for linking physical inactivity and mesocortico-limbic dysfunction per se seems to be missing; identification of which may contribute to higher success rates for interventions targeting lifestyle changes pertaining to physical activity. In the current article, we compile evidence in support of a link between exercise and the mesocortico-limbic system by elucidating interactions on the axis of muscle - irisin - brain derived neurotrophic factor (BDNF) - and dopaminergic function of the midbrain. Irisin is a contraction-regulated myokine formed primarily in skeletal muscle but also in the brain. Irisin stirred considerable interest, when its ability to induce browning of white adipose tissue parallel to increasing thermogenesis was discovered. Furthermore, it may also play a role in the regulation of behavior given it readily enters the central nervous system, where it induces BDNF expression in several brain areas linked to reward processing, e.g. the ventral tegmental area and the hippocampus. BDNF is a neurotropic factor that increases neuronal dopamine content, modulates dopamine release relevant for neuronal plasticity and increased neuronal survival as well as learning and memory. Further linking BDNF to dopaminergic function is BDNF's ability to activate tropomyosin-related kinase B receptor that shares signalization with presynaptic dopamine-3 receptors in the ventral tegmental area. Summarizing, we propose that the skeletal muscle derived irisin may be the link between physical activity and reward-related processes and motivation. Moreover alteration of this axis may contribute to sedentary lifestyle and subsequent non-communicable diseases. Preclinical and clinical experimental models to test this hypothesis are also proposed.
Collapse
Affiliation(s)
- Judit Zsuga
- Department of Health Systems Management and Quality Management for Health Care, Faculty of Public Health, University of Debrecen, Nagyerdei krt 98, 4032 Debrecen, Hungary.
| | - Gabor Tajti
- Department of Health Systems Management and Quality Management for Health Care, Faculty of Public Health, University of Debrecen, Nagyerdei krt 98, 4032 Debrecen, Hungary
| | - Csaba Papp
- Department of Health Systems Management and Quality Management for Health Care, Faculty of Public Health, University of Debrecen, Nagyerdei krt 98, 4032 Debrecen, Hungary
| | - Bela Juhasz
- Department of Pharmacology and Pharmacotherapy, Faculty of Medicine, University of Debrecen, Nagyerdei krt 98, 4032 Debrecen, Hungary
| | - Rudolf Gesztelyi
- Department of Pharmacology, Faculty of Pharmacy, University of Debrecen, Nagyerdei krt 98, 4032 Debrecen, Hungary
| |
Collapse
|
17
|
Colombo M. Why build a virtual brain? Large-scale neural simulations as jump start for cognitive computing. J EXP THEOR ARTIF IN 2016. [DOI: 10.1080/0952813x.2016.1148076] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
18
|
Colombo M, Wright C. Explanatory pluralism: An unrewarding prediction error for free energy theorists. Brain Cogn 2016; 112:3-12. [PMID: 26905647 DOI: 10.1016/j.bandc.2016.02.003] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2015] [Revised: 02/12/2016] [Accepted: 02/13/2016] [Indexed: 01/22/2023]
Abstract
Courtesy of its free energy formulation, the hierarchical predictive processing theory of the brain (PTB) is often claimed to be a grand unifying theory. To test this claim, we examine a central case: activity of mesocorticolimbic dopaminergic (DA) systems. After reviewing the three most prominent hypotheses of DA activity-the anhedonia, incentive salience, and reward prediction error hypotheses-we conclude that the evidence currently vindicates explanatory pluralism. This vindication implies that the grand unifying claims of advocates of PTB are unwarranted. More generally, we suggest that the form of scientific progress in the cognitive sciences is unlikely to be a single overarching grand unifying theory.
Collapse
Affiliation(s)
- Matteo Colombo
- Tilburg Center for Logic, Ethics & Philosophy of Science, Tilburg University, PO Box 90153, 5000 LE Tilburg, The Netherlands.
| | - Cory Wright
- Department of Philosophy, McIntosh Humanities Building 917, California State University, Long Beach, 1250 Bellflower Boulevard, Long Beach, CA 90840-2408, USA.
| |
Collapse
|
19
|
Colombo M, Stankevicius A, Seriès P. Benefits of social vs. non-social feedback on learning and generosity. Results from the Tipping Game. Front Psychol 2014; 5:1154. [PMID: 25346715 PMCID: PMC4191563 DOI: 10.3389/fpsyg.2014.01154] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2014] [Accepted: 08/23/2014] [Indexed: 12/27/2022] Open
Abstract
Although much work has recently been directed at understanding social decision-making, relatively little is known about how different types of feedback impact adaptive changes in social behavior. To address this issue quantitatively, we designed a novel associative learning task called the “Tipping Game,” in which participants had to learn a social norm of tipping in restaurants. Participants were found to make more generous decisions from feedback in the form of facial expressions, in comparison to feedback in the form of symbols such as ticks and crosses. Furthermore, more participants displayed learning in the condition where they received social feedback than participants in the non-social condition. Modeling results showed that the pattern of performance displayed by participants receiving social feedback could be explained by a lower sensitivity to economic costs.
Collapse
Affiliation(s)
- Matteo Colombo
- Tilburg Center for Logic, General Ethics, and Philosophy of Science, Tilburg University, Tilburg Netherlands
| | - Aistis Stankevicius
- Institute for Adaptive and Neural Computation, University of Edinburgh, Edinburgh UK
| | - Peggy Seriès
- Institute for Adaptive and Neural Computation, University of Edinburgh, Edinburgh UK
| |
Collapse
|