1
|
Markowitz JE, Gillis WF, Jay M, Wood J, Harris RW, Cieszkowski R, Scott R, Brann D, Koveal D, Kula T, Weinreb C, Osman MAM, Pinto SR, Uchida N, Linderman SW, Sabatini BL, Datta SR. Spontaneous behaviour is structured by reinforcement without explicit reward. Nature 2023; 614:108-117. [PMID: 36653449 PMCID: PMC9892006 DOI: 10.1038/s41586-022-05611-2] [Citation(s) in RCA: 65] [Impact Index Per Article: 65.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Accepted: 11/30/2022] [Indexed: 01/19/2023]
Abstract
Spontaneous animal behaviour is built from action modules that are concatenated by the brain into sequences1,2. However, the neural mechanisms that guide the composition of naturalistic, self-motivated behaviour remain unknown. Here we show that dopamine systematically fluctuates in the dorsolateral striatum (DLS) as mice spontaneously express sub-second behavioural modules, despite the absence of task structure, sensory cues or exogenous reward. Photometric recordings and calibrated closed-loop optogenetic manipulations during open field behaviour demonstrate that DLS dopamine fluctuations increase sequence variation over seconds, reinforce the use of associated behavioural modules over minutes, and modulate the vigour with which modules are expressed, without directly influencing movement initiation or moment-to-moment kinematics. Although the reinforcing effects of optogenetic DLS dopamine manipulations vary across behavioural modules and individual mice, these differences are well predicted by observed variation in the relationships between endogenous dopamine and module use. Consistent with the possibility that DLS dopamine fluctuations act as a teaching signal, mice build sequences during exploration as if to maximize dopamine. Together, these findings suggest a model in which the same circuits and computations that govern action choices in structured tasks have a key role in sculpting the content of unconstrained, high-dimensional, spontaneous behaviour.
Collapse
Affiliation(s)
- Jeffrey E Markowitz
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
- Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
| | | | - Maya Jay
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Jeffrey Wood
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Ryley W Harris
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | | | - Rebecca Scott
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - David Brann
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Dorothy Koveal
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Tomasz Kula
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Caleb Weinreb
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | | | - Sandra Romero Pinto
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
- Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Naoshige Uchida
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
- Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Scott W Linderman
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA
- Department of Statistics, Stanford University, Stanford, CA, USA
| | - Bernardo L Sabatini
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
- Howard Hughes Medical Institute, Chevy Chase, MD, USA
| | | |
Collapse
|
2
|
Nazari Z, Hadi Nematollahi M, Zareh F, Pouramiri B, Mehrabani M. An Electrochemical Sensor Based on Carbon Quantum Dots and Ionic Liquids for Selective Detection of Dopamine. ChemistrySelect 2023. [DOI: 10.1002/slct.202203630] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
Affiliation(s)
- Zahra Nazari
- Department of Chemistry, Faculty of Science Shahid Bahonar University of Kerman Kerman Iran
| | - Mohammad Hadi Nematollahi
- Neuroscience Research Center, Institute of Neuropharmacology Kerman University of Medical Sciences Kerman Iran
| | - Fatemeh Zareh
- Department of Chemistry, Faculty of Science Shahid Bahonar University of Kerman Kerman Iran
| | | | - Mehrnaz Mehrabani
- Physiology Research Center, Institute of Neuropharmacology Kerman University of Medical Sciences Kerman Iran
| |
Collapse
|
3
|
Galizio A, Friedel JE, Odum AL. An investigation of resurgence of reinforced behavioral variability in humans. J Exp Anal Behav 2020; 114:381-393. [PMID: 33179789 PMCID: PMC7967018 DOI: 10.1002/jeab.637] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2020] [Revised: 10/18/2020] [Accepted: 10/19/2020] [Indexed: 11/09/2022]
Abstract
The present study examined resurgence of reinforced variability in college students, who completed a 3-phase computer-based variability task. In the first phase, baseline, points were delivered for drawing rectangles that sufficiently differed from previous rectangles in terms of a target dimension (size or location, counterbalanced) but were sufficiently similar in terms of the alternative dimension. In the second phase, alternative, points were only delivered for rectangles that were sufficiently different in terms of the alternative dimension, but repetitive in terms of the target dimension. In the third phase, extinction, no points were delivered. In baseline, participants made rectangles that were highly varied in terms of the target dimension and less varied in terms of the alternative dimension, and vice versa in the alternative phase. During extinction, levels of variability increased for the target dimension, providing evidence for resurgence of reinforced variability of a specific dimension of behavior. However, levels of variability also remained high for the alternative dimension, indicating that extinction-induced response variability may also have impacted the results. Although future research is needed to explore other explanations, the results of this study replicate prior research with pigeons and provide some support for the notion of variability as an operant.
Collapse
|
4
|
Adams RA, Moutoussis M, Nour MM, Dahoun T, Lewis D, Illingworth B, Veronese M, Mathys C, de Boer L, Guitart-Masip M, Friston KJ, Howes OD, Roiser JP. Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models. Cereb Cortex 2020; 30:3573-3589. [PMID: 32083297 PMCID: PMC7233027 DOI: 10.1093/cercor/bhz327] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 11/18/2019] [Accepted: 12/05/2019] [Indexed: 12/17/2022] Open
Abstract
Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans.
Collapse
Affiliation(s)
- Rick A Adams
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
- Division of Psychiatry, University College London, London W1T 7NF, UK
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
| | - Michael Moutoussis
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
| | - Matthew M Nour
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King’s College London, London SE5 8AF, UK
| | - Tarik Dahoun
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford OX3 7JX, UK
| | - Declan Lewis
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| | - Benjamin Illingworth
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| | - Mattia Veronese
- Centre for Neuroimaging Sciences, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King's College London, London SE5 8AF, UK
| | - Christoph Mathys
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
- Scuola Internazionale Superiore di Studi Avanzati (SISSA), 34136 Trieste, Italy
- Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich and ETH Zurich, 8032 Zurich, Switzerland
| | - Lieke de Boer
- Aging Research Center, Karolinska Institute, 171 65 Stockholm, Sweden
| | - Marc Guitart-Masip
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
- Aging Research Center, Karolinska Institute, 171 65 Stockholm, Sweden
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK
| | - Oliver D Howes
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King’s College London, London SE5 8AF, UK
| | - Jonathan P Roiser
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| |
Collapse
|
5
|
Arnold MA, Newland MC. Variable behavior and repeated learning in two mouse strains: Developmental and genetic contributions. Behav Processes 2018; 157:509-518. [DOI: 10.1016/j.beproc.2018.06.007] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2017] [Revised: 06/05/2018] [Accepted: 06/15/2018] [Indexed: 12/16/2022]
|
6
|
Galizio A, Frye CCJ, Haynes JM, Friedel JE, Smith BM, Odum AL. Persistence and relapse of reinforced behavioral variability. J Exp Anal Behav 2018; 109:210-237. [DOI: 10.1002/jeab.309] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2017] [Accepted: 12/19/2017] [Indexed: 11/12/2022]
|
7
|
Olsen RA, Macaskill AC, Harper DN. Differential effects of 3,4-methylenedioxymethamphetamine, methamphetamine, meta-Chlorophenylpiperazine, and scopolamine on behavioral repetition versus variation in rats. Pharmacol Biochem Behav 2016; 150-151:170-181. [PMID: 27826121 DOI: 10.1016/j.pbb.2016.11.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/15/2016] [Revised: 11/01/2016] [Accepted: 11/04/2016] [Indexed: 10/20/2022]
Abstract
Acute administration of drugs of abuse, such as MDMA and methamphetamine, disrupts performance on many operant tasks, for example, those used to study memory. This might occur in part because drugs make behavior, in general, more repetitive or more variable, or because they produce a more global disruption to performance. The current study explored this across two experiments by employing Neuringer's 'reinforced variability' procedure. Varied behavior was reinforced at some times during the session and repetitive behavior at other times; lights signalled the behavior required. This procedure allowed an investigation of whether a particular drug made behavior more variable (affected behavior when repetition was required), more repetitive (affected behavior when variability was required), or produced a global disruption (affected both components). In Experiment 1, MDMA increased variability while methamphetamine affected both components. In Experiment 2, m-CPP affected both components while scopolamine affected both components at lower doses and increased variability at higher doses. These results indicate both that the reinforced variability procedure can be used to isolate the specific effects of drugs of abuse on the variability of behavior, and that the specific impact of a given drug needs to be considered when interpreting pharmacological disruptions to operant task performance.
Collapse
Affiliation(s)
- Rebecca A Olsen
- C/- School of Psychology, Victoria University of Wellington, PO Box 600, Wellington, New Zealand
| | - Anne C Macaskill
- C/- School of Psychology, Victoria University of Wellington, PO Box 600, Wellington, New Zealand.
| | - David N Harper
- C/- School of Psychology, Victoria University of Wellington, PO Box 600, Wellington, New Zealand
| |
Collapse
|
8
|
Rabin BM, Heroux NA, Shukitt-Hale B, Carrihill-Knoll KL, Beck Z, Baxter C. Lack of reliability in the disruption of cognitive performance following exposure to protons. RADIATION AND ENVIRONMENTAL BIOPHYSICS 2015; 54:285-95. [PMID: 25935209 DOI: 10.1007/s00411-015-0597-2] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/04/2014] [Accepted: 04/23/2015] [Indexed: 05/27/2023]
Abstract
A series of three replications were run to determine the reliability with which exposure to protons produces a disruption of cognitive performance, using a novel object recognition task and operant responding on an ascending fixed-ratio task. For the first two replications, rats were exposed to head-only exposures to 1000 MeV/n protons at the NASA Space Radiation Laboratory. For the third replication, subjects were given head-only or whole-body exposures to both 1000 and 150 MeV/n protons. The results were characterized by a lack of consistency in the effects of exposure to protons on the performance of these cognitive tasks, both within and between replications. The factors that might influence the lack of consistency and the implications for exploratory class missions are discussed.
Collapse
|
9
|
Abstract
Movement variability is often considered an unwanted byproduct of a noisy nervous system. However, variability can signal a form of implicit exploration, indicating that the nervous system is intentionally varying the motor commands in search of actions that yield the greatest success. Here, we investigated the role of the human basal ganglia in controlling reward-dependent motor variability as measured by trial-to-trial changes in performance during a reaching task. We designed an experiment in which the only performance feedback was success or failure and quantified how reach variability was modulated as a function of the probability of reward. In healthy controls, reach variability increased as the probability of reward decreased. Control of variability depended on the history of past rewards, with the largest trial-to-trial changes occurring immediately after an unrewarded trial. In contrast, in participants with Parkinson's disease, a known example of basal ganglia dysfunction, reward was a poor modulator of variability; that is, the patients showed an impaired ability to increase variability in response to decreases in the probability of reward. This was despite the fact that, after rewarded trials, reach variability in the patients was comparable to healthy controls. In summary, we found that movement variability is partially a form of exploration driven by the recent history of rewards. When the function of the human basal ganglia is compromised, the reward-dependent control of movement variability is impaired, particularly affecting the ability to increase variability after unsuccessful outcomes.
Collapse
|
10
|
Strait DL, Slater J, O'Connell S, Kraus N. Music training relates to the development of neural mechanisms of selective auditory attention. Dev Cogn Neurosci 2015; 12:94-104. [PMID: 25660985 PMCID: PMC6989776 DOI: 10.1016/j.dcn.2015.01.001] [Citation(s) in RCA: 35] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2014] [Revised: 01/06/2015] [Accepted: 01/06/2015] [Indexed: 11/26/2022] Open
Abstract
Does music training shape the development of neural mechanisms of auditory attention? We compared cortical responses to attended speech in child and adult musicians and nonmusicians. Musician children and adults had less prefrontal auditory response variability during attention.
Selective attention decreases trial-to-trial variability in cortical auditory-evoked activity. This effect increases over the course of maturation, potentially reflecting the gradual development of selective attention and inhibitory control. Work in adults indicates that music training may alter the development of this neural response characteristic, especially over brain regions associated with executive control: in adult musicians, attention decreases variability in auditory-evoked responses recorded over prefrontal cortex to a greater extent than in nonmusicians. We aimed to determine whether this musician-associated effect emerges during childhood, when selective attention and inhibitory control are under development. We compared cortical auditory-evoked variability to attended and ignored speech streams in musicians and nonmusicians across three age groups: preschoolers, school-aged children and young adults. Results reveal that childhood music training is associated with reduced auditory-evoked response variability recorded over prefrontal cortex during selective auditory attention in school-aged child and adult musicians. Preschoolers, on the other hand, demonstrate no impact of selective attention on cortical response variability and no musician distinctions. This finding is consistent with the gradual emergence of attention during this period and may suggest no pre-existing differences in this attention-related cortical metric between children who undergo music training and those who do not.
Collapse
Affiliation(s)
- Dana L Strait
- Auditory Neuroscience Laboratory, Northwestern University, Evanston, IL, USA; Institute for Neuroscience, Northwestern University, Chicago, IL, USA
| | - Jessica Slater
- Auditory Neuroscience Laboratory, Northwestern University, Evanston, IL, USA; Department of Communication Sciences, Northwestern University, Evanston, IL, USA
| | - Samantha O'Connell
- Auditory Neuroscience Laboratory, Northwestern University, Evanston, IL, USA
| | - Nina Kraus
- Auditory Neuroscience Laboratory, Northwestern University, Evanston, IL, USA; Institute for Neuroscience, Northwestern University, Chicago, IL, USA; Department of Communication Sciences, Northwestern University, Evanston, IL, USA; Department of Neurobiology and Physiology, Northwestern University, Evanston, IL, USA; Department of Otolaryngology, Northwestern University, Chicago, IL, USA.
| |
Collapse
|
11
|
Eagle DM, Noschang C, d'Angelo LSC, Noble CA, Day JO, Dongelmans ML, Theobald DE, Mar AC, Urcelay GP, Morein-Zamir S, Robbins TW. The dopamine D2/D3 receptor agonist quinpirole increases checking-like behaviour in an operant observing response task with uncertain reinforcement: a novel possible model of OCD. Behav Brain Res 2014; 264:207-29. [PMID: 24406720 PMCID: PMC3989029 DOI: 10.1016/j.bbr.2013.12.040] [Citation(s) in RCA: 41] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2013] [Revised: 12/19/2013] [Accepted: 12/23/2013] [Indexed: 01/18/2023]
Abstract
Excessive checking is a common, debilitating symptom of obsessive-compulsive disorder (OCD). In an established rodent model of OCD checking behaviour, quinpirole (dopamine D2/3-receptor agonist) increased checking in open-field tests, indicating dopaminergic modulation of checking-like behaviours. We designed a novel operant paradigm for rats (observing response task (ORT)) to further examine cognitive processes underpinning checking behaviour and clarify how and why checking develops. We investigated i) how quinpirole increases checking, ii) dependence of these effects on D2/3 receptor function (following treatment with D2/3 receptor antagonist sulpiride) and iii) effects of reward uncertainty. In the ORT, rats pressed an 'observing' lever for information about the location of an 'active' lever that provided food reinforcement. High- and low-checkers (defined from baseline observing) received quinpirole (0.5mg/kg, 10 treatments) or vehicle. Parametric task manipulations assessed observing/checking under increasing task demands relating to reinforcement uncertainty (variable response requirement and active-lever location switching). Treatment with sulpiride further probed the pharmacological basis of long-term behavioural changes. Quinpirole selectively increased checking, both functional observing lever presses (OLPs) and non-functional extra OLPs (EOLPs). The increase in OLPs and EOLPs was long-lasting, without further quinpirole administration. Quinpirole did not affect the immediate ability to use information from checking. Vehicle and quinpirole-treated rats (VEH and QNP respectively) were selectively sensitive to different forms of uncertainty. Sulpiride reduced non-functional EOLPs in QNP rats but had no effect on functional OLPs. These data have implications for treatment of compulsive checking in OCD, particularly for serotonin-reuptake-inhibitor treatment-refractory cases, where supplementation with dopamine receptor antagonists may be beneficial.
Collapse
Affiliation(s)
- Dawn M Eagle
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Downing Site, Cambridge, CB2 3EB, UK.
| | - Cristie Noschang
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Downing Site, Cambridge, CB2 3EB, UK
| | - Laure-Sophie Camilla d'Angelo
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Downing Site, Cambridge, CB2 3EB, UK
| | - Christie A Noble
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Downing Site, Cambridge, CB2 3EB, UK
| | - Jacob O Day
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Downing Site, Cambridge, CB2 3EB, UK
| | - Marie Louise Dongelmans
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Downing Site, Cambridge, CB2 3EB, UK
| | - David E Theobald
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Downing Site, Cambridge, CB2 3EB, UK
| | - Adam C Mar
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Downing Site, Cambridge, CB2 3EB, UK
| | - Gonzalo P Urcelay
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Downing Site, Cambridge, CB2 3EB, UK
| | - Sharon Morein-Zamir
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Downing Site, Cambridge, CB2 3EB, UK
| | - Trevor W Robbins
- Department of Psychology and Behavioural and Clinical Neuroscience Institute, University of Cambridge, Downing Site, Cambridge, CB2 3EB, UK
| |
Collapse
|
12
|
Galea JM, Ruge D, Buijink A, Bestmann S, Rothwell JC. Punishment-induced behavioral and neurophysiological variability reveals dopamine-dependent selection of kinematic movement parameters. J Neurosci 2013; 33:3981-8. [PMID: 23447607 PMCID: PMC3672959 DOI: 10.1523/jneurosci.1294-12.2013] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2012] [Revised: 12/18/2012] [Accepted: 01/15/2013] [Indexed: 11/21/2022] Open
Abstract
Action selection describes the high-level process that selects between competing movements. In animals, behavioral variability is critical for the motor exploration required to select the action that optimizes reward and minimizes cost/punishment and is guided by dopamine (DA). The aim of this study was to test in humans whether low-level movement parameters are affected by punishment and reward in ways similar to high-level action selection. Moreover, we addressed the proposed dependence of behavioral and neurophysiological variability on DA and whether this may underpin the exploration of kinematic parameters. Participants performed an out-and-back index finger movement and were instructed that monetary reward and punishment were based on its maximal acceleration (MA). In fact, the feedback was not contingent on the participant's behavior but predetermined. Blocks highly biased toward punishment were associated with increased MA variability relative to blocks either with reward or without feedback. This increase in behavioral variability was positively correlated with neurophysiological variability, as measured by changes in corticospinal excitability with transcranial magnetic stimulation over the primary motor cortex. Following the administration of a DA antagonist, the variability associated with punishment diminished and the correlation between behavioral and neurophysiological variability no longer existed. Similar changes in variability were not observed when participants executed a predetermined MA, nor did DA influence resting neurophysiological variability. Thus, under conditions of punishment, DA-dependent processes influence the selection of low-level movement parameters. We propose that the enhanced behavioral variability reflects the exploration of kinematic parameters for less punishing, or conversely more rewarding, outcomes.
Collapse
Affiliation(s)
- Joseph M Galea
- Behavioural Brain Sciences, School of Psychology, University of Birmingham, B15 2TT United Kingdom.
| | | | | | | | | |
Collapse
|