1
|
Brown LS, Cho JR, Bolkan SS, Nieh EH, Schottdorf M, Tank DW, Brody CD, Witten IB, Goldman MS. Neural circuit models for evidence accumulation through choice-selective sequences. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.09.01.555612. [PMID: 38234715 PMCID: PMC10793437 DOI: 10.1101/2023.09.01.555612] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/19/2024]
Abstract
Decision making is traditionally thought to be mediated by populations of neurons whose firing rates persistently accumulate evidence across time. However, recent decision-making experiments in rodents have observed neurons across the brain that fire sequentially as a function of spatial position or time, rather than persistently, with the subset of neurons in the sequence depending on the animal's choice. We develop two new candidate circuit models, in which evidence is encoded either in the relative firing rates of two competing chains of neurons or in the network location of a stereotyped pattern ("bump") of neural activity. Encoded evidence is then faithfully transferred between neuronal populations representing different positions or times. Neural recordings from four different brain regions during a decision-making task showed that, during the evidence accumulation period, different brain regions displayed tuning curves consistent with different candidate models for evidence accumulation. This work provides mechanistic models and potential neural substrates for how graded-value information may be precisely accumulated within and transferred between neural populations, a set of computations fundamental to many cognitive operations.
Collapse
|
2
|
Yuan Z, Qi Z, Wang R, Cui Y, An S, Wu G, Feng Q, Lin R, Dai R, Li A, Gong H, Luo Q, Fu L, Luo M. A corticoamygdalar pathway controls reward devaluation and depression using dynamic inhibition code. Neuron 2023; 111:3837-3853.e5. [PMID: 37734380 DOI: 10.1016/j.neuron.2023.08.022] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Revised: 07/03/2023] [Accepted: 08/22/2023] [Indexed: 09/23/2023]
Abstract
Reward devaluation adaptively controls reward intake. It remains unclear how cortical circuits causally encode reward devaluation in healthy and depressed states. Here, we show that the neural pathway from the anterior cingulate cortex (ACC) to the basolateral amygdala (BLA) employs a dynamic inhibition code to control reward devaluation and depression. Fiber photometry and imaging of ACC pyramidal neurons reveal reward-induced inhibition, which weakens during satiation and becomes further attenuated in depression mouse models. Ablating or inhibiting these neurons desensitizes reward devaluation, causes reward intake increase and ultimate obesity, and ameliorates depression, whereas activating the cells sensitizes reward devaluation, suppresses reward consumption, and produces depression-like behaviors. Among various ACC neuron subpopulations, the BLA-projecting subset bidirectionally regulates reward devaluation and depression-like behaviors. Our study thus uncovers a corticoamygdalar circuit that encodes reward devaluation via blunted inhibition and suggests that enhancing inhibition within this circuit may offer a therapeutic approach for treating depression.
Collapse
Affiliation(s)
- Zhengwei Yuan
- Academy for Advanced Interdisciplinary Studies, Peking University, Beijing 100871, China; School of Life Sciences, Tsinghua University, Beijing 100084, China; National Institute of Biological Sciences, Beijing 102206, China; Chinese Institute for Brain Research, Beijing 102206, China; Tsinghua Institute of Multidisciplinary Biomedical Research (TIMBR), Beijing 102206, China
| | - Zhongyang Qi
- National Institute of Biological Sciences, Beijing 102206, China; Wuhan National Laboratory for Optoelectronics-Huazhong, Britton Chance Center for Biomedical Photonics, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Ruiyu Wang
- National Institute of Biological Sciences, Beijing 102206, China; School of Life Sciences, Peking University, Beijing 100871, China
| | - Yuting Cui
- National Institute of Biological Sciences, Beijing 102206, China; Chinese Institute for Brain Research, Beijing 102206, China
| | - Sile An
- Wuhan National Laboratory for Optoelectronics-Huazhong, Britton Chance Center for Biomedical Photonics, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Guoli Wu
- National Institute of Biological Sciences, Beijing 102206, China
| | - Qiru Feng
- National Institute of Biological Sciences, Beijing 102206, China
| | - Rui Lin
- National Institute of Biological Sciences, Beijing 102206, China; Tsinghua Institute of Multidisciplinary Biomedical Research (TIMBR), Beijing 102206, China
| | - Ruicheng Dai
- National Institute of Biological Sciences, Beijing 102206, China; School of Life Sciences, Peking University, Beijing 100871, China; Chinese Institute for Brain Research, Beijing 102206, China
| | - Anan Li
- Wuhan National Laboratory for Optoelectronics-Huazhong, Britton Chance Center for Biomedical Photonics, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Hui Gong
- Wuhan National Laboratory for Optoelectronics-Huazhong, Britton Chance Center for Biomedical Photonics, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Qingming Luo
- Wuhan National Laboratory for Optoelectronics-Huazhong, Britton Chance Center for Biomedical Photonics, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Ling Fu
- Wuhan National Laboratory for Optoelectronics-Huazhong, Britton Chance Center for Biomedical Photonics, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Minmin Luo
- National Institute of Biological Sciences, Beijing 102206, China; Chinese Institute for Brain Research, Beijing 102206, China; Tsinghua Institute of Multidisciplinary Biomedical Research (TIMBR), Beijing 102206, China; Research Unit of Medical Neurobiology, Chinese Academy of Medical Sciences, Beijing 100005, China; New Cornerstone Science Laboratory, Shenzhen 518054, China; Beijing Tiantan Hospital, 100070 Beijing, China.
| |
Collapse
|
3
|
Wang P, Chen S, Deng K, Zhang B, Im H, Feng J, Liu L, Yang Q, Zhao G, He Q, Chen C, Wang H, Wang Q. Distributed attribute representation in the superior parietal lobe during probabilistic decision-making. Hum Brain Mapp 2023; 44:5693-5711. [PMID: 37614216 PMCID: PMC10619403 DOI: 10.1002/hbm.26470] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Revised: 06/18/2023] [Accepted: 08/10/2023] [Indexed: 08/25/2023] Open
Abstract
Several studies have examined the neural substrates of probabilistic decision-making, but few have systematically investigated the neural representations of the two objective attributes of probabilistic rewards, that is, the reward amount and the probability. Specifically, whether there are common or distinct neural activity patterns to represent the objective attributes and their association with the neural representation of the subjective valuation remains largely underexplored. We conducted two studies (nStudy1 = 34, nStudy2 = 41) to uncover distributed neural representations of the objective attributes and subjective value as well as their association with individual probability discounting rates. The amount and probability were independently manipulated to better capture brain signals sensitive to these two attributes and were presented simultaneously in Study 1 and successively in Study 2. Both univariate and multivariate pattern analyses showed that the brain activities in the superior parietal lobule (SPL), including the postcentral gyrus, were modulated by the amount of rewards and probability in both studies. Further, representational similarity analysis revealed a similar neural representation between these two objective attributes and between the attribute and valuation. Moreover, the SPL tracked the subjective value integrated by the hyperbolic function. Probability-related brain activations in the inferior parietal lobule were associated with the variability in individual discounting rates. These findings provide novel insights into a similar neural representation of the two attributes during probabilistic decision-making and perhaps support the common neural coding of stimulus objective properties and subjective value in the field of probabilistic discounting.
Collapse
Affiliation(s)
- Pinchun Wang
- Faculty of PsychologyTianjin Normal UniversityTianjinChina
| | - Shuning Chen
- Faculty of PsychologyTianjin Normal UniversityTianjinChina
| | - Kun Deng
- Faculty of PsychologyTianjin Normal UniversityTianjinChina
| | - Bin Zhang
- Faculty of PsychologyTianjin Normal UniversityTianjinChina
| | - Hohjin Im
- Department of Psychological ScienceUniversity of California IrvineIrvineCaliforniaUSA
| | - Junjiao Feng
- Faculty of PsychologyTianjin Normal UniversityTianjinChina
- Key Research Base of Humanities and Social Sciences of the Ministry of Education, Academy of Psychology and BehaviorTianjin Normal UniversityTianjinChina
- Tianjin Social Science Laboratory of Students' Mental Development and LearningTianjinChina
| | - Liqing Liu
- Faculty of PsychologyTianjin Normal UniversityTianjinChina
- Key Research Base of Humanities and Social Sciences of the Ministry of Education, Academy of Psychology and BehaviorTianjin Normal UniversityTianjinChina
- Tianjin Social Science Laboratory of Students' Mental Development and LearningTianjinChina
| | - Qinghao Yang
- Faculty of PsychologyTianjin Normal UniversityTianjinChina
| | - Guang Zhao
- Faculty of PsychologyTianjin Normal UniversityTianjinChina
- Key Research Base of Humanities and Social Sciences of the Ministry of Education, Academy of Psychology and BehaviorTianjin Normal UniversityTianjinChina
- Tianjin Social Science Laboratory of Students' Mental Development and LearningTianjinChina
| | - Qinghua He
- Faculty of Psychology, MOE Key Laboratory of Cognition and PersonalitySouthwest UniversityChongqingChina
| | - Chunhui Chen
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain ResearchBeijing Normal UniversityBeijingChina
| | - He Wang
- Institute of Biomedical EngineeringChinese Academy of Medical Science & Peking Union Medical CollegeTianjinChina
| | - Qiang Wang
- Faculty of PsychologyTianjin Normal UniversityTianjinChina
- Key Research Base of Humanities and Social Sciences of the Ministry of Education, Academy of Psychology and BehaviorTianjin Normal UniversityTianjinChina
- Tianjin Social Science Laboratory of Students' Mental Development and LearningTianjinChina
| |
Collapse
|
4
|
Gore F, Hernandez M, Ramakrishnan C, Crow AK, Malenka RC, Deisseroth K. Orbitofrontal cortex control of striatum leads economic decision-making. Nat Neurosci 2023; 26:1566-1574. [PMID: 37592039 PMCID: PMC10471500 DOI: 10.1038/s41593-023-01409-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2023] [Accepted: 07/17/2023] [Indexed: 08/19/2023]
Abstract
Animals must continually evaluate stimuli in their environment to decide which opportunities to pursue, and in many cases these decisions can be understood in fundamentally economic terms. Although several brain regions have been individually implicated in these processes, the brain-wide mechanisms relating these regions in decision-making are unclear. Using an economic decision-making task adapted for rats, we find that neural activity in both of two connected brain regions, the ventrolateral orbitofrontal cortex (OFC) and the dorsomedial striatum (DMS), was required for economic decision-making. Relevant neural activity in both brain regions was strikingly similar, dominated by the spatial features of the decision-making process. However, the neural encoding of choice direction in OFC preceded that of DMS, and this temporal relationship was strongly correlated with choice accuracy. Furthermore, activity specifically in the OFC projection to the DMS was required for appropriate economic decision-making. These results demonstrate that choice information in the OFC is relayed to the DMS to lead accurate economic decision-making.
Collapse
Affiliation(s)
- Felicity Gore
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA
- Nancy Pritzker Laboratory, Stanford University, Stanford, CA, USA
| | - Melissa Hernandez
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA
| | - Charu Ramakrishnan
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA
| | - Ailey K Crow
- Department of Bioengineering, Stanford University, Stanford, CA, USA
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA
| | - Robert C Malenka
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA
- Nancy Pritzker Laboratory, Stanford University, Stanford, CA, USA
| | - Karl Deisseroth
- Department of Bioengineering, Stanford University, Stanford, CA, USA.
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA.
- Howard Hughes Medical Institute, Stanford University, Stanford, CA, USA.
| |
Collapse
|
5
|
Yun M, Nejime M, Kawai T, Kunimatsu J, Yamada H, Kim HR, Matsumoto M. Distinct roles of the orbitofrontal cortex, ventral striatum, and dopamine neurons in counterfactual thinking of decision outcomes. SCIENCE ADVANCES 2023; 9:eadh2831. [PMID: 37556536 PMCID: PMC10411892 DOI: 10.1126/sciadv.adh2831] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Accepted: 07/07/2023] [Indexed: 08/11/2023]
Abstract
Individuals often assess past decisions by comparing what was gained with what would have been gained had they acted differently. Thoughts of past alternatives that counter what actually happened are called "counterfactuals." Recent theories emphasize the role of the prefrontal cortex in processing counterfactual outcomes in decision-making, although how subcortical regions contribute to this process remains to be elucidated. Here we report a clear distinction among the roles of the orbitofrontal cortex, ventral striatum and midbrain dopamine neurons in processing counterfactual outcomes in monkeys. Our findings suggest that actually gained and counterfactual outcome signals are both processed in the cortico-subcortical network constituted by these regions but in distinct manners and integrated only in the orbitofrontal cortex in a way to compare these outcomes. This study extends the prefrontal theory of counterfactual thinking and provides key insights regarding how the prefrontal cortex cooperates with subcortical regions to make decisions using counterfactual information.
Collapse
Affiliation(s)
- Mengxi Yun
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
- Graduate School of Comprehensive Human Science, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA, USA
- Transborder Medical Research Center, University of Tsukuba, Tsukuba, Ibaraki 305-8577, Japan
| | - Masafumi Nejime
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
| | - Takashi Kawai
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
| | - Jun Kunimatsu
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
- Transborder Medical Research Center, University of Tsukuba, Tsukuba, Ibaraki 305-8577, Japan
| | - Hiroshi Yamada
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
- Transborder Medical Research Center, University of Tsukuba, Tsukuba, Ibaraki 305-8577, Japan
| | - HyungGoo R. Kim
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Biomedical Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon 16419, Republic of Korea
| | - Masayuki Matsumoto
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
- Transborder Medical Research Center, University of Tsukuba, Tsukuba, Ibaraki 305-8577, Japan
| |
Collapse
|
6
|
Bernhard RM, Frankland SM, Plunkett D, Sievers B, Greene JD. Evidence for Spinozan "Unbelieving" in the Right Inferior Prefrontal Cortex. J Cogn Neurosci 2023; 35:659-680. [PMID: 36638227 DOI: 10.1162/jocn_a_01964] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023]
Abstract
Humans can think about possible states of the world without believing in them, an important capacity for high-level cognition. Here, we use fMRI and a novel "shell game" task to test two competing theories about the nature of belief and its neural basis. According to the Cartesian theory, information is first understood, then assessed for veracity, and ultimately encoded as either believed or not believed. According to the Spinozan theory, comprehension entails belief by default, such that understanding without believing requires an additional process of "unbelieving." Participants (n = 70) were experimentally induced to have beliefs, desires, or mere thoughts about hidden states of the shell game (e.g., believing that the dog is hidden in the upper right corner). That is, participants were induced to have specific "propositional attitudes" toward specific "propositions" in a controlled way. Consistent with the Spinozan theory, we found that thinking about a proposition without believing it is associated with increased activation of the right inferior frontal gyrus. This was true whether the hidden state was desired by the participant (because of reward) or merely thought about. These findings are consistent with a version of the Spinozan theory whereby unbelieving is an inhibitory control process. We consider potential implications of these results for the phenomena of delusional belief and wishful thinking.
Collapse
|
7
|
A Reinforcement Meta-Learning framework of executive function and information demand. Neural Netw 2023; 157:103-113. [DOI: 10.1016/j.neunet.2022.10.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2021] [Revised: 09/05/2022] [Accepted: 10/06/2022] [Indexed: 11/09/2022]
|
8
|
Janssen P, Isa T, Lanciego J, Leech K, Logothetis N, Poo MM, Mitchell AS. Visualizing advances in the future of primate neuroscience research. CURRENT RESEARCH IN NEUROBIOLOGY 2022; 4:100064. [PMID: 36582401 PMCID: PMC9792703 DOI: 10.1016/j.crneur.2022.100064] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2022] [Revised: 09/30/2022] [Accepted: 11/24/2022] [Indexed: 12/15/2022] Open
Abstract
Future neuroscience and biomedical projects involving non-human primates (NHPs) remain essential in our endeavors to understand the complexities and functioning of the mammalian central nervous system. In so doing, the NHP neuroscience researcher must be allowed to incorporate state-of-the-art technologies, including the use of novel viral vectors, gene therapy and transgenic approaches to answer continuing and emerging research questions that can only be addressed in NHP research models. This perspective piece captures these emerging technologies and some specific research questions they can address. At the same time, we highlight some current caveats to global NHP research and collaborations including the lack of common ethical and regulatory frameworks for NHP research, the limitations involving animal transportation and exports, and the ongoing influence of activist groups opposed to NHP research.
Collapse
Affiliation(s)
- Peter Janssen
- Laboratory for Neuro- and Psychophysiology, KU Leuven, Belgium
| | - Tadashi Isa
- Graduate School of Medicine, Kyoto University, Japan
| | - Jose Lanciego
- Department Neurosciences, Center for Applied Medical Research (CIMA), University of Navarra, CiberNed., Pamplona, Spain
| | - Kirk Leech
- European Animal Research Association, United Kingdom
| | - Nikos Logothetis
- International Center for Primate Brain Research, Shanghai, China
| | - Mu-Ming Poo
- International Center for Primate Brain Research, Shanghai, China
| | - Anna S. Mitchell
- School of Psychology, Speech and Hearing, University of Canterbury, Christchurch, New Zealand,Department of Experimental Psychology, University of Oxford, United Kingdom,Corresponding author. School of Psychology, Speech and Hearing, University of Canterbury, Christchurch, New Zealand.
| |
Collapse
|
9
|
Li Y, Daddaoua N, Horan M, Foley NC, Gottlieb J. Uncertainty modulates visual maps during noninstrumental information demand. Nat Commun 2022; 13:5911. [PMID: 36207316 PMCID: PMC9547007 DOI: 10.1038/s41467-022-33585-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Accepted: 09/22/2022] [Indexed: 11/23/2022] Open
Abstract
Animals are intrinsically motivated to obtain information independently of instrumental incentives. This motivation depends on two factors: a desire to resolve uncertainty by gathering accurate information and a desire to obtain positively-valenced observations, which predict favorable rather than unfavorable outcomes. To understand the neural mechanisms, we recorded parietal cortical activity implicated in prioritizing stimuli for spatial attention and gaze, in a task in which monkeys were free (but not trained) to obtain information about probabilistic non-contingent rewards. We show that valence and uncertainty independently modulated parietal neuronal activity, and uncertainty but not reward-related enhancement consistently correlated with behavioral sensitivity. The findings suggest uncertainty-driven and valence-driven information demand depend on partially distinct pathways, with the former being consistently related to parietal responses and the latter depending on additional mechanisms implemented in downstream structures. Curiosity is motivated by uncertainty and valence, but how uncertainty and valence are encoded in the brain remains poorly understood. Here, the authors show that parietal neurons are enhanced by both factors, but that they specifically predict visual information seeking based on uncertainty.
Collapse
Affiliation(s)
- Yvonne Li
- Department of Neuroscience, Columbia University, New York, NY, USA.,Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Nabil Daddaoua
- Department of Neuroscience, Columbia University, New York, NY, USA.,Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Mattias Horan
- Department of Neuroscience, Columbia University, New York, NY, USA
| | - Nicholas C Foley
- Department of Neuroscience, Columbia University, New York, NY, USA.,Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Jacqueline Gottlieb
- Department of Neuroscience, Columbia University, New York, NY, USA. .,Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA. .,Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| |
Collapse
|
10
|
Choi JC, Park HJ, Park JA, Kang DR, Choi YS, Choi S, Lee HG, Choi JH, Choi IH, Yoon MW, Lee JM, Kim J. The increased analgesic efficacy of cold therapy after an unsuccessful analgesic experience is associated with inferior parietal lobule activation. Sci Rep 2022; 12:14687. [PMID: 36038625 PMCID: PMC9424269 DOI: 10.1038/s41598-022-18181-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2021] [Accepted: 08/08/2022] [Indexed: 11/09/2022] Open
Abstract
Prior experiences of successful and failed treatments are known to influence the efficacy of a newly applied treatment. However, whether that carry-over effect applies to non-pharmacological treatments is unknown. This study investigated how a failed treatment history with placebo analgesic cream affected the therapeutic outcomes of cold-pack treatment. The neural correlates underlying those effects were also explored using functional magnetic resonance imaging. The effect of the placebo analgesic cream was induced using placebo conditioning with small (44.5 °C to 43.7 °C, negative experience) and large (44.5 °C to 40.0 °C, positive experience) thermal stimuli changes. After the placebo conditioning, brain responses and self-reported evaluations of the effect of subsequent treatment with a cold-pack were contrasted between the two groups. The negative experience group reported less pain and lower anxiety scores in the cold-pack condition than the positive experience group and exhibited significantly greater activation in the right inferior parietal lobule (IPL), which is known to be involved in pain relief. These findings suggest that an unsatisfying experience with an initial pain-relief treatment could increase the expectations for the complementary treatment outcome and improve the analgesic effect of the subsequent treatment. The IPL could be associated with this expectation-induced pain relief process.
Collapse
Affiliation(s)
- Jae Chan Choi
- Department of Anesthesiology and Pain Medicine, Yonsei University Wonju College of Medicine, Wonju, 26426, Republic of Korea.,Cham Brain Health Institute, 08807, Seoul, Republic of Korea
| | - Hae-Jeong Park
- Department of Nuclear Medicine, Graduate School of Medical Science, BrainKorea21Project, Yonsei University College of Medicine, Seoul, 3722, Republic of Korea
| | - Jeong A Park
- Alzza Health Institute, Seoul, Republic of Korea
| | - Dae Ryong Kang
- Department of Precision Medicine & Biostatistics, Yonsei University Wonju College of Medicine, Wonju-si, 26426, Republic of Korea
| | - Young-Seok Choi
- Department of Electronics and Communications Engineering, Kwangwoon University, Seoul, 01897, Republic of Korea
| | - SoHyun Choi
- Department of Precision Medicine & Biostatistics, Yonsei University Wonju College of Medicine, Wonju-si, 26426, Republic of Korea
| | - Hong Gyu Lee
- Department of Radiology, Yonsei University Wonju College of Medicine, Wonju-si, 26426, Republic of Korea
| | - Jun-Ho Choi
- Department of Practical Arts Education, Chinju National University of Education, Jinju-si, 52673, Republic of Korea
| | - In-Ho Choi
- Department of Architectural Design, Kaywon University of Art and Design, Uiwang-si, 16038, Republic of Korea
| | - Min Woo Yoon
- Department of Anesthesiology and Pain Medicine, Yonsei University Wonju College of Medicine, Wonju, 26426, Republic of Korea
| | - Jong-Min Lee
- Department of Biomedical Engineering, Hanyang University, Seoul, 4763, Republic of Korea
| | - Jinhee Kim
- School of Psychology, Korea University, Seoul, 2841, Republic of Korea.
| |
Collapse
|
11
|
Miller KJ, Botvinick MM, Brody CD. Value representations in the rodent orbitofrontal cortex drive learning, not choice. eLife 2022; 11:64575. [PMID: 35975792 PMCID: PMC9462853 DOI: 10.7554/elife.64575] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2020] [Accepted: 08/01/2022] [Indexed: 11/13/2022] Open
Abstract
Humans and animals make predictions about the rewards they expect to receive in different situations. In formal models of behavior, these predictions are known as value representations, and they play two very different roles. Firstly, they drive choice: the expected values of available options are compared to one another, and the best option is selected. Secondly, they support learning: expected values are compared to rewards actually received, and future expectations are updated accordingly. Whether these different functions are mediated by different neural representations remains an open question. Here, we employ a recently developed multi-step task for rats that computationally separates learning from choosing. We investigate the role of value representations in the rodent orbitofrontal cortex, a key structure for value-based cognition. Electrophysiological recordings and optogenetic perturbations indicate that these representations do not directly drive choice. Instead, they signal expected reward information to a learning process elsewhere in the brain that updates choice mechanisms.
Collapse
Affiliation(s)
| | | | - Carlos D Brody
- Princeton Neuroscience Institute, Princeton University, Princeton, United States
| |
Collapse
|
12
|
Witkowski PP, Park SA, Boorman ED. Neural mechanisms of credit assignment for inferred relationships in a structured world. Neuron 2022; 110:2680-2690.e9. [PMID: 35714610 DOI: 10.1016/j.neuron.2022.05.021] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Revised: 04/12/2022] [Accepted: 05/18/2022] [Indexed: 10/18/2022]
Abstract
Animals abstract compact representations of a task's structure, which supports accelerated learning and flexible behavior. Whether and how such abstracted representations may be used to assign credit for inferred, but unobserved, relationships in structured environments are unknown. We develop a hierarchical reversal-learning task and Bayesian learning model to assess the computational and neural mechanisms underlying how humans infer specific choice-outcome associations via structured knowledge. We find that the medial prefrontal cortex (mPFC) efficiently represents hierarchically related choice-outcome associations governed by the same latent cause, using a generalized code to assign credit for both experienced and inferred outcomes. Furthermore, the mPFC and lateral orbitofrontal cortex track the current "position" within a latent association space that generalizes over stimuli. Collectively, these findings demonstrate the importance of both tracking the current position in an abstracted task space and efficient, generalizable representations in the prefrontal cortex for supporting flexible learning and inference in structured environments.
Collapse
Affiliation(s)
- Phillip P Witkowski
- Center for Mind and Brain, University of California, Davis, Davis, CA 95618; Department of Psychology, University of California, Davis, Davis, CA 95618.
| | - Seongmin A Park
- Center for Mind and Brain, University of California, Davis, Davis, CA 95618
| | - Erie D Boorman
- Center for Mind and Brain, University of California, Davis, Davis, CA 95618; Department of Psychology, University of California, Davis, Davis, CA 95618.
| |
Collapse
|
13
|
Klappenbach M, Lara AE, Locatelli FF. Honey bees can store and retrieve independent memory traces after complex experiences that combine appetitive and aversive associations. J Exp Biol 2022; 225:275573. [PMID: 35485192 DOI: 10.1242/jeb.244229] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2022] [Accepted: 04/19/2022] [Indexed: 11/20/2022]
Abstract
Real-world experiences do often mix appetitive and aversive events. Understanding the ability of animals to extract, store and use this information is an important issue in neurobiology. We used honey bees as model organism to study learning and memory after a differential conditioning that combines appetitive and aversive training trials. First of all, we describe an aversive conditioning paradigm that constitutes a clear opposite of the well known appetitive olfactory conditioning of the proboscis extension response. A neutral odour is presented paired with the bitter substance quinine. Aversive memory is evidenced later as an odour-specific impairment in appetitive conditioning. Then we tested the effect of mixing appetitive and aversive conditioning trials distributed along the same training session. Differential conditioning protocols like this were used before to study the ability to discriminate odours, however they were not focused on whether appetitive and aversive memories are formed. We found that after a differential conditioning, honey bees establish independent appetitive and aversive memories that do not interfere with each other during acquisition or storage. Finally, we moved the question forward to retrieval and memory expression to evaluate what happens when appetitive and the aversive learned odours are mixed during test. Interestingly, opposite memories compete in a way that they do not cancel each other out. Honey bees showed the ability to switch from expressing appetitive to aversive memory depending on their satiation level.
Collapse
Affiliation(s)
- Martín Klappenbach
- Departamento de Fisiología, Biología Molecular y Celular, Facultad de Ciencias Exactas y Naturales, Instituto de Fisiología, Biología Molecular y Neurociencias, Universidad de Buenos Aires-CONICET), Ciudad Universitaria, Buenos Aires, Argentina
| | - Agustín E Lara
- Departamento de Fisiología, Biología Molecular y Celular, Facultad de Ciencias Exactas y Naturales, Instituto de Fisiología, Biología Molecular y Neurociencias, Universidad de Buenos Aires-CONICET), Ciudad Universitaria, Buenos Aires, Argentina
| | - Fernando F Locatelli
- Departamento de Fisiología, Biología Molecular y Celular, Facultad de Ciencias Exactas y Naturales, Instituto de Fisiología, Biología Molecular y Neurociencias, Universidad de Buenos Aires-CONICET), Ciudad Universitaria, Buenos Aires, Argentina
| |
Collapse
|
14
|
Barnes J, Blair MR, Walshe RC, Tupper PF. LAG-1: A dynamic, integrative model of learning, attention, and gaze. PLoS One 2022; 17:e0259511. [PMID: 35298465 PMCID: PMC8929614 DOI: 10.1371/journal.pone.0259511] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2021] [Accepted: 10/21/2021] [Indexed: 11/19/2022] Open
Abstract
It is clear that learning and attention interact, but it is an ongoing challenge to integrate their psychological and neurophysiological descriptions. Here we introduce LAG-1, a dynamic neural field model of learning, attention and gaze, that we fit to human learning and eye-movement data from two category learning experiments. LAG-1 comprises three control systems: one for visuospatial attention, one for saccadic timing and control, and one for category learning. The model is able to extract a kind of information gain from pairwise differences in simple associations between visual features and categories. Providing this gain as a reentrant signal with bottom-up visual information, and in top-down spatial priority, appropriately influences the initiation of saccades. LAG-1 provides a moment-by-moment simulation of the interactions of learning and gaze, and thus simultaneously produces phenomena on many timescales, from the duration of saccades and gaze fixations, to the response times for trials, to the slow optimization of attention toward task relevant information across a whole experiment. With only three free parameters (learning rate, trial impatience, and fixation impatience) LAG-1 produces qualitatively correct fits for learning, behavioural timing and eye movement measures, and also for previously unmodelled empirical phenomena (e.g., fixation orders showing stimulus-specific attention, and decreasing fixation counts during feedback). Because LAG-1 is built to capture attention and gaze generally, we demonstrate how it can be applied to other phenomena of visual cognition such as the free viewing of visual stimuli, visual search, and covert attention.
Collapse
Affiliation(s)
- Jordan Barnes
- Department of Psychology, Simon Fraser University, Burnaby, BC, Canada
| | - Mark R. Blair
- Department of Psychology, Simon Fraser University, Burnaby, BC, Canada
- * E-mail:
| | - R. Calen Walshe
- Center for Perceptual Systems, University of Texas, Austin, Texas, United States of America
| | - Paul F. Tupper
- Department of Mathematics, Simon Fraser University, Burnaby, BC, Canada
| |
Collapse
|
15
|
Ott T, Masset P, Gouvêa TS, Kepecs A. Apparent sunk cost effect in rational agents. SCIENCE ADVANCES 2022; 8:eabi7004. [PMID: 35148186 PMCID: PMC8836799 DOI: 10.1126/sciadv.abi7004] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/28/2021] [Accepted: 12/20/2021] [Indexed: 06/14/2023]
Abstract
Rational decision makers aim to maximize their gains, but humans and other animals often fail to do so, exhibiting biases and distortions in their choice behavior. In a recent study of economic decisions, humans, mice, and rats were reported to succumb to the sunk cost fallacy, making decisions based on irrecoverable past investments to the detriment of expected future returns. We challenge this interpretation because it is subject to a statistical fallacy, a form of attrition bias, and the observed behavior can be explained without invoking a sunk cost-dependent mechanism. Using a computational model, we illustrate how a rational decision maker with a reward-maximizing decision strategy reproduces the reported behavioral pattern and propose an improved task design to dissociate sunk costs from fluctuations in decision valuation. Similar statistical confounds may be common in analyses of cognitive behaviors, highlighting the need to use causal statistical inference and generative models for interpretation.
Collapse
Affiliation(s)
- Torben Ott
- Bernstein Center for Computational Neuroscience Berlin, Humboldt University of Berlin, Berlin, Germany
- Department of Neuroscience and Department of Psychiatry, Washington University in St. Louis, St. Louis, MO, USA
| | - Paul Masset
- Department of Molecular and Cellular Biology and Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Thiago S. Gouvêa
- German Research Center for Artificial Intelligence (DFKI), Oldenburg, Germany
| | - Adam Kepecs
- Department of Neuroscience and Department of Psychiatry, Washington University in St. Louis, St. Louis, MO, USA
| |
Collapse
|
16
|
Goh AXA, Bennett D, Bode S, Chong TTJ. Neurocomputational mechanisms underlying the subjective value of information. Commun Biol 2021; 4:1346. [PMID: 34903804 PMCID: PMC8669024 DOI: 10.1038/s42003-021-02850-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2021] [Accepted: 11/04/2021] [Indexed: 11/09/2022] Open
Abstract
Humans have a striking desire to actively seek new information, even when it is devoid of any instrumental utility. However, the mechanisms that drive individuals' subjective preference for information remain unclear. Here, we used fMRI to examine the processing of subjective information value, by having participants decide how much effort they were willing to trade-off for non-instrumental information. We showed that choices were best described by a model that accounted for: (1) the variability in individuals' estimates of uncertainty, (2) their desire to reduce that uncertainty, and (3) their subjective preference for positively valenced information. Model-based analyses revealed the anterior cingulate as a key node that encodes the subjective value of information across multiple stages of decision-making - including when information was prospectively valued, and when the outcome was definitively delivered. These findings emphasise the multidimensionality of information value, and reveal the neurocomputational mechanisms underlying the variability in individuals' desire to physically pursue informative outcomes.
Collapse
Affiliation(s)
- Ariel X-A Goh
- Turner Institute for Brain and Mental Health, Monash University, Melbourne, VIC, 3800, Australia
- School of Psychological Sciences, Monash University, Melbourne, VIC, 3800, Australia
| | - Daniel Bennett
- Department of Psychiatry, Monash University, Melbourne, VIC, 3800, Australia
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, 08540, USA
| | - Stefan Bode
- Melbourne School of Psychological Sciences, University of Melbourne, Melbourne, VIC, 3010, Australia
| | - Trevor T-J Chong
- Turner Institute for Brain and Mental Health, Monash University, Melbourne, VIC, 3800, Australia.
- School of Psychological Sciences, Monash University, Melbourne, VIC, 3800, Australia.
- Department of Neurology, Alfred Health, Melbourne, VIC, 3004, Australia.
- Department of Clinical Neurosciences, St Vincent's Hospital, Melbourne, VIC, 3065, Australia.
| |
Collapse
|
17
|
Lucantonio F, Kim E, Su Z, Chang AJ, Bari BA, Cohen JY. Aversive stimuli bias corticothalamic responses to motivationally significant cues. eLife 2021; 10:57634. [PMID: 34738905 PMCID: PMC8570692 DOI: 10.7554/elife.57634] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2020] [Accepted: 10/11/2021] [Indexed: 11/19/2022] Open
Abstract
Making predictions about future rewards or punishments is fundamental to adaptive behavior. These processes are influenced by prior experience. For example, prior exposure to aversive stimuli or stressors changes behavioral responses to negative- and positive-value predictive cues. Here, we demonstrate a role for medial prefrontal cortex (mPFC) neurons projecting to the paraventricular nucleus of the thalamus (PVT; mPFC→PVT) in this process. We found that a history of aversive stimuli negatively biased behavioral responses to motivationally relevant cues in mice and that this negative bias was associated with hyperactivity in mPFC→PVT neurons during exposure to those cues. Furthermore, artificially mimicking this hyperactive response with selective optogenetic excitation of the same pathway recapitulated the negative behavioral bias induced by aversive stimuli, whereas optogenetic inactivation of mPFC→PVT neurons prevented the development of the negative bias. Together, our results highlight how information flow within the mPFC→PVT circuit is critical for making predictions about motivationally-relevant outcomes as a function of prior experience.
Collapse
Affiliation(s)
- Federica Lucantonio
- The Solomon H Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, Baltimore, United States
| | - Eunyoung Kim
- The Solomon H Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, Baltimore, United States
| | - Zhixiao Su
- The Solomon H Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, Baltimore, United States
| | - Anna J Chang
- The Solomon H Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, Baltimore, United States
| | - Bilal A Bari
- The Solomon H Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, Baltimore, United States
| | - Jeremiah Y Cohen
- The Solomon H Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, Baltimore, United States
| |
Collapse
|
18
|
Pirrone A, Reina A, Stafford T, Marshall JAR, Gobet F. Magnitude-sensitivity: rethinking decision-making. Trends Cogn Sci 2021; 26:66-80. [PMID: 34750080 DOI: 10.1016/j.tics.2021.10.006] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2021] [Revised: 10/05/2021] [Accepted: 10/05/2021] [Indexed: 11/25/2022]
Abstract
Magnitude-sensitivity refers to the result that performance in decision-making, across domains and organisms, is affected by the total value of the possible alternatives. This simple result offers a window into fundamental issues in decision-making and has led to a reconsideration of ecological decision-making, prominent computational models of decision-making, and optimal decision-making. Moreover, magnitude-sensitivity has inspired the design of new robotic systems that exploit natural solutions and apply optimal decision-making policies. In this article, we review the key theoretical and empirical results about magnitude-sensitivity and highlight the importance that this phenomenon has for the understanding of decision-making. Furthermore, we discuss open questions and ideas for future research.
Collapse
Affiliation(s)
- Angelo Pirrone
- Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science, London, UK.
| | - Andreagiovanni Reina
- Institute for Interdisciplinary Studies on Artificial Intelligence (IRIDIA), Université Libre de Bruxelles, Brussels, Belgium
| | - Tom Stafford
- Department of Psychology, University of Sheffield, Sheffield, UK
| | | | - Fernand Gobet
- Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science, London, UK
| |
Collapse
|
19
|
Torigoe M, Islam T, Kakinuma H, Fung CCA, Isomura T, Shimazaki H, Aoki T, Fukai T, Okamoto H. Zebrafish capable of generating future state prediction error show improved active avoidance behavior in virtual reality. Nat Commun 2021; 12:5712. [PMID: 34588436 PMCID: PMC8481257 DOI: 10.1038/s41467-021-26010-7] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2020] [Accepted: 09/06/2021] [Indexed: 11/08/2022] Open
Abstract
Animals make decisions under the principle of reward value maximization and surprise minimization. It is still unclear how these principles are represented in the brain and are reflected in behavior. We addressed this question using a closed-loop virtual reality system to train adult zebrafish for active avoidance. Analysis of the neural activity of the dorsal pallium during training revealed neural ensembles assigning rules to the colors of the surrounding walls. Additionally, one third of fish generated another ensemble that becomes activated only when the real perceived scenery shows discrepancy from the predicted favorable scenery. The fish with the latter ensemble escape more efficiently than the fish with the former ensembles alone, even though both fish have successfully learned to escape, consistent with the hypothesis that the latter ensemble guides zebrafish to take action to minimize this prediction error. Our results suggest that zebrafish can use both principles of goal-directed behavior, but with different behavioral consequences depending on the repertoire of the adopted principles.
Collapse
Affiliation(s)
- Makio Torigoe
- Lab. for Neural Circuit Dynamics of Decision Making, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan
| | - Tanvir Islam
- Lab. for Neural Circuit Dynamics of Decision Making, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan
- RIKEN CBS-Kao Collaboration Center, Wako, Saitama, 351-0198, Japan
| | - Hisaya Kakinuma
- Lab. for Neural Circuit Dynamics of Decision Making, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan
- RIKEN CBS-Kao Collaboration Center, Wako, Saitama, 351-0198, Japan
| | - Chi Chung Alan Fung
- Neural Coding and Brain Computing Unit, Okinawa Institute of Science and Technology, Onna-son, Okinawa, 904-0495, Japan
| | - Takuya Isomura
- Brain Intelligence Theory Unit, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan
| | - Hideaki Shimazaki
- Center for Human Nature, Artificial Intelligence, and Neuroscience (CHAIN), Hokkaido University, Sapporo, Hokkaido, 060-0812, Japan
| | - Tazu Aoki
- Lab. for Neural Circuit Dynamics of Decision Making, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan
| | - Tomoki Fukai
- Neural Coding and Brain Computing Unit, Okinawa Institute of Science and Technology, Onna-son, Okinawa, 904-0495, Japan
| | - Hitoshi Okamoto
- Lab. for Neural Circuit Dynamics of Decision Making, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan.
- RIKEN CBS-Kao Collaboration Center, Wako, Saitama, 351-0198, Japan.
| |
Collapse
|
20
|
Sareen PF, McCurdy LY, Nitabach MN. A neuronal ensemble encoding adaptive choice during sensory conflict in Drosophila. Nat Commun 2021; 12:4131. [PMID: 34226544 PMCID: PMC8257655 DOI: 10.1038/s41467-021-24423-y] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2020] [Accepted: 06/18/2021] [Indexed: 01/02/2023] Open
Abstract
Feeding decisions are fundamental to survival, and decision making is often disrupted in disease. Here, we show that neural activity in a small population of neurons projecting to the fan-shaped body higher-order central brain region of Drosophila represents food choice during sensory conflict. We found that food deprived flies made tradeoffs between appetitive and aversive values of food. We identified an upstream neuropeptidergic and dopaminergic network that relays internal state and other decision-relevant information to a specific subset of fan-shaped body neurons. These neurons were strongly inhibited by the taste of the rejected food choice, suggesting that they encode behavioral food choice. Our findings reveal that fan-shaped body taste responses to food choices are determined not only by taste quality, but also by previous experience (including choice outcome) and hunger state, which are integrated in the fan-shaped body to encode the decision before relay to downstream motor circuits for behavioral implementation.
Collapse
Affiliation(s)
- Preeti F Sareen
- Department of Cellular and Molecular Physiology, Yale University, New Haven, CT, USA
| | - Li Yan McCurdy
- Department of Cellular and Molecular Physiology, Yale University, New Haven, CT, USA
- Interdepartmental Neuroscience Program, Yale University, New Haven, CT, USA
| | - Michael N Nitabach
- Department of Cellular and Molecular Physiology, Yale University, New Haven, CT, USA.
- Department of Genetics, Yale University, New Haven, CT, USA.
- Department of Neuroscience, Yale University, New Haven, CT, USA.
| |
Collapse
|
21
|
de Gee JW, Correa CMC, Weaver M, Donner TH, van Gaal S. Pupil Dilation and the Slow Wave ERP Reflect Surprise about Choice Outcome Resulting from Intrinsic Variability in Decision Confidence. Cereb Cortex 2021; 31:3565-3578. [PMID: 33822917 PMCID: PMC8196307 DOI: 10.1093/cercor/bhab032] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2020] [Revised: 01/27/2021] [Accepted: 01/27/2021] [Indexed: 12/01/2022] Open
Abstract
Central to human and animal cognition is the ability to learn from feedback in order to optimize future rewards. Such a learning signal might be encoded and broadcasted by the brain's arousal systems, including the noradrenergic locus coeruleus. Pupil responses and the positive slow wave component of event-related potentials reflect rapid changes in the arousal level of the brain. Here, we ask whether and how these variables may reflect surprise: the mismatch between one's expectation about being correct and the outcome of a decision, when expectations fluctuate due to internal factors (e.g., engagement). We show that during an elementary decision task in the face of uncertainty both physiological markers of phasic arousal reflect surprise. We further show that pupil responses and slow wave event-related potential are unrelated to each other and that prediction error computations depend on feedback awareness. These results further advance our understanding of the role of central arousal systems in decision-making under uncertainty.
Collapse
Affiliation(s)
- Jan Willem de Gee
- Department of Psychology, Amsterdam Brain & Cognition (ABC), University of Amsterdam, Nieuwe Achtergracht 129-B, 1018WS, Amsterdam, the Netherlands
- Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Building N43, Martinistraße 52, 20246, Hamburg, Germany
- Department of Neuroscience, Baylor College of Medicine, 1 Baylor Plaza, Houston, TX 77030, USA
- Jan and Dan Duncan Neurological Research Institute, Texas Children’s Hospital, 1250 Moursund St, Houston, TX 77030, USA
| | - Camile M C Correa
- Department of Psychology, Amsterdam Brain & Cognition (ABC), University of Amsterdam, Nieuwe Achtergracht 129-B, 1018WS, Amsterdam, the Netherlands
- Centre of Functionally Integrative Neuroscience, Aarhus University, 44 Nørrebrogade Building 1A, 8000 Aarhus, Denmark
| | - Matthew Weaver
- Department of Psychology, Amsterdam Brain & Cognition (ABC), University of Amsterdam, Nieuwe Achtergracht 129-B, 1018WS, Amsterdam, the Netherlands
| | - Tobias H Donner
- Department of Psychology, Amsterdam Brain & Cognition (ABC), University of Amsterdam, Nieuwe Achtergracht 129-B, 1018WS, Amsterdam, the Netherlands
- Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Building N43, Martinistraße 52, 20246, Hamburg, Germany
| | - Simon van Gaal
- Department of Psychology, Amsterdam Brain & Cognition (ABC), University of Amsterdam, Nieuwe Achtergracht 129-B, 1018WS, Amsterdam, the Netherlands
| |
Collapse
|
22
|
Ikegami T, Ganesh G, Gibo TL, Yoshioka T, Osu R, Kawato M. Hierarchical motor adaptations negotiate failures during force field learning. PLoS Comput Biol 2021; 17:e1008481. [PMID: 33872304 PMCID: PMC8084335 DOI: 10.1371/journal.pcbi.1008481] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2020] [Revised: 04/29/2021] [Accepted: 03/24/2021] [Indexed: 11/19/2022] Open
Abstract
Humans have the amazing ability to learn the dynamics of the body and environment to develop motor skills. Traditional motor studies using arm reaching paradigms have viewed this ability as the process of ‘internal model adaptation’. However, the behaviors have not been fully explored in the case when reaches fail to attain the intended target. Here we examined human reaching under two force fields types; one that induces failures (i.e., target errors), and the other that does not. Our results show the presence of a distinct failure-driven adaptation process that enables quick task success after failures, and before completion of internal model adaptation, but that can result in persistent changes to the undisturbed trajectory. These behaviors can be explained by considering a hierarchical interaction between internal model adaptation and the failure-driven adaptation of reach direction. Our findings suggest that movement failure is negotiated using hierarchical motor adaptations by humans. How do we improve actions after a movement failure? Although negotiating movement failures is obviously crucial, previous motor-control studies have predominantly examined human movement adaptations in the absence of failures, and it remains unclear how failures affect subsequent movement adaptations. Here we examined this issue by developing a novel force field adaptation task where the hand movement during an arm reaching is perturbed by novel forces that induce a large target error, that is a failure. Our experimental observation and computational modeling show that, in addition to the popular ‘internal model learning’ process of motor adaptations, humans also utilize a ‘failure-negotiating’ process, that enables them to quickly improve movements in the presence of failure, even at the expense of increased arm trajectory deflections, which are subsequently reduced gradually with training after the achievement of the task success. Our results suggest that a hierarchical interaction between these two processes is a key for humans to negotiate movement failures.
Collapse
Affiliation(s)
- Tsuyoshi Ikegami
- Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology (NICT), Osaka, Japan
- Brain Information Communication Research Laboratory Group, ATR, Kyoto, Japan
- Graduate School of Frontier Biosciences, Osaka University, Osaka, Japan
- * E-mail:
| | - Gowrishankar Ganesh
- Center for Information and Neural Networks (CiNet), National Institute of Information and Communications Technology (NICT), Osaka, Japan
- Brain Information Communication Research Laboratory Group, ATR, Kyoto, Japan
- Centre National de la Recherche Scientifique (CNRS), Universite Montpellier (UM) Laboratoire d’Informatique, de Robotique et de Microelectronique de, Montpellier (LIRMM), Montpellier, France
| | - Tricia L. Gibo
- Brain Information Communication Research Laboratory Group, ATR, Kyoto, Japan
- Emergo by UL, Utrecht, The Netherlands
| | - Toshinori Yoshioka
- Brain Information Communication Research Laboratory Group, ATR, Kyoto, Japan
| | - Rieko Osu
- Brain Information Communication Research Laboratory Group, ATR, Kyoto, Japan
- Faculty of Human Sciences, Waseda University, Saitama, Japan
| | - Mitsuo Kawato
- Brain Information Communication Research Laboratory Group, ATR, Kyoto, Japan
| |
Collapse
|
23
|
Saleh Y, Le Heron C, Petitet P, Veldsman M, Drew D, Plant O, Schulz U, Sen A, Rothwell PM, Manohar S, Husain M. Apathy in small vessel cerebrovascular disease is associated with deficits in effort-based decision making. Brain 2021; 144:1247-1262. [PMID: 33734344 PMCID: PMC8240747 DOI: 10.1093/brain/awab013] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2020] [Revised: 10/23/2020] [Accepted: 11/04/2020] [Indexed: 11/16/2022] Open
Abstract
Patients with small vessel cerebrovascular disease frequently suffer from apathy, a debilitating neuropsychiatric syndrome, the underlying mechanisms of which remain to be established. Here we investigated the hypothesis that apathy is associated with disrupted decision making in effort-based decision making, and that these alterations are associated with abnormalities in the white matter network connecting brain regions that underpin such decisions. Eighty-two patients with MRI evidence of small vessel disease were assessed using a behavioural paradigm as well as diffusion weighted MRI. The decision-making task involved accepting or rejecting monetary rewards in return for performing different levels of physical effort (hand grip force). Choice data and reaction times were integrated into a drift diffusion model that framed decisions to accept or reject offers as stochastic processes approaching a decision boundary with a particular drift rate. Tract-based spatial statistics were used to assess the relationship between white matter tract integrity and apathy, while accounting for depression. Overall, patients with apathy accepted significantly fewer offers on this decision-making task. Notably, while apathetic patients were less responsive to low rewards, they were also significantly averse to investing in high effort. Significant reductions in white matter integrity were observed to be specifically related to apathy, but not to depression. These included pathways connecting brain regions previously implicated in effort-based decision making in healthy people. The drift rate to decision parameter was significantly associated with both apathy and altered white matter tracts, suggesting that both brain and behavioural changes in apathy are associated with this single parameter. On the other hand, depression was associated with an increase in the decision boundary, consistent with an increase in the amount of evidence required prior to making a decision. These findings demonstrate altered effort-based decision making for reward in apathy, and also highlight dissociable mechanisms underlying apathy and depression in small vessel disease. They provide clear potential brain and behavioural targets for future therapeutic interventions, as well as modelling parameters that can be used to measure the effects of treatment at the behavioural level.
Collapse
Affiliation(s)
- Youssuf Saleh
- Nuffield Department of Clinical Neurosciences, University of Oxford, Level 6, West Wing, John Radcliffe Hospital, Oxford, OX3 9DU, UK.,Department of Experimental Psychology, University of Oxford, Anna Watts Building, Radcliffe Observatory Quarter, Oxford, OX2 6GG, UK
| | - Campbell Le Heron
- Nuffield Department of Clinical Neurosciences, University of Oxford, Level 6, West Wing, John Radcliffe Hospital, Oxford, OX3 9DU, UK.,New Zealand Brain Research Institute, Christchurch 8011, New Zealand.,Department of Medicine, University of Otago, Christchurch 8011, New Zealand
| | - Pierre Petitet
- Department of Experimental Psychology, University of Oxford, Anna Watts Building, Radcliffe Observatory Quarter, Oxford, OX2 6GG, UK
| | - Michele Veldsman
- Department of Experimental Psychology, University of Oxford, Anna Watts Building, Radcliffe Observatory Quarter, Oxford, OX2 6GG, UK
| | - Daniel Drew
- Department of Experimental Psychology, University of Oxford, Anna Watts Building, Radcliffe Observatory Quarter, Oxford, OX2 6GG, UK
| | - Olivia Plant
- Department of Experimental Psychology, University of Oxford, Anna Watts Building, Radcliffe Observatory Quarter, Oxford, OX2 6GG, UK
| | - Ursula Schulz
- Nuffield Department of Clinical Neurosciences, University of Oxford, Level 6, West Wing, John Radcliffe Hospital, Oxford, OX3 9DU, UK
| | - Arjune Sen
- Nuffield Department of Clinical Neurosciences, University of Oxford, Level 6, West Wing, John Radcliffe Hospital, Oxford, OX3 9DU, UK.,Oxford Epilepsy Research Group, NIHR Biomedical Research Centre, Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, Oxford, OX3 9DU, UK
| | - Peter M Rothwell
- Nuffield Department of Clinical Neurosciences, University of Oxford, Level 6, West Wing, John Radcliffe Hospital, Oxford, OX3 9DU, UK.,Wolfson Centre for Prevention of Stroke and Dementia, Nuffield Dept Clinical Neurosciences, University of Oxford, UK
| | - Sanjay Manohar
- Department of Experimental Psychology, University of Oxford, Anna Watts Building, Radcliffe Observatory Quarter, Oxford, OX2 6GG, UK
| | - Masud Husain
- Nuffield Department of Clinical Neurosciences, University of Oxford, Level 6, West Wing, John Radcliffe Hospital, Oxford, OX3 9DU, UK.,Department of Experimental Psychology, University of Oxford, Anna Watts Building, Radcliffe Observatory Quarter, Oxford, OX2 6GG, UK.,NIHR Oxford Biomedical Research Centre, Oxford, UK.,Wellcome Centre for Integrative Neuroimaging, University of Oxford, UK
| |
Collapse
|
24
|
Özdenizci O, Eldeeb S, Demir A, Erdoğmuş D, Akçakaya M. EEG-based texture roughness classification in active tactile exploration with invariant representation learning networks. Biomed Signal Process Control 2021; 67. [PMID: 33927780 DOI: 10.1016/j.bspc.2021.102507] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
During daily activities, humans use their hands to grasp surrounding objects and perceive sensory information which are also employed for perceptual and motor goals. Multiple cortical brain regions are known to be responsible for sensory recognition, perception and motor execution during sensorimotor processing. While various research studies particularly focus on the domain of human sensorimotor control, the relation and processing between motor execution and sensory processing is not yet fully understood. Main goal of our work is to discriminate textured surfaces varying in their roughness levels during active tactile exploration using simultaneously recorded electroencephalogram (EEG) data, while minimizing the variance of distinct motor exploration movement patterns. We perform an experimental study with eight healthy participants who were instructed to use the tip of their dominant hand index finger while rubbing or tapping three different textured surfaces with varying levels of roughness. We use an adversarial invariant representation learning neural network architecture that performs EEG-based classification of different textured surfaces, while simultaneously minimizing the discriminability of motor movement conditions (i.e., rub or tap). Results show that the proposed approach can discriminate between three different textured surfaces with accuracies up to 70%, while suppressing movement related variability from learned representations.
Collapse
Affiliation(s)
- Ozan Özdenizci
- Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, USA
- Institute of Theoretical Computer Science, Graz University of Technology, Graz, Austria
| | - Safaa Eldeeb
- Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Andaç Demir
- Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, USA
| | - Deniz Erdoğmuş
- Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, USA
| | - Murat Akçakaya
- Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, USA
| |
Collapse
|
25
|
Bari BA, Cohen JY. Dynamic decision making and value computations in medial frontal cortex. INTERNATIONAL REVIEW OF NEUROBIOLOGY 2021; 158:83-113. [PMID: 33785157 DOI: 10.1016/bs.irn.2020.12.001] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Abstract
Dynamic decision making requires an intact medial frontal cortex. Recent work has combined theory and single-neuron measurements in frontal cortex to advance models of decision making. We review behavioral tasks that have been used to study dynamic decision making and algorithmic models of these tasks using reinforcement learning theory. We discuss studies linking neurophysiology and quantitative decision variables. We conclude with hypotheses about the role of other cortical and subcortical structures in dynamic decision making, including ascending neuromodulatory systems.
Collapse
Affiliation(s)
- Bilal A Bari
- The Solomon H. Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD, United States
| | - Jeremiah Y Cohen
- The Solomon H. Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD, United States.
| |
Collapse
|
26
|
Soltani A, Rakhshan M, Schafer RJ, Burrows BE, Moore T. Separable Influences of Reward on Visual Processing and Choice. J Cogn Neurosci 2020; 33:248-262. [PMID: 33166195 DOI: 10.1162/jocn_a_01647] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Primate vision is characterized by constant, sequential processing and selection of visual targets to fixate. Although expected reward is known to influence both processing and selection of visual targets, similarities and differences between these effects remain unclear mainly because they have been measured in separate tasks. Using a novel paradigm, we simultaneously measured the effects of reward outcomes and expected reward on target selection and sensitivity to visual motion in monkeys. Monkeys freely chose between two visual targets and received a juice reward with varying probability for eye movements made to either of them. Targets were stationary apertures of drifting gratings, causing the end points of eye movements to these targets to be systematically biased in the direction of motion. We used this motion-induced bias as a measure of sensitivity to visual motion on each trial. We then performed different analyses to explore effects of objective and subjective reward values on choice and sensitivity to visual motion to find similarities and differences between reward effects on these two processes. Specifically, we used different reinforcement learning models to fit choice behavior and estimate subjective reward values based on the integration of reward outcomes over multiple trials. Moreover, to compare the effects of subjective reward value on choice and sensitivity to motion directly, we considered correlations between each of these variables and integrated reward outcomes on a wide range of timescales. We found that, in addition to choice, sensitivity to visual motion was also influenced by subjective reward value, although the motion was irrelevant for receiving reward. Unlike choice, however, sensitivity to visual motion was not affected by objective measures of reward value. Moreover, choice was determined by the difference in subjective reward values of the two options, whereas sensitivity to motion was influenced by the sum of values. Finally, models that best predicted visual processing and choice used sets of estimated reward values based on different types of reward integration and timescales. Together, our results demonstrate separable influences of reward on visual processing and choice, and point to the presence of multiple brain circuits for the integration of reward outcomes.
Collapse
|
27
|
Kurdi B, Dunham Y. Propositional Accounts of Implicit Evaluation: Taking Stock and Looking Ahead. SOCIAL COGNITION 2020. [DOI: 10.1521/soco.2020.38.supp.s42] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
Associative accounts suggest that implicit (indirectly measured) evaluations are sensitive primarily to co-occurrence information (e.g., pairings of gorges with positive experiences) and are represented associatively (e.g., Gorge–Nice). By contrast, recent propositional accounts have argued that implicit evaluations are also responsive to relational information (e.g., gorges causing vs. preventing ennui) and are represented propositionally (e.g., “I find gorges fascinating”). In a review of 30 empirical papers involving exposure to contradictory co-occurrence information and relational information, we found overwhelming evidence for the latter dominating the updating of implicit evaluations, supporting the propositional perspective. However, unlike explicit evaluations, implicit evaluations seem recalcitrant in the face of relational information that requires retrospective revaluation of already encoded co-occurrence information. These findings may be jointly explained by a “common currency” hypothesis under which implicit evaluations emerge from compressed summary representations, which are sensitive to relational information but are not fully propositional.
Collapse
|
28
|
Seidenbecher SE, Sanders JI, von Philipsborn AC, Kvitsiani D. Reward foraging task and model-based analysis reveal how fruit flies learn value of available options. PLoS One 2020; 15:e0239616. [PMID: 33007023 PMCID: PMC7531776 DOI: 10.1371/journal.pone.0239616] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2020] [Accepted: 09/10/2020] [Indexed: 11/18/2022] Open
Abstract
Foraging animals have to evaluate, compare and select food patches in order to increase their fitness. Understanding what drives foraging decisions requires careful manipulation of the value of alternative options while monitoring animals choices. Value-based decision-making tasks in combination with formal learning models have provided both an experimental and theoretical framework to study foraging decisions in lab settings. While these approaches were successfully used in the past to understand what drives choices in mammals, very little work has been done on fruit flies. This is despite the fact that fruit flies have served as model organism for many complex behavioural paradigms. To fill this gap we developed a single-animal, trial-based decision making task, where freely walking flies experienced optogenetic sugar-receptor neuron stimulation. We controlled the value of available options by manipulating the probabilities of optogenetic stimulation. We show that flies integrate reward history of chosen options and forget value of unchosen options. We further discover that flies assign higher values to rewards experienced early in the behavioural session, consistent with formal reinforcement learning models. Finally, we also show that the probabilistic rewards affect walking trajectories of flies, suggesting that accumulated value is controlling the navigation vector of flies in a graded fashion. These findings establish the fruit fly as a model organism to explore the genetic and circuit basis of reward foraging decisions.
Collapse
Affiliation(s)
- Sophie E Seidenbecher
- Danish Research Institute of Translational Neuroscience - DANDRITE, Nordic-EMBL Partnership for Molecular Medicine, Aarhus, Denmark.,Department of Molecular Biology and Genetics, Aarhus University, Aarhus, Denmark
| | - Joshua I Sanders
- Danish Research Institute of Translational Neuroscience - DANDRITE, Nordic-EMBL Partnership for Molecular Medicine, Aarhus, Denmark.,Department of Molecular Biology and Genetics, Aarhus University, Aarhus, Denmark
| | - Anne C von Philipsborn
- Danish Research Institute of Translational Neuroscience - DANDRITE, Nordic-EMBL Partnership for Molecular Medicine, Aarhus, Denmark.,Department of Molecular Biology and Genetics, Aarhus University, Aarhus, Denmark
| | - Duda Kvitsiani
- Danish Research Institute of Translational Neuroscience - DANDRITE, Nordic-EMBL Partnership for Molecular Medicine, Aarhus, Denmark.,Department of Molecular Biology and Genetics, Aarhus University, Aarhus, Denmark
| |
Collapse
|
29
|
Gottlieb J, Cohanpour M, Li Y, Singletary N, Zabeh E. Curiosity, information demand and attentional priority. Curr Opin Behav Sci 2020. [DOI: 10.1016/j.cobeha.2020.07.016] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
|
30
|
Biologically plausible mechanisms underlying motor response correction during reward-based decision-making. Neurocomputing 2020. [DOI: 10.1016/j.neucom.2020.06.103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
31
|
Song F, Zhou S, Gao Y, Hu S, Kong F, Zhao J. Different temporal dynamics of object-based attentional allocation for reward and non-reward objects. J Vis 2020; 20:17. [PMID: 32976595 PMCID: PMC7521185 DOI: 10.1167/jov.20.9.17] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Previous studies have confirmed that both non-reward objects (such as rectangles) and reward objects (such as banknotes) can guide the allocation of our attention; however, it is unclear whether the allocation mode of attention for reward objects is the same as for non-reward objects. This study aims to evaluate different modes of object-based attentional selection elicited by two types of objects: reward objects and non-reward objects. In our analysis, we used a two-rectangle paradigm in which two objects were presented visually. In a series of four experiments, we found a constant object-based effect with non-reward objects, such as rectangles and umbrellas, as stimuli in all of the stimulus onset asynchrony (SOA) conditions (Experiments 1 and 4), but the object-based effect disappeared only at longer SOA with reward objects such as monetary and food objects as stimuli (Experiments 2 and 3). Moreover, we found that monetary and food objects induced similar object-based effects. These results suggest that the temporal dynamics of object-based attentional allocation are different with respect to reward and non-reward objects, and different types of reward objects can guide attentional allocation in a similar way.
Collapse
Affiliation(s)
- Fangxing Song
- School of Psychology, Shaanxi Normal University, Xi'an, China.,Shaanxi Provincial Key Laboratory of Behavior and Cognitive Neuroscience, Xi'an, China
| | - Sicen Zhou
- School of Psychology, Shaanxi Normal University, Xi'an, China.,Shaanxi Provincial Key Laboratory of Behavior and Cognitive Neuroscience, Xi'an, China
| | - Yunfei Gao
- School of Psychology, Shaanxi Normal University, Xi'an, China.,Shaanxi Provincial Key Laboratory of Behavior and Cognitive Neuroscience, Xi'an, China
| | - Saisai Hu
- School of Psychology, Shaanxi Normal University, Xi'an, China.,Shaanxi Provincial Key Laboratory of Behavior and Cognitive Neuroscience, Xi'an, China
| | - Feng Kong
- School of Psychology, Shaanxi Normal University, Xi'an, China.,Shaanxi Provincial Key Laboratory of Behavior and Cognitive Neuroscience, Xi'an, China
| | - Jingjing Zhao
- School of Psychology, Shaanxi Normal University, Xi'an, China.,Shaanxi Provincial Key Laboratory of Behavior and Cognitive Neuroscience, Xi'an, China
| |
Collapse
|
32
|
Properties and temporal dynamics of choice- and action-predictive signals during item recognition decisions. Brain Struct Funct 2020; 225:2271-2286. [PMID: 32772167 PMCID: PMC7473849 DOI: 10.1007/s00429-020-02124-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2020] [Accepted: 07/24/2020] [Indexed: 01/03/2023]
Abstract
Decision-making is in the service of action regardless of whether the decision concerns perceptual information, goods or memories. Compared to recent advances in the neurobiology of perceptual or value-based decisions, however, the neural bases supporting the sampling of evidence in long-term memory, and the transformation of memory-based decisions into appropriate actions, are still poorly understood. In the present fMRI study, we used multivariate pattern analysis to investigate the temporal dynamics of choice- and action-predictive signals during an item recognition task that manipulated the association between memory choices (old/new) and motor responses (eye/hand) across subjects. Choice-predictive activity was mainly observed in striatal, lateral prefrontal and lateral parietal regions, was sensitive to the amount of decision evidence and showed a rapid increase after stimulus onset, followed by a fast decay. Action-predictive signals were found in primary sensory motor, premotor and occipito-parietal regions, were generally observed at the end of the decision phase and were not modulated by decision evidence. These findings suggest that a memory decision variable, potentially represented in a fronto-striato-parietal network, is not directly transformed into an action plan as often observed in perceptual decisions. Regions exhibiting choice predictive activity, and especially the striatum, however, also showed a second peak of decision-related activity that, unlike pure choice- or action-predictive signals, depended on the particular choice-response association. This second peak of activity in the striatum might represent the neural signature of the transformation of a memory decision into an appropriate motor response based on the specific choice-response association.
Collapse
|
33
|
Takagaki K, Krug K. The effects of reward and social context on visual processing for perceptual decision-making. CURRENT OPINION IN PHYSIOLOGY 2020. [DOI: 10.1016/j.cophys.2020.08.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
34
|
Monteiro T, Vasconcelos M, Kacelnik A. Choosing fast and simply: Construction of preferences by starlings through parallel option valuation. PLoS Biol 2020; 18:e3000841. [PMID: 32833962 PMCID: PMC7480835 DOI: 10.1371/journal.pbio.3000841] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2020] [Revised: 09/09/2020] [Accepted: 07/31/2020] [Indexed: 11/28/2022] Open
Abstract
The integration of normative and descriptive analyses of decision processes in humans struggles with the fact that measuring preferences by different procedures yields different rankings and that humans appear irrationally impulsive (namely, show maladaptive preference for immediacy). Failure of procedure invariance has led to the widespread hypothesis that preferences are constructed "on the spot" by cognitive evaluations performed at choice time, implying that choices should take extra time in order to perform the necessary comparisons. We examine this issue in experiments with starlings (Sturnus vulgaris) and show that integrating normative and descriptive arguments is possible and may help reinterpreting human decision results. Our main findings are that (1) ranking alternatives through direct rating (response time) accurately predicts preference in choice, overcoming failures of procedure invariance; (2) preference is not constructed at choice time nor does it involve extra time (we show that the opposite is true); and (3) starlings' choices are not irrationally impulsive but are instead directly interpretable in terms of profitability ranking. Like all nonhuman research, our protocols examine decisions by experience rather than by description, and hence support the conjecture that irrationalities that prevail in research with humans may not be observed in decisions by experience protocols.
Collapse
Affiliation(s)
- Tiago Monteiro
- Department of Zoology, University of Oxford, Oxford, United Kingdom
| | - Marco Vasconcelos
- William James Center for Research, University of Aveiro, Aveiro, Portugal
| | - Alex Kacelnik
- Department of Zoology, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
35
|
Digit eyes: Learning-related changes in information access in a computer game parallel those of oculomotor attention in laboratory studies. Atten Percept Psychophys 2020; 82:2434-2447. [PMID: 32333371 DOI: 10.3758/s13414-020-02019-w] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
Active sensing theory is founded upon the dynamic relationship between information sampling and an observer's evolving goals. Oculomotor activity is a well studied method of sampling; a mouse or a keyboard can also be used to access information past the current screen. We examine information access patterns of StarCraft 2 players at multiple skill levels. The first measures are analogous to existing eye-movement studies: fixation frequency, fixation targets, and fixation duration all change as a function of skill, and are commensurate with known properties of eye movements in learning. Actions that require visual attention at moderate skill levels are eventually performed with little visual attention at all. This (a) confirms the generalizability of laboratory studies of attention and learning using eye movements to digital interface use, and (b) suggests that a wide variety of information access behaviors may be considered as a unified set of phenomena.
Collapse
|
36
|
Erdeniz B, Done J. Towards Automaticity in Reinforcement Learning: A Model-Based Functional Magnetic Resonance Imaging Study. ACTA ACUST UNITED AC 2020; 57:98-107. [PMID: 32550774 DOI: 10.29399/npa.24772] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2019] [Accepted: 11/11/2019] [Indexed: 11/07/2022]
Abstract
Introduction Previous studies showed that over the course of learning many neurons in the medial prefrontal cortex adapt their firing rate towards the options with highest predicted value reward but it was showed that during later learning trials the brain switches to a more automatic processing mode governed by the basal ganglia. Based on this evidence, we hypothesized that during the early learning trials the predicted values of chosen options will be coded by a goal directed system in the medial frontal cortex but during the late trials the predicted values will be coded by the habitual learning system in the dorsal striatum. Methods In this study, using a 3 Tesla functional magnetic resonance imaging scanner (fMRI), blood oxygen level dependent signal (BOLD) data was collected whilst participants (N=12) performed a reinforcement learning task. The task consisted of instrumental conditioning trials wherein each trial a participant choose one of the two available options in order to win or avoid losing money. In addition to that, depending on the experimental condition, participants received either monetary reward (gain money), monetary penalty (lose money) or neural outcome. Results Using model-based analysis for functional magnetic resonance imaging (fMRI) event related designs; region of interest (ROI) analysis was performed to nucleus accumbens, medial frontal cortex, caudate nucleus, putamen and globus pallidus internal and external segments. In order to compare the difference in brain activity for early (goal directed) versus late learning (habitual, automatic) trials, separate ROI analyses were performed for each anatomical sub-region. For the reward condition, we found significant activity in the medial frontal cortex (p<0.05) only for early learning trials but activity is shifted to bilateral putamen (p<0.05) during later trials. However, for the loss condition no significant activity was found for early trials except globus pallidus internal segment showed a significant activity (p<0.05) for later trials. Conclusion We found that during reinforcement learning activation in the brain shifted from the medial frontal regions to dorsal regions of the striatum. These findings suggest that there are two separable (early goal directed and late habitual) learning systems in the brain.
Collapse
Affiliation(s)
- Burak Erdeniz
- Department of Psychology, İzmir University of Economics, İzmir, Turkey
| | - John Done
- Department of Psychology and Sports Sciences, School of Life and Medical Sciences, University of Hertfordshire, Hatfield, United Kingdom
| |
Collapse
|
37
|
|
38
|
Wispinski NJ, Gallivan JP, Chapman CS. Models, movements, and minds: bridging the gap between decision making and action. Ann N Y Acad Sci 2020; 1464:30-51. [DOI: 10.1111/nyas.13973] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2018] [Revised: 08/20/2018] [Accepted: 09/06/2018] [Indexed: 11/29/2022]
Affiliation(s)
| | - Jason P. Gallivan
- Centre for Neuroscience StudiesQueen's University Kingston Ontario Canada
- Department of PsychologyQueen's University Kingston Ontario Canada
- Department of Biomedical and Molecular SciencesQueen's University Kingston Ontario Canada
| | - Craig S. Chapman
- Faculty of Kinesiology, Sport, and RecreationUniversity of Alberta Edmonton Alberta Canada
- Neuroscience and Mental Health Institute, University of Alberta Edmonton Alberta Canada
| |
Collapse
|
39
|
Lak A, Okun M, Moss MM, Gurnani H, Farrell K, Wells MJ, Reddy CB, Kepecs A, Harris KD, Carandini M. Dopaminergic and Prefrontal Basis of Learning from Sensory Confidence and Reward Value. Neuron 2020; 105:700-711.e6. [PMID: 31859030 PMCID: PMC7031700 DOI: 10.1016/j.neuron.2019.11.018] [Citation(s) in RCA: 66] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2019] [Revised: 09/04/2019] [Accepted: 11/11/2019] [Indexed: 01/07/2023]
Abstract
Deciding between stimuli requires combining their learned value with one's sensory confidence. We trained mice in a visual task that probes this combination. Mouse choices reflected not only present confidence and past rewards but also past confidence. Their behavior conformed to a model that combines signal detection with reinforcement learning. In the model, the predicted value of the chosen option is the product of sensory confidence and learned value. We found precise correlates of this variable in the pre-outcome activity of midbrain dopamine neurons and of medial prefrontal cortical neurons. However, only the latter played a causal role: inactivating medial prefrontal cortex before outcome strengthened learning from the outcome. Dopamine neurons played a causal role only after outcome, when they encoded reward prediction errors graded by confidence, influencing subsequent choices. These results reveal neural signals that combine reward value with sensory confidence and guide subsequent learning.
Collapse
Affiliation(s)
- Armin Lak
- UCL Institute of Ophthalmology, University College London, London WC1E 6BT, UK.
| | - Michael Okun
- UCL Queen Square Institute of Neurology, University College London, London WC1E 6BT, UK; Centre for Systems Neuroscience, University of Leicester, Leicester LE1 7RH, UK
| | - Morgane M Moss
- UCL Institute of Ophthalmology, University College London, London WC1E 6BT, UK
| | - Harsha Gurnani
- UCL Institute of Ophthalmology, University College London, London WC1E 6BT, UK
| | - Karolina Farrell
- UCL Institute of Ophthalmology, University College London, London WC1E 6BT, UK
| | - Miles J Wells
- UCL Institute of Ophthalmology, University College London, London WC1E 6BT, UK
| | - Charu Bai Reddy
- UCL Institute of Ophthalmology, University College London, London WC1E 6BT, UK
| | - Adam Kepecs
- Cold Spring Harbor Laboratory, 1 Bungtown Road, Cold Spring Harbor, NY 11724, USA
| | - Kenneth D Harris
- UCL Queen Square Institute of Neurology, University College London, London WC1E 6BT, UK
| | - Matteo Carandini
- UCL Institute of Ophthalmology, University College London, London WC1E 6BT, UK
| |
Collapse
|
40
|
Rakhshan M, Lee V, Chu E, Harris L, Laiks L, Khorsand P, Soltani A. Influence of Expected Reward on Temporal Order Judgment. J Cogn Neurosci 2019; 32:674-690. [PMID: 31851591 DOI: 10.1162/jocn_a_01516] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Perceptual decision-making has been shown to be influenced by reward expected from alternative options or actions, but the underlying neural mechanisms are currently unknown. More specifically, it is debated whether reward effects are mediated through changes in sensory processing, later stages of decision-making, or both. To address this question, we conducted two experiments in which human participants made saccades to what they perceived to be either the first or second of two visually identical but asynchronously presented targets while we manipulated expected reward from correct and incorrect responses on each trial. By comparing reward-induced bias in target selection (i.e., reward bias) during the two experiments, we determined whether reward caused changes in sensory or decision-making processes. We found similar reward biases in the two experiments indicating that reward information mainly influenced later stages of decision-making. Moreover, the observed reward biases were independent of the individual's sensitivity to sensory signals. This suggests that reward effects were determined heuristically via modulation of decision-making processes instead of sensory processing. To further explain our findings and uncover plausible neural mechanisms, we simulated our experiments with a cortical network model and tested alternative mechanisms for how reward could exert its influence. We found that our experimental observations are more compatible with reward-dependent input to the output layer of the decision circuit. Together, our results suggest that, during a temporal judgment task, reward exerts its influence via changing later stages of decision-making (i.e., response bias) rather than early sensory processing (i.e., perceptual bias).
Collapse
|
41
|
Akiba M, Sugimoto K, Aoki R, Murakami R, Miyashita T, Hashimoto R, Hiranuma A, Yamauchi J, Ueno T, Morimoto T. Dopamine modulates the optomotor response to unreliable visual stimuli in Drosophila melanogaster. Eur J Neurosci 2019; 51:822-839. [PMID: 31834948 DOI: 10.1111/ejn.14648] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2019] [Revised: 11/18/2019] [Accepted: 12/05/2019] [Indexed: 01/21/2023]
Abstract
State-dependent modulation of sensory systems has been studied in many organisms and is possibly mediated through neuromodulators such as monoamine neurotransmitters. Among these, dopamine is involved in many aspects of animal behaviour, including movement control, attention, motivation and cognition. However, the precise neural mechanism underlying dopaminergic modulation of behaviour induced by sensory stimuli remains poorly understood. Here, we used Drosophila melanogaster to show that dopamine can modulate the optomotor response to moving visual stimuli including noise. The optomotor response is the head-turning response to moving objects, which is observed in most sight-reliant animals including mammals and insects. First, the effects of the dopamine system on the optomotor response were investigated in mutant flies deficient in dopamine receptors D1R1 or D1R2, which are involved in the modulation of sleep-arousal in flies. We examined the optomotor response in D1R1 knockout (D1R1 KO) and D1R2 knockout (D1R2 KO) flies and found that it was not affected in D1R1 KO flies; however, it was significantly reduced in D1R2 KO flies compared with the wild type. Using cell-type-specific expression of an RNA interference construct of D1R2, we identified the fan-shaped body, a part of the central complex, responsible for dopamine-mediated modulation of the optomotor response. In particular, pontine cells in the fan-shaped body seemed important in the modulation of the optomotor response, and their neural activity was required for the optomotor response. These results suggest a novel role of the central complex in the modulation of a behaviour based on the processing of sensory stimulations.
Collapse
Affiliation(s)
- Masumi Akiba
- Laboratory of Molecular Neuroscience and Neurology, School of Life Sciences, University of Pharmacy and Life Sciences, Tokyo, Japan
| | - Kentaro Sugimoto
- Laboratory of Molecular Neuroscience and Neurology, School of Life Sciences, University of Pharmacy and Life Sciences, Tokyo, Japan.,Department of Computer Science, School of Computing, Tokyo Institute of Technology, Tokyo, Japan
| | - Risa Aoki
- Laboratory of Molecular Neuroscience and Neurology, School of Life Sciences, University of Pharmacy and Life Sciences, Tokyo, Japan
| | - Ryo Murakami
- Laboratory of Molecular Neuroscience and Neurology, School of Life Sciences, University of Pharmacy and Life Sciences, Tokyo, Japan
| | | | - Riho Hashimoto
- Laboratory of Molecular Neuroscience and Neurology, School of Life Sciences, University of Pharmacy and Life Sciences, Tokyo, Japan
| | - Anna Hiranuma
- Laboratory of Molecular Neuroscience and Neurology, School of Life Sciences, University of Pharmacy and Life Sciences, Tokyo, Japan
| | - Junji Yamauchi
- Laboratory of Molecular Neuroscience and Neurology, School of Life Sciences, University of Pharmacy and Life Sciences, Tokyo, Japan
| | - Taro Ueno
- Department of Biomolecular Science, Graduate School of Science, Toho University, Chiba, Japan
| | - Takako Morimoto
- Laboratory of Molecular Neuroscience and Neurology, School of Life Sciences, University of Pharmacy and Life Sciences, Tokyo, Japan
| |
Collapse
|
42
|
Parr AC, Coe BC, Munoz DP, Dorris MC. A novel fMRI paradigm to dissociate the behavioral and neural components of mixed-strategy decision making from non-strategic decisions in humans. Eur J Neurosci 2019; 51:1914-1927. [PMID: 31596980 DOI: 10.1111/ejn.14586] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2019] [Revised: 08/22/2019] [Accepted: 09/18/2019] [Indexed: 11/30/2022]
Abstract
During competitive interactions, such as predator-prey or team sports, the outcome of one's actions is dependent on both their own choices and those of their opponents. Success in these rivalries requires that individuals choose dynamically and unpredictably, often adopting a mixed strategy. Understanding the neural basis of strategic decision making is complicated by the fact that it recruits various cognitive processes that are often shared with non-strategic forms of decision making, such as value estimation, working memory, response inhibition, response selection, and reward processes. Although researchers have explored neural activity within key brain regions during mixed-strategy games, how brain activity differs in the context of strategic interactions versus non-strategic choices is not well understood. We developed a novel behavioral paradigm to dissociate choice behavior during mixed-strategy interactions from non-strategic choices, and we used task-based functional magnetic resonance imaging (fMRI) to contrast brain activation. In a block design, participants competed in the classic mixed-strategy game, "matching pennies," against a dynamic computer opponent designed to exploit predictability in players' response patterns. Results were contrasted with a non-strategic task that had comparable sensory input, motor output, and reward rate; thus, differences in behavior and brain activation reflect strategic processes. The mixed-strategy game was associated with activation of a distributed cortico-striatal network compared to the non-strategic task. We propose that choosing in mixed-strategy contexts requires additional cognitive demands present to a lesser degree during the control task, illustrating the strength of this design in probing function of cognitive systems beyond core sensory, motor, and reward processes.
Collapse
Affiliation(s)
- Ashley C Parr
- Centre for Neuroscience Studies, Queen's University, Kingston, ON, Canada.,Department of Psychiatry, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Brian C Coe
- Centre for Neuroscience Studies, Queen's University, Kingston, ON, Canada
| | - Douglas P Munoz
- Centre for Neuroscience Studies, Queen's University, Kingston, ON, Canada.,Department of Biomedical and Molecular Sciences, Queen's University, Kingston, ON, Canada
| | - Michael C Dorris
- Institute of Neuroscience, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, Shanghai, China
| |
Collapse
|
43
|
Waskom ML, Okazawa G, Kiani R. Designing and Interpreting Psychophysical Investigations of Cognition. Neuron 2019; 104:100-112. [PMID: 31600507 PMCID: PMC6855836 DOI: 10.1016/j.neuron.2019.09.016] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2019] [Revised: 09/03/2019] [Accepted: 09/12/2019] [Indexed: 11/24/2022]
Abstract
Scientific experimentation depends on the artificial control of natural phenomena. The inaccessibility of cognitive processes to direct manipulation can make such control difficult to realize. Here, we discuss approaches for overcoming this challenge. We advocate the incorporation of experimental techniques from sensory psychophysics into the study of cognitive processes such as decision making and executive control. These techniques include the use of simple parameterized stimuli to precisely manipulate available information and computational models to jointly quantify behavior and neural responses. We illustrate the potential for such techniques to drive theoretical development, and we examine important practical details of how to conduct controlled experiments when using them. Finally, we highlight principles guiding the use of computational models in studying the neural basis of cognition.
Collapse
Affiliation(s)
- Michael L Waskom
- Center for Neural Science, New York University, 4 Washington Place, New York, NY 10003, USA
| | - Gouki Okazawa
- Center for Neural Science, New York University, 4 Washington Place, New York, NY 10003, USA
| | - Roozbeh Kiani
- Center for Neural Science, New York University, 4 Washington Place, New York, NY 10003, USA; Neuroscience Institute, NYU Langone Medical Center, 550 First Avenue, New York, NY 10016, USA; Department of Psychology, New York University, 4 Washington Place, New York, NY 10003, USA.
| |
Collapse
|
44
|
Gallivan JP, Chapman CS, Wolpert DM, Flanagan JR. Decision-making in sensorimotor control. Nat Rev Neurosci 2019; 19:519-534. [PMID: 30089888 DOI: 10.1038/s41583-018-0045-9] [Citation(s) in RCA: 137] [Impact Index Per Article: 27.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Skilled sensorimotor interactions with the world result from a series of decision-making processes that determine, on the basis of information extracted during the unfolding sequence of events, which movements to make and when and how to make them. Despite this inherent link between decision-making and sensorimotor control, research into each of these two areas has largely evolved in isolation, and it is only fairly recently that researchers have begun investigating how they interact and, together, influence behaviour. Here, we review recent behavioural, neurophysiological and computational research that highlights the role of decision-making processes in the selection, planning and control of goal-directed movements in humans and nonhuman primates.
Collapse
Affiliation(s)
- Jason P Gallivan
- Centre for Neuroscience Studies and Department of Psychology, Queen's University, Kingston, Ontario, Canada. .,Department of Biomedical and Molecular Sciences, Queen's University, Kingston, Ontario, Canada.
| | - Craig S Chapman
- Faculty of Kinesiology, Sport, and Recreation and Neuroscience and Mental Health Institute, University of Alberta, Edmonton, Alberta, Canada
| | - Daniel M Wolpert
- Department of Engineering, University of Cambridge, Cambridge, UK.,Zuckerman Mind Brain Behavior Institute, Department of Neuroscience, Columbia University, New York, NY, USA
| | - J Randall Flanagan
- Centre for Neuroscience Studies and Department of Psychology, Queen's University, Kingston, Ontario, Canada.
| |
Collapse
|
45
|
Martinez-Saito M, Konovalov R, Piradov MA, Shestakova A, Gutkin B, Klucharev V. Action in auctions: neural and computational mechanisms of bidding behaviour. Eur J Neurosci 2019; 50:3327-3348. [PMID: 31219633 PMCID: PMC6899836 DOI: 10.1111/ejn.14492] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2018] [Revised: 05/01/2019] [Accepted: 05/24/2019] [Indexed: 11/27/2022]
Abstract
Competition for resources is a fundamental characteristic of evolution. Auctions have been widely used to model competition of individuals for resources, and bidding behaviour plays a major role in social competition. Yet, how humans learn to bid efficiently remains an open question. We used model‐based neuroimaging to investigate the neural mechanisms of bidding behaviour under different types of competition. Twenty‐seven subjects (nine male) played a prototypical bidding game: a double action, with three “market” types, which differed in the number of competitors. We compared different computational learning models of bidding: directional learning models (DL), where the model bid is “nudged” depending on whether it was accepted or rejected, along with standard reinforcement learning models (RL). We found that DL fit the behaviour best and resulted in higher payoffs. We found the binary learning signal associated with DL to be represented by neural activity in the striatum distinctly posterior to a weaker reward prediction error signal. We posited that DL is an efficient heuristic for valuation when the action (bid) space is continuous. Indeed, we found that the posterior parietal cortex represents the continuous action space of the task, and the frontopolar prefrontal cortex distinguishes among conditions of social competition. Based on our findings, we proposed a conceptual model that accounts for a sequence of processes that are required to perform successful and flexible bidding under different types of competition.
Collapse
Affiliation(s)
- Mario Martinez-Saito
- Centre for Cognition and Decision Making, Institute of Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, Russian Federation
| | | | | | - Anna Shestakova
- Centre for Cognition and Decision Making, Institute of Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, Russian Federation
| | - Boris Gutkin
- Centre for Cognition and Decision Making, Institute of Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, Russian Federation.,Group for Neural Theory, LNC INSERM U960, PSL* Research University Ecole Normale Superieure, Paris, France
| | - Vasily Klucharev
- Centre for Cognition and Decision Making, Institute of Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, Russian Federation
| |
Collapse
|
46
|
Sugrue LP, Desikan RS. Precision neuroradiology: mapping the nodes and networks that link genes to behaviour. Br J Radiol 2019; 92:20190093. [PMID: 31294609 PMCID: PMC6732927 DOI: 10.1259/bjr.20190093] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
Abstract
What is the future of neuroradiology in the era of precision medicine? As with any big change, this transformation in medicine presents both challenges and opportunities, and to flourish in this new environment we will have to adapt. It is difficult to predict exactly how neuroradiology will evolve in this shifting landscape, but there will be changes in both what we image and what we do. In terms of imaging, we will need to move beyond simply imaging brain anatomy and toward imaging function, both at the molecular and circuit level. In terms of what we do, we will need to move from the periphery of the clinical enterprise toward its center, with a new emphasis on integrating imaging with genetic and clinical data to form a comprehensive picture of the patient that can be used to direct further testing and care.The payoff is that these changes will align neuroradiology with the emerging field of precision psychiatry, which promises to replace symptom-based diagnosis and trial-and-error treatment of psychiatric disorders with diagnoses based on quantifiable genetic, imaging, physiologic, and behavioural criteria and therapies targeted to the particular pathophysiology of individual patients. Here we review some of the recent developments in behavioural genetics and neuroscience that are laying the foundation for precision psychiatry. By no means comprehensive, our goal is to introduce some of the perspectives and techniques that are likely to be relevant to the precision neuroradiologist of the future.
Collapse
Affiliation(s)
- Leo P Sugrue
- 1Departments of Radiology and Biomedical Imaging, University California, San Francisco, USA
| | - Rahul S Desikan
- 1Departments of Radiology and Biomedical Imaging, University California, San Francisco, USA.,2Department of Neurology, University California, San Francisco, USA
| |
Collapse
|
47
|
Horan M, Daddaoua N, Gottlieb J. Parietal neurons encode information sampling based on decision uncertainty. Nat Neurosci 2019; 22:1327-1335. [PMID: 31285613 PMCID: PMC6660422 DOI: 10.1038/s41593-019-0440-1] [Citation(s) in RCA: 39] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2018] [Accepted: 05/28/2019] [Indexed: 01/19/2023]
Abstract
In natural behavior animals actively gather information that is relevant for learning or actions, but the mechanisms of active sampling are rarely investigated. We tested parietal neurons involved in oculomotor control in a task in which monkeys made saccades to gather visual information before reporting a decision based on the information. We show that the neurons encode, before the saccade, the information gains (reduction in decision uncertainty) that the saccade was expected to bring, correlating with the monkeys’ efficiency in processing the information in the post-saccadic fixation. Informational sensitivity is independent of the neurons’ reward sensitivity, which is unreliable across task contexts, inconsistent with the view that the cells encode economic utility. Instead, we suggest that parietal cells are involved in implementing active sampling policies, showing uncertainty-dependent boosts of neural gain that facilitate the selection of relevant cues and the efficient use of the information delivered by these cues.
Collapse
Affiliation(s)
- Mattias Horan
- Department of Neuroscience, Columbia University, New York, NY, USA
| | - Nabil Daddaoua
- Department of Neuroscience, Columbia University, New York, NY, USA.,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Jacqueline Gottlieb
- Department of Neuroscience, Columbia University, New York, NY, USA. .,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA. .,The Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| |
Collapse
|
48
|
Banerjee S, Grover S, Ganesh S, Sridharan D. Sensory and decisional components of endogenous attention are dissociable. J Neurophysiol 2019; 122:1538-1554. [PMID: 31268805 PMCID: PMC6843089 DOI: 10.1152/jn.00257.2019] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Endogenous cueing of attention enhances sensory processing of the attended stimulus (perceptual sensitivity) and prioritizes information from the attended location for guiding behavioral decisions (spatial choice bias). Here, we test whether sensitivity and bias effects of endogenous spatial attention are under the control of common or distinct mechanisms. Human observers performed a multialternative visuospatial attention task with probabilistic spatial cues. Observers' behavioral choices were analyzed with a recently developed multidimensional signal detection model (the m-ADC model). The model effectively decoupled the effects of spatial cueing on sensitivity from those on spatial bias and revealed striking dissociations between them. Sensitivity was highest at the cued location and not significantly different among uncued locations, suggesting a spotlight-like allocation of sensory resources at the cued location. On the other hand, bias varied systematically with cue validity, suggesting a graded allocation of decisional priority across locations. Cueing-induced modulations of sensitivity and bias were uncorrelated within and across subjects. Bias, but not sensitivity, correlated with key metrics of prioritized decision-making, including reaction times and decision optimality indices. In addition, we developed a novel metric, differential risk curvature, for distinguishing bias effects of attention from those of signal expectation. Differential risk curvature correlated selectively with m-ADC model estimates of bias but not with estimates of sensitivity. Our results reveal dissociable effects of endogenous attention on perceptual sensitivity and choice bias in a multialternative choice task and motivate the search for the distinct neural correlates of each.NEW & NOTEWORTHY Attention is often studied as a unitary phenomenon. Yet, attention can both enhance the perception of important stimuli (sensitivity) and prioritize such stimuli for decision-making (bias). Employing a multialternative spatial attention task with probabilistic cueing, we show that attention affects sensitivity and bias through dissociable mechanisms. Specifically, the effects on sensitivity alone match the notion of an attentional "spotlight." Our behavioral model enables quantifying component processes of attention, and identifying their respective neural correlates.
Collapse
Affiliation(s)
- Sanjna Banerjee
- Centre for Neuroscience, Indian Institute of Science, Bangalore, India
| | - Shrey Grover
- Centre for Neuroscience, Indian Institute of Science, Bangalore, India
| | - Suhas Ganesh
- Centre for Neuroscience, Indian Institute of Science, Bangalore, India
| | | |
Collapse
|
49
|
Ott T, Masset P, Kepecs A. The Neurobiology of Confidence: From Beliefs to Neurons. COLD SPRING HARBOR SYMPOSIA ON QUANTITATIVE BIOLOGY 2019; 83:9-16. [PMID: 31270145 DOI: 10.1101/sqb.2018.83.038794] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
How confident are you? As humans, aware of our subjective sense of confidence, we can readily answer. Knowing your level of confidence helps to optimize both routine decisions such as whether to go back and check if the front door was locked and momentous ones like finding a partner for life. Yet the inherently subjective nature of confidence has limited investigations by neurobiologists. Here, we provide an overview of recent advances in this field and lay out a conceptual framework that lets us translate psychological questions about subjective confidence into the language of neuroscience. We show how statistical notions of confidence provide a bridge between our subjective sense of confidence and confidence-guided behaviors in nonhuman animals, thus enabling the study of the underlying neurobiology. We discuss confidence as a core cognitive process that enables organisms to optimize behavior such as learning or resource allocation and that serves as the basis of metacognitive reasoning. These approaches place confidence on a solid footing and pave the way for a mechanistic understanding of how the brain implements confidence-based algorithms to guide behavior.
Collapse
Affiliation(s)
- Torben Ott
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York 11724, USA
| | - Paul Masset
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York 11724, USA.,Watson School of Biological Sciences, Cold Spring Harbor, New York 11724, USA.,Department of Molecular and Cellular Biology & Center for Brain Science, Harvard University, Cambridge, Massachusetts 02138, USA
| | - Adam Kepecs
- Cold Spring Harbor Laboratory, Cold Spring Harbor, New York 11724, USA
| |
Collapse
|
50
|
Parallel Emergence of a Compartmentalized Striatum with the Phylogenetic Development of the Cerebral Cortex. Brain Sci 2019; 9:brainsci9040090. [PMID: 31010240 PMCID: PMC6523536 DOI: 10.3390/brainsci9040090] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2019] [Revised: 04/09/2019] [Accepted: 04/17/2019] [Indexed: 01/05/2023] Open
Abstract
The intricate neuronal architecture of the striatum plays a pivotal role in the functioning of the basal ganglia circuits involved in the control of various aspects of motor, cognitive, and emotional functions. Unlike the cerebral cortex, which has a laminar structure, the striatum is primarily composed of two functional subdivisions (i.e., the striosome and matrix compartments) arranged in a mosaic fashion. This review addresses whether striatal compartmentalization is present in non-mammalian vertebrates, in which simple cognitive and behavioral functions are executed by primitive sensori-motor systems. Studies show that neuronal subpopulations that share neurochemical and connective properties with striosomal and matrix neurons are present in the striata of not only anamniotes (fishes and amphibians), but also amniotes (reptiles and birds). However, these neurons do not form clearly segregated compartments in these vertebrates, suggesting that such compartmentalization is unique to mammals. In the ontogeny of the mammalian forebrain, the later-born matrix neurons disperse the early-born striosome neurons into clusters to form the compartments in tandem with the development of striatal afferents from the cortex. We propose that striatal compartmentalization in mammals emerged in parallel with the evolution of the cortex and possibly enhanced complex processing of sensory information and behavioral flexibility phylogenetically.
Collapse
|