1
|
Howard JD, Edmonds D, Schoenbaum G, Kahnt T. Distributed midbrain responses signal the content of positive identity prediction errors. Curr Biol 2024; 34:4240-4247.e4. [PMID: 39197457 PMCID: PMC11421979 DOI: 10.1016/j.cub.2024.07.105] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2024] [Revised: 06/12/2024] [Accepted: 07/31/2024] [Indexed: 09/01/2024]
Abstract
Recent work across species has shown that midbrain dopamine neurons signal not only errors in the prediction of reward value but also in the prediction of value-neutral sensory features. To support learning of associative structures in downstream areas, identity prediction errors (iPEs) should signal specific information about the mis-predicted outcome. Here, we used pattern-based analysis of functional magnetic resonance imaging (fMRI) data acquired during reversal learning to characterize the information content of iPE responses in the human midbrain. We find that fMRI responses to value-neutral identity errors contain information about the identity of the unexpectedly received reward (positive iPE+) but not about the identity of the omitted reward (negative iPE-). Exploratory analyses revealed representations of iPE- in the dorsomedial prefrontal cortex. These results demonstrate that ensemble midbrain responses to value-neutral identity errors convey information about the identity of unexpectedly received outcomes, which could shape the formation of novel stimulus-outcome associations that constitute cognitive maps.
Collapse
Affiliation(s)
- James D Howard
- Department of Psychology, Brandeis University, Waltham, MA 02453, USA.
| | - Donnisa Edmonds
- Department of Neurology, Northwestern University, Chicago, IL 60611, USA
| | - Geoffrey Schoenbaum
- Intramural Research Program, National Institute on Drug Abuse, Baltimore, MD 21224, USA
| | - Thorsten Kahnt
- Intramural Research Program, National Institute on Drug Abuse, Baltimore, MD 21224, USA.
| |
Collapse
|
2
|
Zimmerman CA, Bolkan SS, Pan-Vazquez A, Wu B, Keppler EF, Meares-Garcia JB, Guthman EM, Fetcho RN, McMannon B, Lee J, Hoag AT, Lynch LA, Janarthanan SR, López Luna JF, Bondy AG, Falkner AL, Wang SSH, Witten IB. A neural mechanism for learning from delayed postingestive feedback. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.10.06.561214. [PMID: 37873112 PMCID: PMC10592633 DOI: 10.1101/2023.10.06.561214] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/25/2023]
Abstract
Animals learn the value of foods based on their postingestive effects and thereby develop aversions to foods that are toxic1-6 and preferences to those that are nutritious7-14. However, it remains unclear how the brain is able to assign credit to flavors experienced during a meal with postingestive feedback signals that can arise after a substantial delay. Here, we reveal an unexpected role for postingestive reactivation of neural flavor representations in this temporal credit assignment process. To begin, we leverage the fact that mice learn to associate novel15-18, but not familiar, flavors with delayed gastric malaise signals to investigate how the brain represents flavors that support aversive postingestive learning. Surveying cellular resolution brainwide activation patterns reveals that a network of amygdala regions is unique in being preferentially activated by novel flavors across every stage of the learning process: the initial meal, delayed malaise, and memory retrieval. By combining high-density recordings in the amygdala with optogenetic stimulation of genetically defined hindbrain malaise cells, we find that postingestive malaise signals potently and specifically reactivate amygdalar novel flavor representations from a recent meal. The degree of malaise-driven reactivation of individual neurons predicts strengthening of flavor responses upon memory retrieval, leading to stabilization of the population-level representation of the recently consumed flavor. In contrast, meals without postingestive consequences degrade neural flavor representations as flavors become familiar and safe. Thus, our findings demonstrate that interoceptive reactivation of amygdalar flavor representations provides a neural mechanism to resolve the temporal credit assignment problem inherent to postingestive learning.
Collapse
Affiliation(s)
| | - Scott S Bolkan
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | | | - Bichan Wu
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Emma F Keppler
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | | | - Eartha Mae Guthman
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Robert N Fetcho
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Brenna McMannon
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Junuk Lee
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Austin T Hoag
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Laura A Lynch
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | | | - Juan F López Luna
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Adrian G Bondy
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Annegret L Falkner
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Samuel S-H Wang
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Ilana B Witten
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
- Howard Hughes Medical Institute, Princeton University, Princeton, NJ, USA
| |
Collapse
|
3
|
Witkowski PP, Rondot L, Kurth-Nelson Z, Garvert MM, Dolan RJ, Behrens TEJ, Boorman ED. Neural mechanisms of credit assignment for delayed outcomes during contingent learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.08.06.606895. [PMID: 39149338 PMCID: PMC11326259 DOI: 10.1101/2024.08.06.606895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/17/2024]
Abstract
Adaptive behavior in complex environments critically relies on the ability to appropriately link specific choices or actions to their outcomes. However, the neural mechanisms that support the ability to credit only those past choices believed to have caused the observed outcomes remain unclear. Here, we leverage multivariate pattern analyses of functional magnetic resonance imaging (fMRI) data and an adaptive learning task to shed light on the underlying neural mechanisms of such specific credit assignment. We find that the lateral orbitofrontal cortex (lOFC) and hippocampus (HC) code for the causal choice identity when credit needs to be assigned for choices that are separated from outcomes by a long delay, even when this delayed transition is punctuated by interim decisions. Further, we show when interim decisions must be made, learning is additionally supported by lateral frontopolar cortex (FPl). Our results indicate that FPl holds previous causal choices in a "pending" state until a relevant outcome is observed, and the fidelity of these representations predicts the fidelity of subsequent causal choice representations in lOFC and HC during credit assignment. Together, these results highlight the importance of the timely reinstatement of specific causes in lOFC and HC in learning choice-outcome relationships when delays and choices intervene, a critical component of real-world learning and decision making.
Collapse
Affiliation(s)
- Phillip P. Witkowski
- Center for Mind and Brain, University of California Davis, Davis, CA, 95618
- Department of Psychology, University of California Davis, Davis, CA, 95618
- National Institute on Drug Abuse Intramural Research Program, National Institutes of Health, Baltimore, MD, USA
| | - Lindsay Rondot
- Center for Mind and Brain, University of California Davis, Davis, CA, 95618
- Department of Psychology, University of California Davis, Davis, CA, 95618
| | - Zeb Kurth-Nelson
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Google DeepMind, London, UK
| | - Mona M. Garvert
- Faculty of Human Sciences, Julius-Maximilians-Universität Würzburg, Würzburg, Germany
| | - Raymond J Dolan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
| | - Timothy E. J. Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford, UK
- Sainsbury Wellcome Centre for Neural Circuits and Behaviour, University College London, London, UK
| | - Erie D. Boorman
- Center for Mind and Brain, University of California Davis, Davis, CA, 95618
- Department of Psychology, University of California Davis, Davis, CA, 95618
| |
Collapse
|
4
|
Qiu L, Qiu Y, Liao J, Li J, Zhang X, Chen K, Huang Q, Huang R. Functional specialization of medial and lateral orbitofrontal cortex in inferential decision-making. iScience 2024; 27:110007. [PMID: 38868183 PMCID: PMC11167445 DOI: 10.1016/j.isci.2024.110007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2023] [Revised: 02/03/2024] [Accepted: 05/14/2024] [Indexed: 06/14/2024] Open
Abstract
Inferring prospective outcomes and updating behavior are prerequisites for making flexible decisions in the changing world. These abilities are highly associated with the functions of the orbitofrontal cortex (OFC) in humans and animals. The functional specialization of OFC subregions in decision-making has been established in animals. However, the understanding of how human OFC contributes to decision-making remains limited. Therefore, we studied this issue by examining the information representation and functional interactions of human OFC subregions during inference-based decision-making. We found that the medial OFC (mOFC) and lateral OFC (lOFC) collectively represented the inferred outcomes which, however, were context-general coding in the mOFC and context-specific in the lOFC. Furthermore, the mOFC-motor and lOFC-frontoparietal functional connectivity may indicate the motor execution of mOFC and the cognitive control of lOFC during behavioral updating. In conclusion, our findings support the dissociable functional roles of OFC subregions in decision-making.
Collapse
Affiliation(s)
- Lixin Qiu
- School of Psychology; Center for Studies of Psychological Application; Guangdong Key Laboratory of Mental Health and Cognitive Science, Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; South China Normal University, Guangzhou 510631, China
| | - Yidan Qiu
- School of Psychology; Center for Studies of Psychological Application; Guangdong Key Laboratory of Mental Health and Cognitive Science, Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; South China Normal University, Guangzhou 510631, China
| | - Jiajun Liao
- School of Psychology; Center for Studies of Psychological Application; Guangdong Key Laboratory of Mental Health and Cognitive Science, Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; South China Normal University, Guangzhou 510631, China
| | - Jinhui Li
- School of Psychology; Center for Studies of Psychological Application; Guangdong Key Laboratory of Mental Health and Cognitive Science, Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; South China Normal University, Guangzhou 510631, China
| | - Xiaoying Zhang
- School of Psychology; Center for Studies of Psychological Application; Guangdong Key Laboratory of Mental Health and Cognitive Science, Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; South China Normal University, Guangzhou 510631, China
| | - Kemeng Chen
- School of Psychology; Center for Studies of Psychological Application; Guangdong Key Laboratory of Mental Health and Cognitive Science, Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; South China Normal University, Guangzhou 510631, China
| | - Qinda Huang
- School of Psychology; Center for Studies of Psychological Application; Guangdong Key Laboratory of Mental Health and Cognitive Science, Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; South China Normal University, Guangzhou 510631, China
| | - Ruiwang Huang
- School of Psychology; Center for Studies of Psychological Application; Guangdong Key Laboratory of Mental Health and Cognitive Science, Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education; South China Normal University, Guangzhou 510631, China
| |
Collapse
|
5
|
Gabriel DB, Havugimana F, Liley AE, Aguilar I, Yeasin M, Simon NW. Lateral Orbitofrontal Cortex Encodes Presence of Risk and Subjective Risk Preference During Decision-Making. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.08.588332. [PMID: 38645204 PMCID: PMC11030364 DOI: 10.1101/2024.04.08.588332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/23/2024]
Abstract
Adaptive decision-making requires consideration of objective risks and rewards associated with each option, as well as subjective preference for risky/safe alternatives. Inaccurate risk/reward estimations can engender excessive risk-taking, a central trait in many psychiatric disorders. The lateral orbitofrontal cortex (lOFC) has been linked to many disorders associated with excessively risky behavior and is ideally situated to mediate risky decision-making. Here, we used single-unit electrophysiology to measure neuronal activity from lOFC of freely moving rats performing in a punishment-based risky decision-making task. Subjects chose between a small, safe reward and a large reward associated with either 0% or 50% risk of concurrent punishment. lOFC activity repeatedly encoded current risk in the environment throughout the decision-making sequence, signaling risk before, during, and after a choice. In addition, lOFC encoded reward magnitude, although this information was only evident during action selection. A Random Forest classifier successfully used neural data accurately to predict the risk of punishment in any given trial, and the ability to predict choice via lOFC activity differentiated between and risk-preferring and risk-averse rats. Finally, risk preferring subjects demonstrated reduced lOFC encoding of risk and increased encoding of reward magnitude. These findings suggest lOFC may serve as a central decision-making hub in which external, environmental information converges with internal, subjective information to guide decision-making in the face of punishment risk.
Collapse
Affiliation(s)
- Daniel B.K. Gabriel
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis, IN 46202
| | - Felix Havugimana
- Department of Computer Engineering, University of Memphis, Memphis, TN, 38152
| | - Anna E. Liley
- Institut du Cerveau/Paris Brain Institute, Paris, France, 75013
| | - Ivan Aguilar
- Department of Psychology, University of Memphis, Memphis, TN, 38152
| | - Mohammed Yeasin
- Department of Computer Engineering, University of Memphis, Memphis, TN, 38152
| | | |
Collapse
|
6
|
Liu Q, Zhao Y, Attanti S, Voss JL, Schoenbaum G, Kahnt T. Midbrain signaling of identity prediction errors depends on orbitofrontal cortex networks. Nat Commun 2024; 15:1704. [PMID: 38402210 PMCID: PMC10894191 DOI: 10.1038/s41467-024-45880-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 02/06/2024] [Indexed: 02/26/2024] Open
Abstract
Outcome-guided behavior requires knowledge about the identity of future rewards. Previous work across species has shown that the dopaminergic midbrain responds to violations in expected reward identity and that the lateral orbitofrontal cortex (OFC) represents reward identity expectations. Here we used network-targeted transcranial magnetic stimulation (TMS) and functional magnetic resonance imaging (fMRI) during a trans-reinforcer reversal learning task to test the hypothesis that outcome expectations in the lateral OFC contribute to the computation of identity prediction errors (iPE) in the midbrain. Network-targeted TMS aiming at lateral OFC reduced the global connectedness of the lateral OFC and impaired reward identity learning in the first block of trials. Critically, TMS disrupted neural representations of expected reward identity in the OFC and modulated iPE responses in the midbrain. These results support the idea that iPE signals in the dopaminergic midbrain are computed based on outcome expectations represented in the lateral OFC.
Collapse
Affiliation(s)
- Qingfang Liu
- National Institute on Drug Abuse Intramural Research Program, Baltimore, MD, 21224, USA
| | - Yao Zhao
- National Institute on Drug Abuse Intramural Research Program, Baltimore, MD, 21224, USA
| | - Sumedha Attanti
- Mayo Clinic Alix School of Medicine, Scottsdale, AZ, 85259, USA
| | - Joel L Voss
- Department of Neurology, The University of Chicago, Chicago, IL, 60611, USA
| | - Geoffrey Schoenbaum
- National Institute on Drug Abuse Intramural Research Program, Baltimore, MD, 21224, USA
| | - Thorsten Kahnt
- National Institute on Drug Abuse Intramural Research Program, Baltimore, MD, 21224, USA.
| |
Collapse
|
7
|
Witkowski PP, Geng JJ. Prefrontal Cortex Codes Representations of Target Identity and Feature Uncertainty. J Neurosci 2023; 43:8769-8776. [PMID: 37875376 PMCID: PMC10727173 DOI: 10.1523/jneurosci.1117-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Revised: 09/04/2023] [Accepted: 10/07/2023] [Indexed: 10/26/2023] Open
Abstract
Many objects in the real world have features that vary over time, creating uncertainty in how they will look in the future. This uncertainty makes statistical knowledge about the likelihood of features critical to attention demanding processes such as visual search. However, little is known about how the uncertainty of visual features is integrated into predictions about search targets in the brain. In the current study, we test the idea that regions prefrontal cortex code statistical knowledge about search targets before the onset of search. Across 20 human participants (13 female; 7 male), we observe target identity in the multivariate pattern and uncertainty in the overall activation of dorsolateral prefrontal cortex (DLPFC) and inferior frontal junction (IFJ) in advance of the search display. This indicates that the target identity (mean) and uncertainty (variance) of the target distribution are coded independently within the same regions. Furthermore, once the search display appears the univariate IFJ signal scaled with the distance of the actual target from the expected mean, but more so when expected variability was low. These results inform neural theories of attention by showing how the prefrontal cortex represents both the identity and expected variability of features in service of top-down attentional control.SIGNIFICANCE STATEMENT Theories of attention and working memory posit that when we engage in complex cognitive tasks our performance is determined by how precisely we remember task-relevant information. However, in the real world the properties of objects change over time, creating uncertainty about many aspects of the task. There is currently a gap in our understanding of how neural systems represent this uncertainty and combine it with target identity information in anticipation of attention demanding cognitive tasks. In this study, we show that the prefrontal cortex represents identity and uncertainty as unique codes before task onset. These results advance theories of attention by showing that the prefrontal cortex codes both target identity and uncertainty to implement top-down attentional control.
Collapse
Affiliation(s)
- Phillip P Witkowski
- Center for Mind and Brain, University of California, Davis, Davis, California 95618
- Department of Psychology, University of California, Davis, Davis, California 95618
| | - Joy J Geng
- Center for Mind and Brain, University of California, Davis, Davis, California 95618
- Department of Psychology, University of California, Davis, Davis, California 95618
| |
Collapse
|
8
|
Tegelbeckers J, Porter DB, Voss JL, Schoenbaum G, Kahnt T. Lateral orbitofrontal cortex integrates predictive information across multiple cues to guide behavior. Curr Biol 2023; 33:4496-4504.e5. [PMID: 37804827 PMCID: PMC10622115 DOI: 10.1016/j.cub.2023.09.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Revised: 08/21/2023] [Accepted: 09/13/2023] [Indexed: 10/09/2023]
Abstract
Individuals are often faced with multiple cues that concurrently predict the same outcome, and combining these predictions may benefit behavior. Previous work has studied the neural basis of decision-making, predominantly using isolated sensory stimuli, and so the mechanisms that allow us to leverage multiple cues remain unclear. In two experiments, we used neuroimaging and network-targeted brain stimulation to probe how the brain integrates outcome predictions to guide adaptive behavior. We identified neural signatures of outcome integration in the lateral orbitofrontal cortex (OFC), where concurrently presented cues evoke stronger pattern-based representations of expected outcomes. Moreover, perturbing lateral OFC network activity impairs subjects' ability to leverage predictions from multiple cues to facilitate responding. Intriguingly, we found similar behavioral and brain mechanisms for reward-predicting cues and for cues predicting the absence of reward. These findings highlight a causal role for the lateral OFC in utilizing outcome predictions from multiple cues to guide behavior.
Collapse
Affiliation(s)
- Jana Tegelbeckers
- Northwestern University, Feinberg School of Medicine, 420 E Superior St, Chicago, IL 60611, USA; Otto von Guericke University, Universitaetsplatz 2, 39106 Magdeburg, Germany.
| | - Daria B Porter
- Northwestern University, Feinberg School of Medicine, 420 E Superior St, Chicago, IL 60611, USA
| | - Joel L Voss
- University of Chicago, 5841 S Maryland Ave, Chicago, IL 60637, USA
| | - Geoffrey Schoenbaum
- National Institute on Drug Abuse Intramural Research Program, 251 Bayview Blvd, Baltimore, MD 21224, USA
| | - Thorsten Kahnt
- National Institute on Drug Abuse Intramural Research Program, 251 Bayview Blvd, Baltimore, MD 21224, USA.
| |
Collapse
|
9
|
Lamba A, Nassar MR, FeldmanHall O. Prefrontal cortex state representations shape human credit assignment. eLife 2023; 12:e84888. [PMID: 37399050 PMCID: PMC10351919 DOI: 10.7554/elife.84888] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Accepted: 06/16/2023] [Indexed: 07/04/2023] Open
Abstract
People learn adaptively from feedback, but the rate of such learning differs drastically across individuals and contexts. Here, we examine whether this variability reflects differences in what is learned. Leveraging a neurocomputational approach that merges fMRI and an iterative reward learning task, we link the specificity of credit assignment-how well people are able to appropriately attribute outcomes to their causes-to the precision of neural codes in the prefrontal cortex (PFC). Participants credit task-relevant cues more precisely in social compared vto nonsocial contexts, a process that is mediated by high-fidelity (i.e., distinct and consistent) state representations in the PFC. Specifically, the medial PFC and orbitofrontal cortex work in concert to match the neural codes from feedback to those at choice, and the strength of these common neural codes predicts credit assignment precision. Together this work provides a window into how neural representations drive adaptive learning.
Collapse
Affiliation(s)
- Amrita Lamba
- Department of Cognitive Linguistic & Psychological Sciences, Brown UniversityProvidenceUnited States
| | - Matthew R Nassar
- Department of Neuroscience, Brown UniversityProvidenceUnited States
- Carney Institute of Brain Sciences, Brown UniversityProvidenceUnited States
| | - Oriel FeldmanHall
- Department of Cognitive Linguistic & Psychological Sciences, Brown UniversityProvidenceUnited States
- Carney Institute of Brain Sciences, Brown UniversityProvidenceUnited States
| |
Collapse
|
10
|
De Martino B, Cortese A. Goals, usefulness and abstraction in value-based choice. Trends Cogn Sci 2023; 27:65-80. [PMID: 36446707 DOI: 10.1016/j.tics.2022.11.001] [Citation(s) in RCA: 13] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Revised: 10/26/2022] [Accepted: 11/01/2022] [Indexed: 11/27/2022]
Abstract
Colombian drug lord Pablo Escobar, while on the run, purportedly burned two million dollars in banknotes to keep his daughter warm. A stark reminder that, in life, circumstances and goals can quickly change, forcing us to reassess and modify our values on-the-fly. Studies in decision-making and neuroeconomics have often implicitly equated value to reward, emphasising the hedonic and automatic aspect of the value computation, while overlooking its functional (concept-like) nature. Here we outline the computational and biological principles that enable the brain to compute the usefulness of an option or action by creating abstractions that flexibly adapt to changing goals. We present different algorithmic architectures, comparing ideas from artificial intelligence (AI) and cognitive neuroscience with psychological theories and, when possible, drawing parallels.
Collapse
Affiliation(s)
- Benedetto De Martino
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK; Computational Neuroscience Laboratories, ATR Institute International, 619-0288 Kyoto, Japan.
| | - Aurelio Cortese
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK; Computational Neuroscience Laboratories, ATR Institute International, 619-0288 Kyoto, Japan.
| |
Collapse
|