1
|
Philippe R, Janet R, Khalvati K, Rao RPN, Lee D, Dreher JC. Neurocomputational mechanisms involved in adaptation to fluctuating intentions of others. Nat Commun 2024; 15:3189. [PMID: 38609372 PMCID: PMC11014977 DOI: 10.1038/s41467-024-47491-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2022] [Accepted: 03/12/2024] [Indexed: 04/14/2024] Open
Abstract
Humans frequently interact with agents whose intentions can fluctuate between competition and cooperation over time. It is unclear how the brain adapts to fluctuating intentions of others when the nature of the interactions (to cooperate or compete) is not explicitly and truthfully signaled. Here, we use model-based fMRI and a task in which participants thought they were playing with another player. In fact, they played with an algorithm that alternated without signaling between cooperative and competitive strategies. We show that a neurocomputational mechanism with arbitration between competitive and cooperative experts outperforms other learning models in predicting choice behavior. At the brain level, the fMRI results show that the ventral striatum and ventromedial prefrontal cortex track the difference of reliability between these experts. When attributing competitive intentions, we find increased coupling between these regions and a network that distinguishes prediction errors related to competition and cooperation. These findings provide a neurocomputational account of how the brain arbitrates dynamically between cooperative and competitive intentions when making adaptive social decisions.
Collapse
Affiliation(s)
- Rémi Philippe
- CNRS-Institut des Sciences Cognitives Marc Jeannerod, UMR5229, Neuroeconomics, reward, and decision making laboratory, Lyon, France
- Université Claude Bernard Lyon 1, Lyon, France
| | - Rémi Janet
- CNRS-Institut des Sciences Cognitives Marc Jeannerod, UMR5229, Neuroeconomics, reward, and decision making laboratory, Lyon, France
- Université Claude Bernard Lyon 1, Lyon, France
| | - Koosha Khalvati
- Paul G. Allen School of Computer Science and Engineering, University of Washington, Seattle, WA, USA
| | - Rajesh P N Rao
- Paul G. Allen School of Computer Science and Engineering, University of Washington, Seattle, WA, USA
- Center for Neurotechnology, University of Washington, Seattle, WA, USA
| | - Daeyeol Lee
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD, USA
- Kavli Discovery Neuroscience Institute, Johns Hopkins University, Baltimore, MD, USA
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD, USA
| | - Jean-Claude Dreher
- CNRS-Institut des Sciences Cognitives Marc Jeannerod, UMR5229, Neuroeconomics, reward, and decision making laboratory, Lyon, France.
- Université Claude Bernard Lyon 1, Lyon, France.
| |
Collapse
|
2
|
Kazinka R, Kwashie AND, Pratt DN, Vilares I, MacDonald AW. Value Representations of Spite Sensitivity in Psychosis on the Minnesota Trust Game. BIOLOGICAL PSYCHIATRY. COGNITIVE NEUROSCIENCE AND NEUROIMAGING 2024; 9:429-436. [PMID: 38096987 PMCID: PMC10999326 DOI: 10.1016/j.bpsc.2023.11.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Revised: 10/20/2023] [Accepted: 11/05/2023] [Indexed: 02/17/2024]
Abstract
BACKGROUND Spite sensitivity provides a valuable construct to understand persecutory ideation and its underlying neural mechanisms. We examined the relationship between persecution and spite sensitivity in psychosis to identify their neural substrates. METHODS In a 3T magnetic resonance imaging scanner, 49 participants with psychosis played the Minnesota Trust Game, in which they decided whether to take a small amount of money or trust a partner to choose between fair and unfair distributions of money. In some conditions, the partner benefited from the unfair option, while in others, the partner lost money. Participants who were untrusting in the second condition (suspiciousness) showed heightened sensitivity to spite. Behavioral measures included mistrust during the 2 conditions of the game, which were compared with Brief Psychiatric Rating Scale persecution and computational modeling. Functional connectivity and blood oxygen level-dependent analyses were also conducted on a priori regions during spite-sensitive decisions. RESULTS Behavioral results replicated previous findings; participants who experienced more persecutory ideation trusted less, specifically in the suspiciousness condition. Functional connectivity findings showed that decreased connectivity between the orbitofrontal cortex-insula and the left frontoparietal network was associated with increased persecutory ideation and estimated spite-guilt (a marker of spite sensitivity). Additionally, we found differences between conditions in caudate nucleus, medial prefrontal cortex, and lateral orbitofrontal cortex activation. CONCLUSIONS These findings provide a new perspective on the origin of positive symptoms by identifying primary brain circuits that are related to both spite sensitivity and persecutory ideation.
Collapse
Affiliation(s)
- Rebecca Kazinka
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota; Department of Psychiatry and Behavioral Medicine, University of Minnesota School of Medicine, Minneapolis, Minnesota
| | | | - Danielle N Pratt
- Department of Psychology, Northwestern University, Evanston, Illinois
| | - Iris Vilares
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota
| | - Angus W MacDonald
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota.
| |
Collapse
|
3
|
Ota K, Charles L, Haggard P. Autonomous behaviour and the limits of human volition. Cognition 2024; 244:105684. [PMID: 38101173 DOI: 10.1016/j.cognition.2023.105684] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Revised: 11/22/2023] [Accepted: 11/28/2023] [Indexed: 12/17/2023]
Abstract
Humans and some other animals can autonomously generate action choices that contribute to solving complex problems. However, experimental investigations of the cognitive bases of human autonomy are challenging, because experimental paradigms typically constrain behaviour using controlled contexts, and elicit behaviour by external triggers. In contrast, autonomy and freedom imply unconstrained behaviour initiated by endogenous triggers. Here we propose a new theoretical construct of adaptive autonomy, meaning the capacity to make behavioural choices that are free from constraints of both immediate external triggers and of routine response patterns, but nevertheless show appropriate coordination with the environment. Participants (N = 152) played a competitive game in which they had to choose the right time to act, in the face of an opponent who punished (in separate blocks) either choice biases (such as always responding early), sequential patterns of action timing across trials (such as early, late, early, late…), or predictable action-outcome dependence (such as win-stay, lose-shift). Adaptive autonomy was quantified as the ability to maintain performance when each of these influences on action selection was punished. We found that participants could become free from habitual choices regarding when to act and could also become free from sequential action patterns. However, they were not able to free themselves from influences of action-outcome dependence, even when these resulted in poor performance. These results point to a new concept of autonomous behaviour as flexible adaptation of voluntary action choices in a way that avoids stereotypy. In a sequential analysis, we also demonstrated that participants increased their reliance on belief learning in which they attempt to understand the competitor's beliefs and intentions, when transition bias and reinforcement bias were punished. Taken together, our study points to a cognitive mechanism of adaptive autonomy in which competitive interactions with other agents could promote both social cognition and volition in the form of non-stereotyped action choices.
Collapse
Affiliation(s)
- Keiji Ota
- Institute of Cognitive Neuroscience, University College London, London, United Kingdom; Department of Psychology, School of Biological and Behavioural Sciences, Queen Mary University of London, London, United Kingdom.
| | - Lucie Charles
- Institute of Cognitive Neuroscience, University College London, London, United Kingdom; Department of Psychology, School of Biological and Behavioural Sciences, Queen Mary University of London, London, United Kingdom
| | - Patrick Haggard
- Institute of Cognitive Neuroscience, University College London, London, United Kingdom
| |
Collapse
|
4
|
Nougaret S, Ferrucci L, Ceccarelli F, Sacchetti S, Benozzo D, Fascianelli V, Saunders RC, Renaud L, Genovesio A. Neurons in the monkey frontopolar cortex encode learning stage and goal during a fast learning task. PLoS Biol 2024; 22:e3002500. [PMID: 38363801 PMCID: PMC10903959 DOI: 10.1371/journal.pbio.3002500] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2023] [Revised: 02/29/2024] [Accepted: 01/17/2024] [Indexed: 02/18/2024] Open
Abstract
The frontopolar cortex (FPC) is, to date, one of the least understood regions of the prefrontal cortex. The current understanding of its function suggests that it plays a role in the control of exploratory behaviors by coordinating the activities of other prefrontal cortex areas involved in decision-making and exploiting actions based on their outcomes. Based on this hypothesis, FPC would drive fast-learning processes through a valuation of the different alternatives. In our study, we used a modified version of a well-known paradigm, the object-in-place (OIP) task, to test this hypothesis in electrophysiology. This paradigm is designed to maximize learning, enabling monkeys to learn in one trial, which is an ability specifically impaired after a lesion of the FPC. We showed that FPC neurons presented an extremely specific pattern of activity by representing the learning stage, exploration versus exploitation, and the goal of the action. However, our results do not support the hypothesis that neurons in the frontal pole compute an evaluation of different alternatives. Indeed, the position of the chosen target was strongly encoded at its acquisition, but the position of the unchosen target was not. Once learned, this representation was also found at the problem presentation, suggesting a monitoring activity of the synthetic goal preceding its acquisition. Our results highlight important features of FPC neurons in fast-learning processes without confirming their role in the disengagement of cognitive control from the current goals.
Collapse
Affiliation(s)
- Simon Nougaret
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Lorenzo Ferrucci
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Francesco Ceccarelli
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
- PhD program in Behavioral Neuroscience, Sapienza University of Rome, Rome, Italy
| | - Stefano Sacchetti
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Danilo Benozzo
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Valeria Fascianelli
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| | - Richard C. Saunders
- Laboratory of Neuropsychology, National Institute of Mental Health, Bethesda, Maryland, United States of America
| | - Luc Renaud
- Institut de Neurosciences de la Timone, UMR7289, Centre National de la Recherche Scientifique and Aix-Marseille Université, Marseille, France
| | - Aldo Genovesio
- Department of Physiology and Pharmacology, Sapienza University of Rome, Rome, Italy
| |
Collapse
|
5
|
Zhang Y, Ye W, Yin J, Wu Q, Huang Y, Hao N, Cui L, Zhang M, Cai D. Exploring the role of mutual prediction in inter-brain synchronization during competitive interactions: an fNIRS hyperscanning investigation. Cereb Cortex 2024; 34:bhad483. [PMID: 38100358 DOI: 10.1093/cercor/bhad483] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Revised: 11/27/2023] [Accepted: 11/28/2023] [Indexed: 12/17/2023] Open
Abstract
Mutual prediction is crucial for understanding the mediation of bodily actions in social interactions. Despite this importance, limited studies have investigated neurobehavioral patterns under the mutual prediction hypothesis in natural competitive scenarios. To address this gap, our study employed functional near-infrared spectroscopy hyperscanning to examine the dynamics of real-time rock-paper-scissors games using a computerized paradigm with 54 participants. Firstly, our results revealed activations in the right inferior frontal gyrus, bilateral dorsolateral prefrontal cortex, and bilateral frontopolar cortex, each displaying distinct temporal profiles indicative of diverse cognitive processes during the task. Subsequently, a task-related increase in inter-brain synchrony was explicitly identified in the right dorsolateral prefrontal cortex, which supported the mutual prediction hypothesis across the two brains. Moreover, our investigation uncovered a close association between the coherence value in the right dorsolateral prefrontal cortex and the dynamic predictive performances of dyads using inter-subject representational similarity analysis. Finally, heightened inter-brain synchrony values were observed in the right dorsolateral prefrontal cortex before a draw compared to a no-draw scenario in the second block, suggesting that cross-brain signal patterns could be reflected in behavioral responses during competition. In summary, these findings provided initial support for expanding the understanding of cognitive processes underpinning natural competitive engagements.
Collapse
Affiliation(s)
- Yuxuan Zhang
- School of Psychology, Shanghai Normal University, Shanghai 200234, China
| | - Weihao Ye
- School of Psychology, Shanghai Normal University, Shanghai 200234, China
- School of Psychology, Zhejiang Normal University, Zhejiang 321004, China
| | - Junting Yin
- School of Psychology, Shanghai Normal University, Shanghai 200234, China
| | - Qin Wu
- School of Psychology, Shanghai Normal University, Shanghai 200234, China
| | - Yao Huang
- School of Psychology, Shanghai Normal University, Shanghai 200234, China
| | - Na Hao
- School of Psychology, Shanghai Normal University, Shanghai 200234, China
| | - Liying Cui
- School of Psychology, Shanghai Normal University, Shanghai 200234, China
| | - Mingming Zhang
- School of Psychology, Shanghai Normal University, Shanghai 200234, China
| | - Dan Cai
- School of Psychology, Shanghai Normal University, Shanghai 200234, China
| |
Collapse
|
6
|
Alizadeh Mansouri F, Buckley MJ, Tanaka K. Mapping causal links between prefrontal cortical regions and intra-individual behavioral variability. Nat Commun 2024; 15:140. [PMID: 38168052 PMCID: PMC10762061 DOI: 10.1038/s41467-023-44341-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Accepted: 12/08/2023] [Indexed: 01/05/2024] Open
Abstract
Intra-individual behavioral variability is significantly heightened by aging or neuropsychological disorders, however it is unknown which brain regions are causally linked to such variabilities. We examine response time (RT) variability in 21 macaque monkeys performing a rule-guided decision-making task. In monkeys with selective-bilateral lesions in the anterior cingulate cortex (ACC) or in the dorsolateral prefrontal cortex, cognitive flexibility is impaired, but the RT variability is significantly diminished. Bilateral lesions within the frontopolar cortex or within the mid-dorsolateral prefrontal cortex, has no significant effect on cognitive flexibility or RT variability. In monkeys with lesions in the posterior cingulate cortex, the RT variability significantly increases without any deficit in cognitive flexibility. The effect of lesions in the orbitofrontal cortex (OFC) is unique in that it leads to deficits in cognitive flexibility and a significant increase in RT variability. Our findings indicate remarkable dissociations in contribution of frontal cortical regions to behavioral variability. They suggest that the altered variability in OFC-lesioned monkeys is related to deficits in assessing and accumulating evidence to inform a rule-guided decision, whereas in ACC-lesioned monkeys it results from a non-adaptive decrease in decision threshold and consequently immature impulsive responses.
Collapse
Affiliation(s)
- Farshad Alizadeh Mansouri
- Cognitive Neuroscience Laboratory, Department of Physiology and Neuroscience Program, Biomedicine Discovery Institute, Monash University, Clayton, VIC, 3800, Australia.
- RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan.
| | - Mark J Buckley
- Department of Experimental Psychology, Oxford University, Oxford, OX1 3UD, UK
| | - Keiji Tanaka
- RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan
| |
Collapse
|
7
|
Yun M, Nejime M, Kawai T, Kunimatsu J, Yamada H, Kim HR, Matsumoto M. Distinct roles of the orbitofrontal cortex, ventral striatum, and dopamine neurons in counterfactual thinking of decision outcomes. SCIENCE ADVANCES 2023; 9:eadh2831. [PMID: 37556536 PMCID: PMC10411892 DOI: 10.1126/sciadv.adh2831] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Accepted: 07/07/2023] [Indexed: 08/11/2023]
Abstract
Individuals often assess past decisions by comparing what was gained with what would have been gained had they acted differently. Thoughts of past alternatives that counter what actually happened are called "counterfactuals." Recent theories emphasize the role of the prefrontal cortex in processing counterfactual outcomes in decision-making, although how subcortical regions contribute to this process remains to be elucidated. Here we report a clear distinction among the roles of the orbitofrontal cortex, ventral striatum and midbrain dopamine neurons in processing counterfactual outcomes in monkeys. Our findings suggest that actually gained and counterfactual outcome signals are both processed in the cortico-subcortical network constituted by these regions but in distinct manners and integrated only in the orbitofrontal cortex in a way to compare these outcomes. This study extends the prefrontal theory of counterfactual thinking and provides key insights regarding how the prefrontal cortex cooperates with subcortical regions to make decisions using counterfactual information.
Collapse
Affiliation(s)
- Mengxi Yun
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
- Graduate School of Comprehensive Human Science, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA, USA
- Transborder Medical Research Center, University of Tsukuba, Tsukuba, Ibaraki 305-8577, Japan
| | - Masafumi Nejime
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
| | - Takashi Kawai
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
| | - Jun Kunimatsu
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
- Transborder Medical Research Center, University of Tsukuba, Tsukuba, Ibaraki 305-8577, Japan
| | - Hiroshi Yamada
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
- Transborder Medical Research Center, University of Tsukuba, Tsukuba, Ibaraki 305-8577, Japan
| | - HyungGoo R. Kim
- Department of Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Biomedical Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon 16419, Republic of Korea
| | - Masayuki Matsumoto
- Division of Biomedical Science, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki 305-8575, Japan
- Transborder Medical Research Center, University of Tsukuba, Tsukuba, Ibaraki 305-8577, Japan
| |
Collapse
|
8
|
Wang H, Kwan AC. Competitive and cooperative games for probing the neural basis of social decision-making in animals. Neurosci Biobehav Rev 2023; 149:105158. [PMID: 37019249 PMCID: PMC10175234 DOI: 10.1016/j.neubiorev.2023.105158] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 03/29/2023] [Accepted: 04/02/2023] [Indexed: 04/07/2023]
Abstract
In a social environment, it is essential for animals to consider the behavior of others when making decisions. To quantitatively assess such social decisions, games offer unique advantages. Games may have competitive and cooperative components, modeling situations with antagonistic and shared objectives between players. Games can be analyzed by mathematical frameworks, including game theory and reinforcement learning, such that an animal's choice behavior can be compared against the optimal strategy. However, so far games have been underappreciated in neuroscience research, particularly for rodent studies. In this review, we survey the varieties of competitive and cooperative games that have been tested, contrasting strategies employed by non-human primates and birds with rodents. We provide examples of how games can be used to uncover neural mechanisms and explore species-specific behavioral differences. We assess critically the limitations of current paradigms and propose improvements. Together, the synthesis of current literature highlights the advantages of using games to probe the neural basis of social decisions for neuroscience studies.
Collapse
Affiliation(s)
- Hongli Wang
- Interdepartmental Neuroscience Program, Yale University School of Medicine, New Haven, CT, USA
| | - Alex C Kwan
- Department of Psychiatry, Yale University School of Medicine, New Haven, CT, USA; Department of Neuroscience, Yale University School of Medicine, New Haven, CT, USA; Meinig School of Biomedical Engineering, Cornell University, Ithaca, NY, USA; Department of Psychiatry, Weill Cornell Medicine, New York, NY 10065, USA.
| |
Collapse
|
9
|
Jahn CI, Grohn J, Cuell S, Emberton A, Bouret S, Walton ME, Kolling N, Sallet J. Neural responses in macaque prefrontal cortex are linked to strategic exploration. PLoS Biol 2023; 21:e3001985. [PMID: 36716348 PMCID: PMC9910800 DOI: 10.1371/journal.pbio.3001985] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2022] [Revised: 02/09/2023] [Accepted: 01/03/2023] [Indexed: 02/01/2023] Open
Abstract
Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on scarce resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback about the unchosen option. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen, which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.
Collapse
Affiliation(s)
- Caroline I. Jahn
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Motivation, Brain and Behavior Team, Institut du Cerveau et de la Moelle Epinière, Paris, France
- Sorbonne Paris Cité universités, Université Paris Descartes, Frontières du Vivant, Paris, France
- * E-mail: (CIJ); (JG); (NK); (JS)
| | - Jan Grohn
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- * E-mail: (CIJ); (JG); (NK); (JS)
| | - Steven Cuell
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Andrew Emberton
- Biomedical Science Services, University of Oxford, Oxford, United Kingdom
| | - Sebastien Bouret
- Motivation, Brain and Behavior Team, Institut du Cerveau et de la Moelle Epinière, Paris, France
| | - Mark E. Walton
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Nils Kolling
- Wellcome Centre for Integrative Neuroimaging, OBHA, University of Oxford, Headington, United Kingdom
- Univ Lyon, Université Lyon 1, Inserm, Stem Cell and Brain Research Institute U1208, Bron, France
- * E-mail: (CIJ); (JG); (NK); (JS)
| | - Jérôme Sallet
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Univ Lyon, Université Lyon 1, Inserm, Stem Cell and Brain Research Institute U1208, Bron, France
- * E-mail: (CIJ); (JG); (NK); (JS)
| |
Collapse
|
10
|
Zhao H, Zhang T, Cheng T, Chen C, Zhai Y, Liang X, Cheng N, Long Y, Li Y, Wang Z, Lu C. Neurocomputational mechanisms of young children's observational learning of delayed gratification. Cereb Cortex 2022; 33:6063-6076. [PMID: 36562999 DOI: 10.1093/cercor/bhac484] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Revised: 11/13/2022] [Accepted: 11/17/2022] [Indexed: 12/24/2022] Open
Abstract
The ability to delay gratification is crucial for a successful and healthy life. An effective way for young children to learn this ability is to observe the action of adult models. However, the underlying neurocomputational mechanism remains unknown. Here, we tested the hypotheses that children employed either the simple imitation strategy or the goal-inference strategy when learning from adult models in a high-uncertainty context. Results of computational modeling indicated that children used the goal-inference strategy regardless of whether the adult model was their mother or a stranger. At the neural level, results showed that successful learning of delayed gratification was associated with enhanced interpersonal neural synchronization (INS) between children and the adult models in the dorsal lateral prefrontal cortex but was not associated with children's own single-brain activity. Moreover, the discounting of future reward's value obtained from computational modeling of the goal-inference strategy was positively correlated with the strength of INS. These findings from our exploratory study suggest that, even for 3-year-olds, the goal-inference strategy is used to learn delayed gratification from adult models, and the learning strategy is associated with neural interaction between the brains of children and adult models.
Collapse
Affiliation(s)
- Hui Zhao
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, P.R. China
| | - Tengfei Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, P.R. China
| | - Tong Cheng
- Research Center for Child Development, School of Psychology, Capital Normal University, Beijing 100048, P.R. China
| | - Chuansheng Chen
- Department of Psychological Science, University of California, Irvine, CA 92697, United States
| | - Yu Zhai
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, P.R. China
| | - Xi Liang
- Research Center for Child Development, School of Psychology, Capital Normal University, Beijing 100048, P.R. China
| | - Nanhua Cheng
- Research Center for Child Development, School of Psychology, Capital Normal University, Beijing 100048, P.R. China
| | - Yuhang Long
- Institute of Developmental Psychology, Faculty of Psychology, Beijing Normal University, Beijing 100875, China
| | - Ying Li
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, P.R. China
| | - Zhengyan Wang
- Research Center for Child Development, School of Psychology, Capital Normal University, Beijing 100048, P.R. China
| | - Chunming Lu
- State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, P.R. China
| |
Collapse
|
11
|
Durand-de Cuttoli R, Martínez-Rivera FJ, Li L, Minier-Toribio A, Holt LM, Cathomas F, Yasmin F, Elhassa SO, Shaikh JF, Ahmed S, Russo SJ, Nestler EJ, Sweis BM. Distinct forms of regret linked to resilience versus susceptibility to stress are regulated by region-specific CREB function in mice. SCIENCE ADVANCES 2022; 8:eadd5579. [PMID: 36260683 PMCID: PMC9581472 DOI: 10.1126/sciadv.add5579] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Revised: 06/23/2022] [Accepted: 08/30/2022] [Indexed: 05/31/2023]
Abstract
Regret describes recognizing alternative actions could have led to better outcomes. It remains unclear whether regret derives from generalized mistake appraisal or instead comprises dissociable, action-specific processes. Using a neuroeconomic task, we found that mice were sensitive to fundamentally distinct types of regret following exposure to chronic social defeat stress or manipulations of CREB, a transcription factor implicated in stress action. Bias to make compensatory decisions after rejecting high-value offers (regret type I) was unique to stress-susceptible mice. Bias following the converse operation, accepting low-value offers (regret type II), was enhanced in stress-resilient mice and absent in stress-susceptible mice. CREB function in either the prefrontal cortex or nucleus accumbens was required to suppress regret type I but bidirectionally regulated regret type II. We provide insight into how maladaptive stress response traits relate to distinct forms of counterfactual thinking, which could steer therapy for mood disorders, such as depression, toward circuit-specific computations through a careful description of decision narrative.
Collapse
Affiliation(s)
- Romain Durand-de Cuttoli
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Freddyson J. Martínez-Rivera
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Long Li
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Angélica Minier-Toribio
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Leanne M. Holt
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Flurin Cathomas
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Farzana Yasmin
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Salma O. Elhassa
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Jasmine F. Shaikh
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Sanjana Ahmed
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Scott J. Russo
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Eric J. Nestler
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Brian M. Sweis
- Nash Family Department of Neuroscience, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| |
Collapse
|
12
|
Zhang T, Zhang Q, Wu J, Wang M, Li W, Yan J, Zhang J, Jin Z, Li L. The critical role of the orbitofrontal cortex for regret in an economic decision-making task. Brain Struct Funct 2022; 227:2751-2767. [DOI: 10.1007/s00429-022-02568-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2022] [Accepted: 09/06/2022] [Indexed: 11/28/2022]
|
13
|
Riceberg JS, Srinivasan A, Guise KG, Shapiro ML. Hippocampal signals modify orbitofrontal representations to learn new paths. Curr Biol 2022; 32:3407-3413.e6. [PMID: 35764092 PMCID: PMC11073633 DOI: 10.1016/j.cub.2022.06.010] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2021] [Revised: 04/17/2022] [Accepted: 06/07/2022] [Indexed: 10/17/2022]
Abstract
We often remember the consequences of past choices to adapt to changing circumstances. Recalling past events requires the hippocampus (HPC), and using stimuli to anticipate outcome values requires the orbitofrontal cortex (OFC).1-3 Spatial reversal tasks require both structures to navigate newly rewarded paths.4,5 Both HPC place6 and OFC value cells7,8 fire in phase with theta (4-12 Hz) oscillations. Both structures are described as cognitive maps: HPC maps space9 and OFC maps task states.10 These similarities imply that OFC-HPC interactions are crucial for using memory to predict outcomes when circumstances change, but the mechanisms remain largely unknown. To investigate possible interactions, we simultaneously recorded ensembles in OFC and CA1 as rats learned spatial reversals in a plus maze. Striking interactions occurred only while rats learned their first reversal: CA1 population vectors predicted changes in OFC activity but not vice versa, OFC spikes phase locked to hippocampal theta oscillations, mixed pairs of CA1 and OFC neurons fired together within single theta cycles, and CA1 led OFC spikes by ∼30 ms. After the new contingency became familiar, CA1 ensembles stably represented distinct spatial paths, whereas OFC ensembles developed more generalized goal arm representations in different paths to identical rewards. These frontotemporal interactions, engaged selectively when new task features inform decision-making, suggest a mechanism for linking novel episodes with expected outcomes, when HPC signals trigger "cognitive remapping" by OFC.11.
Collapse
Affiliation(s)
- Justin S Riceberg
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College, 47 New Scotland Avenue, MC-136, Albany, NY 12208, USA; Department of Psychiatry, Icahn School of Medicine at Mount Sinai, Hess Center for Science and Medicine, 1470 Madison Avenue, New York, NY 10029, USA.
| | - Aditya Srinivasan
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College, 47 New Scotland Avenue, MC-136, Albany, NY 12208, USA
| | - Kevin G Guise
- Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, Hess Center for Science and Medicine, 1470 Madison Avenue, New York, NY 10029, USA
| | - Matthew L Shapiro
- Department of Neuroscience and Experimental Therapeutics, Albany Medical College, 47 New Scotland Avenue, MC-136, Albany, NY 12208, USA
| |
Collapse
|
14
|
Dahal R, MacLellan K, Vavrek D, Dyson BJ. Assessing behavioural profiles following neutral, positive and negative feedback. PLoS One 2022; 17:e0270475. [PMID: 35788745 PMCID: PMC9255737 DOI: 10.1371/journal.pone.0270475] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2022] [Accepted: 06/10/2022] [Indexed: 12/02/2022] Open
Abstract
Previous data suggest zero-value, neutral outcomes (draw) are subjectively assigned negative rather than positive valence. The combined observations of faster rather than slower reaction times, subsequent actions defined by shift rather than stay behaviour, reduced flexibility, and, larger rather than smaller deviations from optimal performance following draws all align with the consequences of explicitly negative outcomes such as losses. We further tested the relationships between neutral, positive and negative outcomes by manipulating value salience and observing their behavioural profiles. Despite speeded reaction times and a non-significant bias towards shift behaviour similar to losses when draws were assigned the value of 0 (Experiment 1), the degree of shift behaviour approached an approximation of optimal performance when the draw value was explicitly positive (+1). This was in contrast to when the draw value was explicitly negative (-1), which led to a significant increase in the degree of shift behaviour (Experiment 2). Similar modifications were absent when the same value manipulations were applied to win or lose trials (Experiment 3). Rather than viewing draws as neutral and valence-free outcomes, the processing cascade generated by draws produces a complex behavioural profile containing elements found in response to both explicitly positive and explicitly negative results.
Collapse
Affiliation(s)
| | | | | | - Benjamin James Dyson
- University of Alberta, Edmonton, Canada
- University of Sussex, Brighton, United Kingdom
- Toronto Metropolian University, Toronto, Canada
- * E-mail:
| |
Collapse
|
15
|
Jiang Y, Wu H, Mi Q, Zhu L. Neurocomputations of strategic behavior: From iterated to novel interactions. WIRES COGNITIVE SCIENCE 2022; 13:e1598. [PMID: 35441465 PMCID: PMC9542218 DOI: 10.1002/wcs.1598] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/10/2021] [Revised: 03/27/2022] [Accepted: 03/29/2022] [Indexed: 11/15/2022]
Abstract
Strategic interactions, where an individual's payoff depends on the decisions of multiple intelligent agents, are ubiquitous among social animals. They span a variety of important social behaviors such as competition, cooperation, coordination, and communication, and often involve complex, intertwining cognitive operations ranging from basic reward processing to higher‐order mentalization. Here, we review the progress and challenges in probing the neural and cognitive mechanisms of strategic behavior of interacting individuals, drawing an analogy to recent developments in studies of reward‐seeking behavior, in particular, how research focuses in the field of strategic behavior have been expanded from adaptive behavior based on trial‐and‐error to flexible decisions based on limited prior experience. We highlight two important research questions in the field of strategic behavior: (i) How does the brain exploit past experience for learning to behave strategically? and (ii) How does the brain decide what to do in novel strategic situations in the absence of direct experience? For the former, we discuss the utility of learning models that have effectively connected various types of neural data with strategic learning behavior and helped elucidate the interplay among multiple learning processes. For the latter, we review the recent evidence and propose a neural generative mechanism by which the brain makes novel strategic choices through simulating others' goal‐directed actions according to rational or bounded‐rational principles obtained through indirect social knowledge. This article is categorized under:Economics > Interactive Decision‐Making Psychology > Reasoning and Decision Making Neuroscience > Cognition
Collapse
Affiliation(s)
- Yaomin Jiang
- School of Psychological and Cognitive Sciences, Beijing Key Laboratory of Behavior and Mental Health, IDG/McGovern Institute for Brain Research, Peking‐Tsinghua Center for Life Sciences Peking University Beijing China
| | - Hai‐Tao Wu
- School of Psychological and Cognitive Sciences, Beijing Key Laboratory of Behavior and Mental Health, IDG/McGovern Institute for Brain Research, Peking‐Tsinghua Center for Life Sciences Peking University Beijing China
| | - Qingtian Mi
- School of Psychological and Cognitive Sciences, Beijing Key Laboratory of Behavior and Mental Health, IDG/McGovern Institute for Brain Research, Peking‐Tsinghua Center for Life Sciences Peking University Beijing China
| | - Lusha Zhu
- School of Psychological and Cognitive Sciences, Beijing Key Laboratory of Behavior and Mental Health, IDG/McGovern Institute for Brain Research, Peking‐Tsinghua Center for Life Sciences Peking University Beijing China
| |
Collapse
|
16
|
Abstract
People with damage to the orbitofrontal cortex (OFC) have specific problems making decisions, whereas their other cognitive functions are spared. Neurophysiological studies have shown that OFC neurons fire in proportion to the value of anticipated outcomes. Thus, a central role of the OFC is to guide optimal decision-making by signalling values associated with different choices. Until recently, this view of OFC function dominated the field. New data, however, suggest that the OFC may have a much broader role in cognition by representing cognitive maps that can be used to guide behaviour and that value is just one of many variables that are important for behavioural control. In this Review, we critically evaluate these two alternative accounts of OFC function and examine how they might be reconciled.
Collapse
Affiliation(s)
- Eric B Knudsen
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California, USA
| | - Joni D Wallis
- Department of Psychology and Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California, USA.
| |
Collapse
|
17
|
Parr AC, Calancie OG, Coe BC, Khalid-Khan S, Munoz DP. Impulsivity and Emotional Dysregulation Predict Choice Behavior During a Mixed-Strategy Game in Adolescents With Borderline Personality Disorder. Front Neurosci 2022; 15:667399. [PMID: 35237117 PMCID: PMC8882924 DOI: 10.3389/fnins.2021.667399] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2021] [Accepted: 12/28/2021] [Indexed: 11/13/2022] Open
Abstract
Impulsivity and emotional dysregulation are two core features of borderline personality disorder (BPD), and the neural mechanisms recruited during mixed-strategy interactions overlap with frontolimbic networks that have been implicated in BPD. We investigated strategic choice patterns during the classic two-player game, Matching Pennies, where the most efficient strategy is to choose each option randomly from trial-to-trial to avoid exploitation by one’s opponent. Twenty-seven female adolescents with BPD (mean age: 16 years) and twenty-seven age-matched female controls (mean age: 16 years) participated in an experiment that explored the relationship between strategic choice behavior and impulsivity in both groups and emotional dysregulation in BPD. Relative to controls, BPD participants showed marginally fewer reinforcement learning biases, particularly decreased lose-shift biases, increased variability in reaction times (coefficient of variation; CV), and a greater percentage of anticipatory decisions. A subset of BPD participants with high levels of impulsivity showed higher overall reward rates, and greater modulation of reaction times by outcome, particularly following loss trials, relative to control and BPD participants with lower levels of impulsivity. Additionally, BPD participants with higher levels of emotional dysregulation showed marginally increased reward rate and increased entropy in choice patterns. Together, our preliminary results suggest that impulsivity and emotional dysregulation may contribute to variability in mixed-strategy decision-making in female adolescents with BPD.
Collapse
Affiliation(s)
- Ashley C. Parr
- Centre for Neuroscience Studies, Queen’s University, Kingston, ON, Canada
- Department of Psychiatry, University of Pittsburgh, Pittsburgh, PA, United States
- Division of Child and Youth Mental Health, Kingston Health Sciences Centre, Kingston, ON, Canada
- *Correspondence: Ashley C. Parr,
| | - Olivia G. Calancie
- Centre for Neuroscience Studies, Queen’s University, Kingston, ON, Canada
- Division of Child and Youth Mental Health, Kingston Health Sciences Centre, Kingston, ON, Canada
| | - Brian C. Coe
- Centre for Neuroscience Studies, Queen’s University, Kingston, ON, Canada
| | - Sarosh Khalid-Khan
- Centre for Neuroscience Studies, Queen’s University, Kingston, ON, Canada
- Division of Child and Youth Mental Health, Kingston Health Sciences Centre, Kingston, ON, Canada
| | - Douglas P. Munoz
- Centre for Neuroscience Studies, Queen’s University, Kingston, ON, Canada
- Department of Biomedical and Molecular Sciences, Queen’s University, Kingston, ON, Canada
- Douglas P. Munoz,
| |
Collapse
|
18
|
Fine JM, Hayden BY. The whole prefrontal cortex is premotor cortex. Philos Trans R Soc Lond B Biol Sci 2022; 377:20200524. [PMID: 34957853 PMCID: PMC8710885 DOI: 10.1098/rstb.2020.0524] [Citation(s) in RCA: 38] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2021] [Accepted: 10/01/2021] [Indexed: 11/12/2022] Open
Abstract
We propose that the entirety of the prefrontal cortex (PFC) can be seen as fundamentally premotor in nature. By this, we mean that the PFC consists of an action abstraction hierarchy whose core function is the potentiation and depotentiation of possible action plans at different levels of granularity. We argue that the apex of the hierarchy should revolve around the process of goal-selection, which we posit is inherently a form of optimization over action abstraction. Anatomical and functional evidence supports the idea that this hierarchy originates on the orbital surface of the brain and extends dorsally to motor cortex. Accordingly, our viewpoint positions the orbitofrontal cortex in a key role in the optimization of goal-selection policies, and suggests that its other proposed roles are aspects of this more general function. Our proposed perspective will reframe outstanding questions, open up new areas of inquiry and align theories of prefrontal function with evolutionary principles. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.
Collapse
Affiliation(s)
- Justin M. Fine
- Department of Neuroscience, Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Benjamin Y. Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| |
Collapse
|
19
|
Groman SM, Thompson SL, Lee D, Taylor JR. Reinforcement learning detuned in addiction: integrative and translational approaches. Trends Neurosci 2022; 45:96-105. [PMID: 34920884 PMCID: PMC8770604 DOI: 10.1016/j.tins.2021.11.007] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2021] [Revised: 11/04/2021] [Accepted: 11/19/2021] [Indexed: 02/03/2023]
Abstract
Suboptimal decision-making strategies have been proposed to contribute to the pathophysiology of addiction. Decision-making, however, arises from a collection of computational components that can independently influence behavior. Disruptions in these different components can lead to decision-making deficits that appear similar behaviorally, but differ at the computational, and likely the neurobiological, level. Here, we discuss recent studies that have used computational approaches to investigate the decision-making processes underlying addiction. Studies in animal models have found that value updating following positive, but not negative, outcomes is predictive of drug use, whereas value updating following negative, but not positive, outcomes is disrupted following drug self-administration. We contextualize these findings with studies on the circuit and biological mechanisms of decision-making to develop a framework for revealing the biobehavioral mechanisms of addiction.
Collapse
Affiliation(s)
- Stephanie M. Groman
- Department of Neuroscience, University of Minnesota,Department of Psychiatry, Yale University,Correspondence to be directed to: Stephanie Groman, 321 Church Street SE, 4-125 Jackson Hall Minneapolis MN 55455,
| | | | - Daeyeol Lee
- The Zanvyl Krieger Mind/Brain Institute, The Solomon H Snyder Department of Neuroscience, Department of Psychological and Brain Sciences, Kavli Neuroscience Discovery Institute, Johns Hopkins University
| | - Jane R. Taylor
- Department of Psychiatry, Yale University,Department of Neuroscience, Yale University,Department of Psychology, Yale University
| |
Collapse
|
20
|
Differential coding of goals and actions in ventral and dorsal corticostriatal circuits during goal-directed behavior. Cell Rep 2022; 38:110198. [PMID: 34986350 PMCID: PMC9608360 DOI: 10.1016/j.celrep.2021.110198] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2021] [Revised: 07/08/2021] [Accepted: 12/10/2021] [Indexed: 02/04/2023] Open
Abstract
Goal-directed behavior requires identifying objects in the environment that can satisfy internal needs and executing actions to obtain those objects. The current study examines ventral and dorsal corticostriatal circuits that support complementary aspects of goal-directed behavior. We analyze activity from the amygdala, ventral striatum, orbitofrontal cortex, and lateral prefrontal cortex (LPFC) while monkeys perform a three-armed bandit task. Information about chosen stimuli and their value is primarily encoded in the amygdala, ventral striatum, and orbitofrontal cortex, while the spatial information is primarily encoded in the LPFC. Before the options are presented, information about the to-be-chosen stimulus is represented in the amygdala, ventral striatum, and orbitofrontal cortex; at the time of choice, the information is passed to the LPFC to direct a saccade. Thus, learned value information specifying behavioral goals is maintained throughout the ventral corticostriatal circuit, and it is routed through the dorsal circuit at the time actions are selected.
Collapse
|
21
|
Mansouri FA, Buckley MJ, Tanaka K. The neural substrate and underlying mechanisms of executive control fluctuations in primates. Prog Neurobiol 2022; 209:102216. [PMID: 34995695 DOI: 10.1016/j.pneurobio.2022.102216] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2021] [Revised: 12/16/2021] [Accepted: 01/03/2022] [Indexed: 01/23/2023]
Abstract
Trial-by-trial alterations in response time have been linked to fluctuations of executive control and transient lapses of attention. Here, we report remarkable homologies in performance-dependent fluctuations of response time between humans and monkeys. We examined the effects of selective bilateral lesions in four frontal regions on control fluctuations in the context of a rule-shifting task. Lesions within orbitofrontal cortex (OFC), but not within superior-lateral prefrontal cortex, significantly exaggerated the performance-dependent fluctuations of control and prevented its restoration following feedback. Lesions within dorsolateral prefrontal cortex (DLPFC) or within anterior-cingulate cortex (ACC) led to instability of control and disruption of its link with monkeys' upcoming decisions. Examining the activity of DLPFC and OFC cells shed more lights on the underlying neuronal mechanisms by showing that before the start of each trial, OFC cell activity conveyed detailed information regarding the current state of executive control and the likelihood of success or failure in the future decisions. This further emphasizes the crucial role of OFC in the trial-by-trial allocation (setting) of control to the ongoing task. These findings bring insights to the neural architecture of executive control in primates and suggest that DLPFC and ACC support sustained executive control, but OFC is more involved in setting and restoring the control.
Collapse
Affiliation(s)
- Farshad Alizadeh Mansouri
- Cognitive Neuroscience Laboratory, Department of Physiology, Monash Biomedicine Discovery Institute, Monash University, Victoria, 3800, Australia; ARC Centre of Excellence for Integrative Brain Function, Monash University, VIC, Australia.
| | - Mark J Buckley
- Department of Experimental Psychology, Oxford University, Oxford, OX1 3UD, UK
| | - Keiji Tanaka
- Cognitive Brain Mapping Laboratory, RIKEN Center for Brain Science, Wako, Saitama, 351-0198, Japan
| |
Collapse
|
22
|
Dalle Molle R, de Mendonça Filho EJ, Minuzzi L, Machado TD, Reis RS, Rodrigues DM, Mucellini AB, Franco AR, Buchweitz A, Toazza R, Bortoluzzi A, Salum GA, Boscenco S, Meaney MJ, Levitan RD, Manfro GG, Silveira PP. Thrifty-Eating Behavior Phenotype at the Food Court - Programming Goes Beyond Food Preferences. Front Endocrinol (Lausanne) 2022; 13:882532. [PMID: 35677721 PMCID: PMC9168906 DOI: 10.3389/fendo.2022.882532] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/23/2022] [Accepted: 04/14/2022] [Indexed: 11/18/2022] Open
Abstract
INTRODUCTION Prenatal growth impairment leads to higher preference for palatable foods in comparison to normal prenatal growth subjects, which can contribute to increased body fat mass and a higher risk for developing chronic diseases in small-for-gestational-age (SGA) individuals throughout life. This study aimed to investigate the effect of SGA on feeding behavior in children and adolescents, as well as resting-state connectivity between areas related to reward, self-control, and value determination, such as orbitofrontal cortex (OFC), dorsolateral prefrontal cortex (DL-PFC), amygdala and dorsal striatum (DS). METHODS Caregivers and their offspring were recruited from two independent cohorts in Brazil (PROTAIA) and Canada (MAVAN). Both cohorts included anthropometric measurements, food choice tasks, and resting-state functional magnetic resonance imaging (fMRI) data. RESULTS In the Brazilian sample (17 ± 0.28 years, n=70), 21.4% of adolescents were classified as SGA. They exhibited lower monetary-related expenditure to buy a snack compared to controls in the food choice test. Decreased functional connectivity (n=40) between left OFC and left DL-PFC; and between right OFC and: left amygdala, right DS, and left DS were observed in the Brazilian SGA participants. Canadian SGA participants (14.9%) had non-significant differences in comparison with controls in a food choice task at 4 years old ( ± 0.01, n=315). At a follow-up brain scan visit (10.21 ± 0.140 years, n=49), SGA participants (28.6%) exhibited higher connectivity between the left OFC and left DL-PFC, also higher connectivity between the left OFC and right DL-PFC. We did not observe significant anthropometric neither nutrients' intake differences between groups in both samples. CONCLUSIONS Resting-state fMRI results showed that SGA individuals had altered connectivity between areas involved in encoding the subjective value for available goods and decision-making in both samples, which can pose them in disadvantage when facing food options daily. Over the years, the cumulative exposure to particular food cues together with the altered behavior towards food, such as food purchasing, as seen in the adolescent cohort, can play a role in the long-term risk for developing chronic non-communicable diseases.
Collapse
Affiliation(s)
- Roberta Dalle Molle
- Programa de Pós-Graduação em Ciências da Nutrição, Universidade Federal de Ciências da Saúde de Porto Alegre, Porto Alegre, Brazil
- Ludmer Centre for Neuroinformatics and Mental Health, Douglas Hospital Research Center, Montreal, QC, Canada
- Programa de Pós-Graduação em Saúde da Criança e do Adolescente, Faculdade de Medicina, Hospital de Clínicas de Porto Alegre, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
| | - Euclides José de Mendonça Filho
- Ludmer Centre for Neuroinformatics and Mental Health, Douglas Hospital Research Center, Montreal, QC, Canada
- Department of Psychiatry, McGill University, Montreal, QC, Canada
| | - Luciano Minuzzi
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada
| | - Tania Diniz Machado
- Programa de Pós-Graduação em Saúde da Criança e do Adolescente, Faculdade de Medicina, Hospital de Clínicas de Porto Alegre, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
| | - Roberta Sena Reis
- Programa de Pós-Graduação em Saúde da Criança e do Adolescente, Faculdade de Medicina, Hospital de Clínicas de Porto Alegre, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
- Faculdade de Nutrição, Universidade Federal de Goiás, Goiânia, Brazil
| | - Danitsa Marcos Rodrigues
- Programa de Pós-Graduação em Neurociências, Instituto de Ciências Básicas da Saúde, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
| | - Amanda Brondani Mucellini
- Programa de Pós-Graduação em Ciências Médicas: Psiquiatria, Faculdade de Medicina, Hospital de Clínicas de Porto Alegre, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
| | - Alexandre Rosa Franco
- Instituto do Cérebro (InsCer), Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS), Porto Alegre, Brazil
- Faculdade de Medicina, Programa de Pós-Graduação em Medicina e Ciências da Saúde, Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS), Porto Alegre, Brazil
- Faculdade de Engenharia, Programa de Pós-Graduação em Engenharia Elétrica, PUCRS, Porto Alegre, Brazil
| | - Augusto Buchweitz
- Instituto do Cérebro (InsCer), Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS), Porto Alegre, Brazil
- Faculdade de Medicina, Programa de Pós-Graduação em Medicina e Ciências da Saúde, Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS), Porto Alegre, Brazil
- Faculdade de Letras, Programa de Pós-Graduação em Letras, Linguística, Pontifícia Universidade Católica do Rio Grande do Sul (PUCRS), Porto Alegre, Brazil
| | - Rudineia Toazza
- Programa de Pós-Graduação em Neurociências, Instituto de Ciências Básicas da Saúde, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
| | - Andressa Bortoluzzi
- Programa de Pós-Graduação em Neurociências, Instituto de Ciências Básicas da Saúde, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
| | - Giovanni Abrahão Salum
- Programa de Pós-Graduação em Ciências Médicas: Psiquiatria, Faculdade de Medicina, Hospital de Clínicas de Porto Alegre, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
| | - Sonia Boscenco
- Department of Psychiatry, McGill University, Montreal, QC, Canada
| | - Michael J. Meaney
- Singapore Institute for Clinical Sciences, Agency for Science, Technology and Research (A∗STAR), Singapore, Singapore
| | - Robert D. Levitan
- Department of Psychiatry, University of Toronto and Centre for Addiction and Mental Health, Toronto, ON, Canada
| | - Gisele Gus Manfro
- Programa de Pós-Graduação em Neurociências, Instituto de Ciências Básicas da Saúde, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
- Programa de Pós-Graduação em Ciências Médicas: Psiquiatria, Faculdade de Medicina, Hospital de Clínicas de Porto Alegre, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil
| | - Patricia Pelufo Silveira
- Ludmer Centre for Neuroinformatics and Mental Health, Douglas Hospital Research Center, Montreal, QC, Canada
- Department of Psychiatry, McGill University, Montreal, QC, Canada
- *Correspondence: Patricia Pelufo Silveira,
| |
Collapse
|
23
|
The Neural Instantiation of an Abstract Cognitive Map for Economic Choice. Neuroscience 2021; 477:106-114. [PMID: 34543674 DOI: 10.1016/j.neuroscience.2021.09.011] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2021] [Revised: 09/01/2021] [Accepted: 09/09/2021] [Indexed: 11/24/2022]
Abstract
Since the discovery of cognitive maps in rodent hippocampus (HC), the cognitive map has evolved from originally referring to spatial representations encoding locations and objects in Euclidean spaces to a general low-dimensional organization of information along selected feature dimensions. A cognitive map includes hypothetical constructs that bridge between environmental stimuli and the final overt behavior. To neuroeconomists, utility and utility functions are such constructs with neurobiological basis that drive choice behavior. Emergence of distinct functional neuron groups in the primate orbitofrontal cortex (OFC) during simple economic choice indicates the formation of an abstract cognitive map for organizing information of goods for value computation. Experimental evidence suggests that organization of neuronal activity in such cognitive map reflects the abstraction of core task features. Thus, such map can be adapted to accommodate economic choices under various task contexts.
Collapse
|
24
|
Subramanian A, Chitlangia S, Baths V. Reinforcement learning and its connections with neuroscience and psychology. Neural Netw 2021; 145:271-287. [PMID: 34781215 DOI: 10.1016/j.neunet.2021.10.003] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2021] [Revised: 09/26/2021] [Accepted: 10/01/2021] [Indexed: 11/19/2022]
Abstract
Reinforcement learning methods have recently been very successful at performing complex sequential tasks like playing Atari games, Go and Poker. These algorithms have outperformed humans in several tasks by learning from scratch, using only scalar rewards obtained through interaction with their environment. While there certainly has been considerable independent innovation to produce such results, many core ideas in reinforcement learning are inspired by phenomena in animal learning, psychology and neuroscience. In this paper, we comprehensively review a large number of findings in both neuroscience and psychology that evidence reinforcement learning as a promising candidate for modeling learning and decision making in the brain. In doing so, we construct a mapping between various classes of modern RL algorithms and specific findings in both neurophysiological and behavioral literature. We then discuss the implications of this observed relationship between RL, neuroscience and psychology and its role in advancing research in both AI and brain science.
Collapse
Affiliation(s)
- Ajay Subramanian
- Department of Psychology, New York University, New York, New York, 10003, USA; Cognitive Neuroscience Lab, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India.
| | - Sharad Chitlangia
- Amazon; Cognitive Neuroscience Lab, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India.
| | - Veeky Baths
- Cognitive Neuroscience Lab, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India; Department of Biological Sciences, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India.
| |
Collapse
|
25
|
Wong A, Merholz G, Maoz U. Characterizing human random-sequence generation in competitive and non-competitive environments using Lempel-Ziv complexity. Sci Rep 2021; 11:20662. [PMID: 34667239 PMCID: PMC8526708 DOI: 10.1038/s41598-021-99967-6] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2021] [Accepted: 09/29/2021] [Indexed: 11/13/2022] Open
Abstract
The human ability for random-sequence generation (RSG) is limited but improves in a competitive game environment with feedback. However, it remains unclear how random people can be during games and whether RSG during games can improve when explicitly informing people that they must be as random as possible to win the game. Nor is it known whether any such improvement in RSG transfers outside the game environment. To investigate this, we designed a pre/post intervention paradigm around a Rock-Paper-Scissors game followed by a questionnaire. During the game, we manipulated participants’ level of awareness of the computer’s strategy; they were either (a) not informed of the computer’s algorithm or (b) explicitly informed that the computer used patterns in their choice history against them, so they must be maximally random to win. Using a compressibility metric of randomness, our results demonstrate that human RSG can reach levels statistically indistinguishable from computer pseudo-random generators in a competitive-game setting. However, our results also suggest that human RSG cannot be further improved by explicitly informing participants that they need to be random to win. In addition, the higher RSG in the game setting does not transfer outside the game environment. Furthermore, we found that the underrepresentation of long repetitions of the same entry in the series explains up to 29% of the variability in human RSG, and we discuss what might make up the variance left unexplained.
Collapse
Affiliation(s)
- Alice Wong
- Department of Psychology, University of California Los Angeles, Los Angeles, CA, USA.,Institute for Interdisciplinary Brain and Behavioral Sciences, Chapman University, Irvine, CA, USA.,Computational and Data Sciences, Schmid College of Science and Technology, Chapman University, Orange, CA, USA
| | - Garance Merholz
- Department of Psychology, University of California Los Angeles, Los Angeles, CA, USA.,Université de Paris, CNRS, Integrative Neuroscience and Cognition Center, 75006, Paris, France
| | - Uri Maoz
- Department of Psychology, University of California Los Angeles, Los Angeles, CA, USA. .,Institute for Interdisciplinary Brain and Behavioral Sciences, Chapman University, Irvine, CA, USA. .,Department of Psychology, Crean College of Health and Behavioral Sciences, Chapman University, One University Drive, Orange, CA, 92866, USA. .,Anderson School of Management, University of California Los Angeles, Los Angeles, CA, USA. .,Biology and Bioengineering, California Institute of Technology, Pasadena, CA, USA.
| |
Collapse
|
26
|
Bennett MS. Five Breakthroughs: A First Approximation of Brain Evolution From Early Bilaterians to Humans. Front Neuroanat 2021; 15:693346. [PMID: 34489649 PMCID: PMC8418099 DOI: 10.3389/fnana.2021.693346] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2021] [Accepted: 07/13/2021] [Indexed: 11/13/2022] Open
Abstract
Retracing the evolutionary steps by which human brains evolved can offer insights into the underlying mechanisms of human brain function as well as the phylogenetic origin of various features of human behavior. To this end, this article presents a model for interpreting the physical and behavioral modifications throughout major milestones in human brain evolution. This model introduces the concept of a "breakthrough" as a useful tool for interpreting suites of brain modifications and the various adaptive behaviors these modifications enabled. This offers a unique view into the ordered steps by which human brains evolved and suggests several unique hypotheses on the mechanisms of human brain function.
Collapse
|
27
|
Biderman N, Shohamy D. Memory and decision making interact to shape the value of unchosen options. Nat Commun 2021; 12:4648. [PMID: 34330909 PMCID: PMC8324852 DOI: 10.1038/s41467-021-24907-x] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2021] [Accepted: 07/08/2021] [Indexed: 02/07/2023] Open
Abstract
The goal of deliberation is to separate between options so that we can commit to one and leave the other behind. However, deliberation can, paradoxically, also form an association in memory between the chosen and unchosen options. Here, we consider this possibility and examine its consequences for how outcomes affect not only the value of the options we chose, but also, by association, the value of options we did not choose. In five experiments (total n = 612), including a preregistered experiment (n = 235), we found that the value assigned to unchosen options is inversely related to their chosen counterparts. Moreover, this inverse relationship was associated with participants' memory of the pairs they chose between. Our findings suggest that deciding between options does not end the competition between them. Deliberation binds choice options together in memory such that the learned value of one can affect the inferred value of the other.
Collapse
Affiliation(s)
- Natalie Biderman
- Department of Psychology and Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia University, New York, NY, USA.
| | - Daphna Shohamy
- Department of Psychology and Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia University, New York, NY, USA.
- The Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| |
Collapse
|
28
|
Bennett MS. What Behavioral Abilities Emerged at Key Milestones in Human Brain Evolution? 13 Hypotheses on the 600-Million-Year Phylogenetic History of Human Intelligence. Front Psychol 2021; 12:685853. [PMID: 34393912 PMCID: PMC8358274 DOI: 10.3389/fpsyg.2021.685853] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2021] [Accepted: 06/16/2021] [Indexed: 01/24/2023] Open
Abstract
This paper presents 13 hypotheses regarding the specific behavioral abilities that emerged at key milestones during the 600-million-year phylogenetic history from early bilaterians to extant humans. The behavioral, intellectual, and cognitive faculties of humans are complex and varied: we have abilities as diverse as map-based navigation, theory of mind, counterfactual learning, episodic memory, and language. But these faculties, which emerge from the complex human brain, are likely to have evolved from simpler prototypes in the simpler brains of our ancestors. Understanding the order in which behavioral abilities evolved can shed light on how and why our brains evolved. To propose these hypotheses, I review the available data from comparative psychology and evolutionary neuroscience.
Collapse
|
29
|
Knudsen EB, Wallis JD. Hippocampal neurons construct a map of an abstract value space. Cell 2021; 184:4640-4650.e10. [PMID: 34348112 DOI: 10.1016/j.cell.2021.07.010] [Citation(s) in RCA: 37] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2020] [Revised: 05/11/2021] [Accepted: 07/08/2021] [Indexed: 02/06/2023]
Abstract
The hippocampus is thought to encode a "cognitive map," a structural organization of knowledge about relationships in the world. Place cells, spatially selective hippocampal neurons that have been extensively studied in rodents, are one component of this map, describing the relative position of environmental features. However, whether this map extends to abstract, cognitive information remains unknown. Using the relative reward value of cues to define continuous "paths" through an abstract value space, we show that single neurons in primate hippocampus encode this space through value place fields, much like a rodent's place neurons encode paths through physical space. Value place fields remapped when cues changed but also became increasingly correlated across contexts, allowing maps to become generalized. Our findings help explain the critical contribution of the hippocampus to value-based decision-making, providing a mechanism by which knowledge of relationships in the world can be incorporated into reward predictions for guiding decisions.
Collapse
Affiliation(s)
- Eric B Knudsen
- Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, CA, USA.
| | - Joni D Wallis
- Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, CA, USA; Department of Psychology, University of California at Berkeley, Berkeley, CA, USA
| |
Collapse
|
30
|
Over-representation of fundamental decision variables in the prefrontal cortex underlies decision bias. Neurosci Res 2021; 173:1-13. [PMID: 34274406 DOI: 10.1016/j.neures.2021.07.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Revised: 06/15/2021] [Accepted: 07/13/2021] [Indexed: 11/24/2022]
Abstract
The brain is organized into anatomically distinct structures consisting of a variety of projection neurons. While such evolutionarily conserved neural circuit organization underlies the innate ability of animals to swiftly adapt to environments, they can cause biased cognition and behavior. Although recent studies have begun to address the causal importance of projection-neuron types as distinct computational units, it remains unclear how projection types are functionally organized in encoding variables during cognitive tasks. This review focuses on the neural computation of decision making in the prefrontal cortex and discusses what decision variables are encoded by single neurons, neuronal populations, and projection type, alongside how specific projection types constrain decision making. We focus particularly on "over-representations" of distinct decision variables in the prefrontal cortex that reflect the biological and subjective significance of the variables for the decision makers. We suggest that task-specific over-representation in the prefrontal cortex involves the refinement of the given decision making, while generalized over-representation of fundamental decision variables is associated with suboptimal decision biases, including pathological ones such as those in patients with psychiatric disorders. Such over-representation of the fundamental decision variables in the prefrontal cortex appear to be tightly constrained by afferent and efferent connections that can be optogenetically intervened on. These ideas may provide critical insights into potential therapeutic targets for psychiatric disorders, including addiction and depression.
Collapse
|
31
|
Allaert J, De Raedt R, van der Veen FM, Baeken C, Vanderhasselt MA. Prefrontal tDCS attenuates counterfactual thinking in female individuals prone to self-critical rumination. Sci Rep 2021; 11:11601. [PMID: 34078934 PMCID: PMC8172930 DOI: 10.1038/s41598-021-90677-7] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2020] [Accepted: 04/30/2021] [Indexed: 12/12/2022] Open
Abstract
The tendency to ruminate (i.e., repetitive negative self-referential thoughts that perpetuate depressive mood) is associated with (a) an elevated propensity to maladaptively experience counterfactual thinking (CFT) and regret, and (b) hypo-activity of the left dorsolateral prefrontal cortex (DLPFC). The goal of this study was to investigate whether anodal transcranial direct current stimulation (tDCS) over the left DLPFC, in function of self-critical rumination tendencies, momentarily reduces counterfactual thinking and regret (assessed via self-report and psychophysiological indices). Eighty healthy participants with different levels of self-critical rumination received either anodal or sham tDCS while performing a decision making task in which they were repeatedly confronted with optimal, suboptimal, and non-optimal choice outcomes. The results showed that among rumination-prone individuals, anodal (versus sham) tDCS was associated with decreased CFT and attenuated psychophysiological reactivity to the differential choice outcomes. Conversely, among low rumination-prone individuals, anodal (versus sham) tDCS was associated with increased CFT and regret, but in absence of any effects on psychophysiological reactivity. Potential working mechanisms for these differential tDCS effects are discussed. Taken together, these results provide initial converging evidence for the adaptive effects of left prefrontal tDCS on CFT and regret to personal choice outcomes among individuals prone to engage in self-critical rumination.
Collapse
Affiliation(s)
- Jens Allaert
- Department of Head and Skin, Ghent University, University Hospital Ghent (UZ Ghent), 1K12F, Corneel Heymanslaan 10, 9000, Ghent, Belgium. .,Ghent Experimental Psychiatry (GHEP) Lab, Ghent University, Ghent, Belgium. .,Department of Experimental Clinical and Health Psychology, Ghent University, Ghent, Belgium.
| | - Rudi De Raedt
- Department of Experimental Clinical and Health Psychology, Ghent University, Ghent, Belgium
| | - Frederik M van der Veen
- Department of Psychology, Education and Child Studies, Erasmus University Rotterdam, Rotterdam, The Netherlands
| | - Chris Baeken
- Department of Head and Skin, Ghent University, University Hospital Ghent (UZ Ghent), 1K12F, Corneel Heymanslaan 10, 9000, Ghent, Belgium.,Ghent Experimental Psychiatry (GHEP) Lab, Ghent University, Ghent, Belgium.,Department of Experimental Clinical and Health Psychology, Ghent University, Ghent, Belgium.,Department of Psychiatry, Vrije Universiteit Brussel (VUB), University Hospital UZBrussel, Brussels, Belgium
| | - Marie-Anne Vanderhasselt
- Department of Head and Skin, Ghent University, University Hospital Ghent (UZ Ghent), 1K12F, Corneel Heymanslaan 10, 9000, Ghent, Belgium.,Ghent Experimental Psychiatry (GHEP) Lab, Ghent University, Ghent, Belgium.,Department of Experimental Clinical and Health Psychology, Ghent University, Ghent, Belgium
| |
Collapse
|
32
|
Reward and fictive prediction error signals in ventral striatum: asymmetry between factual and counterfactual processing. Brain Struct Funct 2021; 226:1553-1569. [PMID: 33839955 DOI: 10.1007/s00429-021-02270-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2020] [Accepted: 03/27/2021] [Indexed: 10/21/2022]
Abstract
Reward prediction error, the difference between the expected and obtained reward, is known to act as a reinforcement learning neural signal. In the current study, we propose a model fitting approach that combines behavioral and neural data to fit computational models of reinforcement learning. Briefly, we penalized subject-specific fitted parameters that moved away too far from the group median, except when that deviation led to an improvement in the model's fit to neural responses. By means of a probabilistic monetary learning task and fMRI, we compared our approach with standard model fitting methods. Q-learning outperformed actor-critic at both behavioral and neural level, although the inclusion of neuroimaging data into model fitting improved the fit of actor-critic models. We observed both action-value and state-value prediction error signals in the striatum, while standard model fitting approaches failed to capture state-value signals. Finally, left ventral striatum correlated with reward prediction error while right ventral striatum with fictive prediction error, suggesting a functional hemispheric asymmetry regarding prediction-error driven learning.
Collapse
|
33
|
Piantadosi ST. The computational origin of representation. Minds Mach (Dordr) 2021; 31:1-58. [PMID: 34305318 PMCID: PMC8300595 DOI: 10.1007/s11023-020-09540-9] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2019] [Accepted: 08/29/2020] [Indexed: 01/29/2023]
Abstract
Each of our theories of mental representation provides some insight into how the mind works. However, these insights often seem incompatible, as the debates between symbolic, dynamical, emergentist, sub-symbolic, and grounded approaches to cognition attest. Mental representations-whatever they are-must share many features with each of our theories of representation, and yet there are few hypotheses about how a synthesis could be possible. Here, I develop a theory of the underpinnings of symbolic cognition that shows how sub-symbolic dynamics may give rise to higher-level cognitive representations of structures, systems of knowledge, and algorithmic processes. This theory implements a version of conceptual role semantics by positing an internal universal representation language in which learners may create mental models to capture dynamics they observe in the world. The theory formalizes one account of how truly novel conceptual content may arise, allowing us to explain how even elementary logical and computational operations may be learned from a more primitive basis. I provide an implementation that learns to represent a variety of structures, including logic, number, kinship trees, regular languages, context-free languages, domains of theories like magnetism, dominance hierarchies, list structures, quantification, and computational primitives like repetition, reversal, and recursion. This account is based on simple discrete dynamical processes that could be implemented in a variety of different physical or biological systems. In particular, I describe how the required dynamics can be directly implemented in a connectionist framework. The resulting theory provides an "assembly language" for cognition, where high-level theories of symbolic computation can be implemented in simple dynamics that themselves could be encoded in biologically plausible systems.
Collapse
|
34
|
Dyson BJ. Variability in competitive decision-making speed and quality against exploiting and exploitative opponents. Sci Rep 2021; 11:2859. [PMID: 33536472 PMCID: PMC7859242 DOI: 10.1038/s41598-021-82269-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2020] [Accepted: 01/18/2021] [Indexed: 12/02/2022] Open
Abstract
A presumption in previous work has been that sub-optimality in competitive performance following loss is the result of a reduction in decision-making time (i.e., post-error speeding). The main goal of this paper is to test the relationship between decision-making speed and quality, with the hypothesis that slowing down decision-making should increase the likelihood of successful performance in cases where a model of opponent domination can be implemented. Across Experiments 1–3, the speed and quality of competitive decision-making was examined in a zero-sum game as a function of the nature of the opponent (unexploitable, exploiting, exploitable). Performance was also examined against the nature of a credit (or token) system used as a within-experimental manipulation (no credit, fixed credit, variable credit). To compliment reaction time variation as a function of outcome, both the fixed credit and variable credit conditions were designed to slow down decision-making, relative to a no credit condition where the game could be played in quick succession and without interruption. The data confirmed that (a) self-imposed reductions in processing time following losses (post-error speeding) were causal factors in determining poorer-quality behaviour, (b) the expression of lose-shift was less flexible than the expression of win-stay, and, (c) the use of a variable credit system may enhance the perceived control participants have against exploitable opponents. Future work should seek to disentangle temporal delay and response interruption as determinants of decision-making quality against numerous styles of opponency.
Collapse
Affiliation(s)
- Benjamin James Dyson
- Department of Psychology, University of Alberta, P-217 Biological Sciences Building, Edmonton, AB, T6G 2E9, Canada. .,Ryerson University, Toronto, Canada. .,University of Sussex, Brighton, UK.
| |
Collapse
|
35
|
Watzek J, Brosnan SF. Capuchin and rhesus monkeys show sunk cost effects in a psychomotor task. Sci Rep 2020; 10:20396. [PMID: 33230238 PMCID: PMC7683735 DOI: 10.1038/s41598-020-77301-w] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2020] [Accepted: 11/03/2020] [Indexed: 11/10/2022] Open
Abstract
Human decision-making is often swayed by irrecoverable investments even though it should only be based on future-and not past-costs and benefits. Although this sunk cost effect is widely documented and can lead to devastating losses, the underlying psychological mechanisms are unclear. To tease apart possible explanations through a comparative approach, we assessed capuchin and rhesus monkeys' susceptibility to sunk costs in a psychomotor task. Monkeys needed to track a moving target with a joystick-controlled cursor for variable durations. They could stop at any time, ending the trial without reward. To minimize the work required for a reward, monkeys should have always persisted for at least 1 s, but should have abandoned the trial if that did not yield a reward. Capuchin monkeys and especially rhesus macaques persisted to trial completion even when it was suboptimal, and were more likely to complete the trial the longer they had already tracked the target. These effects were less pronounced, although still present, when the change in expected tracking duration was signalled visually. These results show that sunk cost effects can arise in the absence of human-unique factors and may emerge, in part, because persisting can resolve uncertainty.
Collapse
Affiliation(s)
- Julia Watzek
- Department of Psychology, Language Research Center, Georgia State University, Atlanta, GA, USA.
| | - Sarah F Brosnan
- Department of Psychology, Language Research Center, Georgia State University, Atlanta, GA, USA
- Department of Philosophy, Neuroscience Institute, Center for Behavioral Neuroscience, Georgia State University, Atlanta, GA, USA
| |
Collapse
|
36
|
Switching Competitors Reduces Win-Stay but Not Lose-Shift Behaviour: The Role of Outcome-Action Association Strength on Reinforcement Learning. GAMES 2020. [DOI: 10.3390/g11030025] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Predictability is a hallmark of poor-quality decision-making during competition. One source of predictability is the strong association between current outcome and future action, as dictated by the reinforcement learning principles of win–stay and lose–shift. We tested the idea that predictability could be reduced during competition by weakening the associations between outcome and action. To do this, participants completed a competitive zero-sum game in which the opponent from the current trial was either replayed (opponent repeat) thereby strengthening the association, or, replaced (opponent change) by a different competitor thereby weakening the association. We observed that win–stay behavior was reduced during opponent change trials but lose–shiftbehavior remained reliably predictable. Consistent with the group data, the number of individuals who exhibited predictable behavior following wins decreased for opponent change relative to opponent repeat trials. Our data show that future actions are more under internal control following positive relative to negative outcomes, and that externally breaking the bonds between outcome and action via opponent association also allows us to become less prone to exploitation.
Collapse
|
37
|
What Are Memories For? The Hippocampus Bridges Past Experience with Future Decisions. Trends Cogn Sci 2020; 24:542-556. [DOI: 10.1016/j.tics.2020.04.004] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2020] [Revised: 04/24/2020] [Accepted: 04/26/2020] [Indexed: 01/07/2023]
|
38
|
The Rostrolateral Prefrontal Cortex Mediates a Preference for High-Agency Environments. J Neurosci 2020; 40:4401-4409. [PMID: 32327532 DOI: 10.1523/jneurosci.2463-19.2020] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2019] [Revised: 04/02/2020] [Accepted: 04/06/2020] [Indexed: 11/21/2022] Open
Abstract
The ability to exert flexible instrumental control over one's environment is a defining feature of adaptive decision-making. Here, we investigated neural substrates mediating a preference for environments with greater instrumental divergence, the distance between outcome probability distributions associated with alternative actions. A formal index of agency, instrumental divergence allows an organism to flexibly obtain the currently most desired outcome as preferences change. As such, it may have intrinsic utility, guiding decisions toward environments that maximize instrumental power. Consistent with this notion, we found that a measure of expected value that treats instrumental divergence as a reward surrogate provided a better account of male and female human participants' choice preferences than did a conventional model, sensitive only to monetary reward. Using model-based fMRI, we found that activity in the rostrolateral and ventromedial PFC, regions associated with abstract cognitive inferences and subjective value computations, respectively, scaled with the divergence-based account of expected value. Implications for a neural common currency of information theoretic and motivational variables are discussed.SIGNIFICANCE STATEMENT Agency is a central concept in philosophy and psychology. While research thus far has focused on cognitive and perceptual measures of agency, recent work demonstrating a strong preference for high-agency environments indicates a salient motivational dimension. Here, using instrumental divergence, the distance between outcome distributions associated with alternative actions, as a formal index of agency, we found that brain regions associated with directed exploration and subjective value computations, respectively, were selectively modulated by a model that treated agency as a reward surrogate, over models that assigned utility only to monetary payoffs. In a subset of regions, such effects were predicted by the influence of instrumental divergence on economic choice preferences. Our results elucidate neural mechanisms mediating the utility of agency.
Collapse
|
39
|
Closed-Loop Theta Stimulation in the Orbitofrontal Cortex Prevents Reward-Based Learning. Neuron 2020; 106:537-547.e4. [PMID: 32160515 DOI: 10.1016/j.neuron.2020.02.003] [Citation(s) in RCA: 46] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2019] [Revised: 10/31/2019] [Accepted: 02/03/2020] [Indexed: 12/30/2022]
Abstract
Neuronal oscillations in the frontal cortex have been hypothesized to play a role in the organization of high-level cognition. Within the orbitofrontal cortex (OFC), there is a prominent oscillation in the theta frequency (4-8 Hz) during reward-guided behavior, but it is unclear whether this oscillation has causal significance. One methodological challenge is that it is difficult to manipulate theta without affecting other neural signals, such as single-neuron firing rates. A potential solution is to use closed-loop control to record theta in real time and use this signal to control the application of electrical microstimulation to the OFC. Using this method, we show that theta oscillations in the OFC are critically important for reward-guided learning and that they are driven by theta oscillations in the hippocampus (HPC). The ability to disrupt OFC computations via spatially localized and temporally precise stimulation could lead to novel treatment strategies for neuropsychiatric disorders involving OFC dysfunction.
Collapse
|
40
|
Howard JD, Reynolds R, Smith DE, Voss JL, Schoenbaum G, Kahnt T. Targeted Stimulation of Human Orbitofrontal Networks Disrupts Outcome-Guided Behavior. Curr Biol 2020; 30:490-498.e4. [PMID: 31956033 DOI: 10.1016/j.cub.2019.12.007] [Citation(s) in RCA: 44] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2019] [Revised: 11/07/2019] [Accepted: 12/03/2019] [Indexed: 01/08/2023]
Abstract
Outcome-guided behavior requires knowledge about the current value of expected outcomes. Such behavior can be isolated in the reinforcer devaluation task, which assesses the ability to infer the current value of specific rewards after devaluation. Animal lesion studies demonstrate that orbitofrontal cortex (OFC) is necessary for normal behavior in this task, but a causal role for human OFC in outcome-guided behavior has not been established. Here, we used sham-controlled, non-invasive, continuous theta-burst stimulation (cTBS) to temporarily disrupt human OFC network activity by stimulating a site in the lateral prefrontal cortex that is strongly connected to OFC prior to devaluation of food odor rewards. Subjects in the sham group appropriately avoided Pavlovian cues associated with devalued food odors. However, subjects in the stimulation group persistently chose those cues, even though devaluation of food odors themselves was unaffected by cTBS. This behavioral impairment was mirrored in changes in resting-state functional magnetic resonance imaging (rs-fMRI) activity such that subjects in the stimulation group exhibited reduced OFC network connectivity after cTBS, and the magnitude of this reduction was correlated with choices after devaluation. These findings demonstrate the feasibility of indirectly targeting the human OFC with non-invasive cTBS and indicate that OFC is specifically required for inferring the value of expected outcomes.
Collapse
Affiliation(s)
- James D Howard
- Department of Neurology, Feinberg School of Medicine, Northwestern University, 320 E. Superior Street, Searle 11-643, Chicago, IL 60611, USA.
| | - Rachel Reynolds
- Department of Neurology, Feinberg School of Medicine, Northwestern University, 320 E. Superior Street, Searle 11-643, Chicago, IL 60611, USA
| | - Devyn E Smith
- Department of Neurology, Feinberg School of Medicine, Northwestern University, 320 E. Superior Street, Searle 11-643, Chicago, IL 60611, USA
| | - Joel L Voss
- Department of Neurology, Feinberg School of Medicine, Northwestern University, 320 E. Superior Street, Searle 11-643, Chicago, IL 60611, USA; Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, 303 E. Chicago Avenue, Ward 19, Chicago, IL 60611, USA; Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, 446 E. Ontario Street, Suite 7-200, Chicago, IL 60611, USA
| | - Geoffrey Schoenbaum
- Intramural Research Program, National Institute on Drug Abuse, 251 Bayview Boulevard, Room 04A505, Suite 200, Baltimore, MD 21224, USA
| | - Thorsten Kahnt
- Department of Neurology, Feinberg School of Medicine, Northwestern University, 320 E. Superior Street, Searle 11-643, Chicago, IL 60611, USA; Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, 446 E. Ontario Street, Suite 7-200, Chicago, IL 60611, USA; Department of Psychology, Weinberg College of Arts and Sciences, Northwestern University, Swift Hall 102, 2029 Sheridan Road, Evanston, IL 60208, USA.
| |
Collapse
|
41
|
Sleezer BJ, Hayden BY. Neuroscience: Reevaluating the Role of Orbitofrontal Cortex. Curr Biol 2019; 29:R1314-R1316. [PMID: 31846680 DOI: 10.1016/j.cub.2019.10.063] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
A new optogenetic lesion study shows that the orbitofrontal cortex is essential for integrating information about recent rewards - which may either increase or decrease demand for more - with learned preferences to drive behavior.
Collapse
Affiliation(s)
- Brianna J Sleezer
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY 14853, USA
| | - Benjamin Yost Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, University of Minnesota, Minneapolis, MN 55455, USA.
| |
Collapse
|
42
|
Frontal cortex neuron types categorically encode single decision variables. Nature 2019; 576:446-451. [DOI: 10.1038/s41586-019-1816-9] [Citation(s) in RCA: 86] [Impact Index Per Article: 17.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2017] [Accepted: 10/15/2019] [Indexed: 12/11/2022]
|
43
|
Parr AC, Coe BC, Munoz DP, Dorris MC. A novel fMRI paradigm to dissociate the behavioral and neural components of mixed-strategy decision making from non-strategic decisions in humans. Eur J Neurosci 2019; 51:1914-1927. [PMID: 31596980 DOI: 10.1111/ejn.14586] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2019] [Revised: 08/22/2019] [Accepted: 09/18/2019] [Indexed: 11/30/2022]
Abstract
During competitive interactions, such as predator-prey or team sports, the outcome of one's actions is dependent on both their own choices and those of their opponents. Success in these rivalries requires that individuals choose dynamically and unpredictably, often adopting a mixed strategy. Understanding the neural basis of strategic decision making is complicated by the fact that it recruits various cognitive processes that are often shared with non-strategic forms of decision making, such as value estimation, working memory, response inhibition, response selection, and reward processes. Although researchers have explored neural activity within key brain regions during mixed-strategy games, how brain activity differs in the context of strategic interactions versus non-strategic choices is not well understood. We developed a novel behavioral paradigm to dissociate choice behavior during mixed-strategy interactions from non-strategic choices, and we used task-based functional magnetic resonance imaging (fMRI) to contrast brain activation. In a block design, participants competed in the classic mixed-strategy game, "matching pennies," against a dynamic computer opponent designed to exploit predictability in players' response patterns. Results were contrasted with a non-strategic task that had comparable sensory input, motor output, and reward rate; thus, differences in behavior and brain activation reflect strategic processes. The mixed-strategy game was associated with activation of a distributed cortico-striatal network compared to the non-strategic task. We propose that choosing in mixed-strategy contexts requires additional cognitive demands present to a lesser degree during the control task, illustrating the strength of this design in probing function of cognitive systems beyond core sensory, motor, and reward processes.
Collapse
Affiliation(s)
- Ashley C Parr
- Centre for Neuroscience Studies, Queen's University, Kingston, ON, Canada.,Department of Psychiatry, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Brian C Coe
- Centre for Neuroscience Studies, Queen's University, Kingston, ON, Canada
| | - Douglas P Munoz
- Centre for Neuroscience Studies, Queen's University, Kingston, ON, Canada.,Department of Biomedical and Molecular Sciences, Queen's University, Kingston, ON, Canada
| | - Michael C Dorris
- Institute of Neuroscience, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, Shanghai, China
| |
Collapse
|
44
|
Dyson BJ, Steward BA, Meneghetti T, Forder L. Behavioural and neural limits in competitive decision making: The roles of outcome, opponency and observation. Biol Psychol 2019; 149:107778. [PMID: 31593749 DOI: 10.1016/j.biopsycho.2019.107778] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2019] [Revised: 07/31/2019] [Accepted: 09/24/2019] [Indexed: 11/25/2022]
Abstract
To understand the boundaries we set for ourselves in terms of environmental responsibility during competition, we examined a neural index of outcome valence (feedback-related negativity; FRN) in relation to an early index of visual attention (N1), a later index of motivational significance (P3), and, eventual behaviour. In Experiment 1 (n = 36), participants either were (play) or were not (observe) responsible for action selection. In Experiment 2 (n = 36), opponents additionally either could (exploitable) or could not (unexploitable) be beaten. Various failures in reinforcement learning expression were revealed including large-scale approximations of random behaviour. Against unexploitable opponents, N1 determined the extent to which negative and positive outcomes were perceived as distinct categories by FRN. Against exploitable opponents, FRN determined the extent to which P3 generated neural gain for future events. Differential activation of the N1 - FRN - P3 processing chain provides a framework for understanding the behavioural dynamism observed during competitive decision making.
Collapse
Affiliation(s)
- Benjamin James Dyson
- University of Alberta, Canada; University of Sussex, UK; Ryerson University, Canada.
| | | | | | | |
Collapse
|
45
|
Behavioural Isomorphism, Cognitive Economy and Recursive Thought in Non-Transitive Game Strategy. GAMES 2019. [DOI: 10.3390/g10030032] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Game spaces in which an organism must repeatedly compete with an opponent for mutually exclusive outcomes are critical methodologies for understanding decision-making under pressure. In the non-transitive game rock, paper, scissors (RPS), the only technique that guarantees the lack of exploitation is to perform randomly in accordance with mixed-strategy. However, such behavior is thought to be outside bounded rationality and so decision-making can become deterministic, predictable, and ultimately exploitable. This review identifies similarities across economics, neuroscience, nonlinear dynamics, human, and animal cognition literatures, and provides a taxonomy of RPS strategy. RPS strategies are discussed in terms of (a) whether the relevant computations require sensitivity to item frequency, the cyclic relationships between responses, or the outcome of the previous trial, and (b) whether the strategy is framed around the self or other. The negative implication of this taxonomy is that despite the differences in cognitive economy and recursive thought, many of the identified strategies are behaviorally isomorphic. This makes it difficult to infer strategy from behavior. The positive implication is that this isomorphism can be used as a novel design feature in furthering our understanding of the attribution, agency, and acquisition of strategy in RPS and other game spaces.
Collapse
|
46
|
Jung K, Jeong J, Kralik JD. A Computational Model of Attention Control in Multi-Attribute, Context-Dependent Decision Making. Front Comput Neurosci 2019; 13:40. [PMID: 31354461 PMCID: PMC6635580 DOI: 10.3389/fncom.2019.00040] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2019] [Accepted: 06/11/2019] [Indexed: 11/17/2022] Open
Abstract
Real-life decisions often require a comparison of multi-attribute options with various benefits and costs, and the evaluation of each option depends partly on the others in the choice set (i.e., the choice context). Although reinforcement learning models have successfully described choice behavior, how to account for multi-attribute information when making a context-dependent decision remains unclear. Here we develop a computational model of attention control that includes context effects on multi-attribute decisions, linking a context-dependent choice model with a reinforcement learning model. The overall model suggests that the distinctiveness of attributes guides an individual's preferences among multi-attribute options via an attention-control mechanism that determines whether choices are selectively biased toward the most distinctive attribute (selective attention) or proportionally distributed based on the relative distinctiveness of attributes (divided attention). To test the model, we conducted a behavioral experiment in rhesus monkeys, in which they made simple multi-attribute decisions over three conditions that manipulated the degree of distinctiveness between alternatives: (1) four foods of different size and calorie; (2) four pieces of the same food in different colors; and (3) four identical pieces of food. The model simulation of the choice behavior captured the preference bias (i.e., overall preference structure) and the choice persistence (repeated choices) in the empirical data, providing evidence for the respective influences of attention and memory on preference bias and choice persistence. Our study provides insights into computations underlying multi-attribute decisions, linking attentional control to decision-making processes.
Collapse
Affiliation(s)
- Kanghoon Jung
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, United States
| | - Jaeseung Jeong
- Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea
| | - Jerald D Kralik
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, United States
| |
Collapse
|
47
|
Groman SM, Keistler C, Keip AJ, Hammarlund E, DiLeone RJ, Pittenger C, Lee D, Taylor JR. Orbitofrontal Circuits Control Multiple Reinforcement-Learning Processes. Neuron 2019; 103:734-746.e3. [PMID: 31253468 DOI: 10.1016/j.neuron.2019.05.042] [Citation(s) in RCA: 80] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2019] [Revised: 04/18/2019] [Accepted: 05/24/2019] [Indexed: 12/18/2022]
Abstract
Adaptive decision making in dynamic environments requires multiple reinforcement-learning steps that may be implemented by dissociable neural circuits. Here, we used a novel directionally specific viral ablation approach to investigate the function of several anatomically defined orbitofrontal cortex (OFC) circuits during adaptive, flexible decision making in rats trained on a probabilistic reversal learning task. Ablation of OFC neurons projecting to the nucleus accumbens selectively disrupted performance following a reversal, by disrupting the use of negative outcomes to guide subsequent choices. Ablation of amygdala neurons projecting to the OFC also impaired reversal performance, but due to disruptions in the use of positive outcomes to guide subsequent choices. Ablation of OFC neurons projecting to the amygdala, by contrast, enhanced reversal performance by destabilizing action values. Our data are inconsistent with a unitary function of the OFC in decision making. Rather, distinct OFC-amygdala-striatal circuits mediate distinct components of the action-value updating and maintenance necessary for decision making.
Collapse
Affiliation(s)
| | - Colby Keistler
- Department of Psychiatry, Yale University, New Haven, CT 06515, USA
| | - Alex J Keip
- Department of Psychiatry, Yale University, New Haven, CT 06515, USA
| | - Emma Hammarlund
- Department of Psychiatry, Yale University, New Haven, CT 06515, USA
| | - Ralph J DiLeone
- Department of Psychiatry, Yale University, New Haven, CT 06515, USA; Department of Neuroscience, Yale University, New Haven, CT 06515, USA
| | - Christopher Pittenger
- Department of Psychiatry, Yale University, New Haven, CT 06515, USA; Child Study Center, Yale University, New Haven, CT 06515, USA
| | - Daeyeol Lee
- Department of Psychiatry, Yale University, New Haven, CT 06515, USA; Department of Neuroscience, Yale University, New Haven, CT 06515, USA; Department of Psychology, Yale University, New Haven, CT 06515, USA
| | - Jane R Taylor
- Department of Psychiatry, Yale University, New Haven, CT 06515, USA; Department of Neuroscience, Yale University, New Haven, CT 06515, USA; Department of Psychology, Yale University, New Haven, CT 06515, USA.
| |
Collapse
|
48
|
Bault N, di Pellegrino G, Puppi M, Opolczynski G, Monti A, Braghittoni D, Thibaut F, Rustichini A, Coricelli G. Dissociation between Private and Social Counterfactual Value Signals Following Ventromedial Prefrontal Cortex Damage. J Cogn Neurosci 2019; 31:639-656. [DOI: 10.1162/jocn_a_01372] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Individuals learn by comparing the outcome of chosen and unchosen actions. A negative counterfactual value signal is generated when this comparison is unfavorable. This can happen in private as well as in social settings—where the foregone outcome results from the choice of another person. We hypothesized that, despite sharing similar features such as supporting learning, these two counterfactual signals might implicate distinct brain networks. We conducted a neuropsychological study on the role of private and social counterfactual value signals in risky decision-making. Patients with lesions in the ventromedial prefrontal cortex (vmPFC), lesion controls, and healthy controls repeatedly chose between lotteries. In private trials, participants could observe the outcomes of their choices and the outcomes of the unselected lotteries. In social trials, participants could also see the other player's choices and outcome. At the time of outcome, vmPFC patients were insensitive to private counterfactual value signals, whereas their responses to social comparison were similar to those of control participants. At the time of choice, intact vmPFC was necessary to integrate counterfactual signals in decisions, although amelioration was observed during the course of the task, possibly driven by social trials. We conclude that if the vmPFC is critical in processing private counterfactual signals and in integrating those signals in decision-making, then distinct brain areas might support the processing of social counterfactual signals.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | - Giorgio Coricelli
- University of Trento, Mattarello, Italy
- University of Southern California
| |
Collapse
|
49
|
Fouragnan EF, Chau BKH, Folloni D, Kolling N, Verhagen L, Klein-Flügge M, Tankelevitch L, Papageorgiou GK, Aubry JF, Sallet J, Rushworth MFS. The macaque anterior cingulate cortex translates counterfactual choice value into actual behavioral change. Nat Neurosci 2019; 22:797-808. [PMID: 30988525 PMCID: PMC7116825 DOI: 10.1038/s41593-019-0375-6] [Citation(s) in RCA: 109] [Impact Index Per Article: 21.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2018] [Accepted: 03/06/2019] [Indexed: 12/17/2022]
Abstract
The neural mechanisms mediating sensory-guided decision making have received considerable attention but animals often pursue behaviors for which there is currently no sensory evidence. Such behaviors are guided by internal representations of choice values that have to be maintained even when these choices are unavailable. We investigated how four macaque monkeys maintained representations of the value of counterfactual choices– choices that could not be taken at the current moment but which could be taken in the future. Using functional magnetic resonance imaging, we found two different patterns of activity co-varying with values of counterfactual choices in a circuit spanning hippocampus, anterior lateral prefrontal cortex, and anterior cingulate cortex (ACC). ACC activity also reflected whether the internal value representations would be translated into actual behavioral change. To establish the causal importance of ACC for this translation process, we used a novel technique, Transcranial Focused Ultrasound Stimulation, to reversibly disrupt ACC activity.
Collapse
Affiliation(s)
- Elsa F Fouragnan
- School of Psychology, University of Plymouth, Plymouth, UK. .,Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Bolton K H Chau
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK.,Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, China
| | - Davide Folloni
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Nils Kolling
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Lennart Verhagen
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Miriam Klein-Flügge
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Lev Tankelevitch
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Georgios K Papageorgiou
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK.,McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Jean-Francois Aubry
- Physics for Medicine Paris, Inserm, ESPCI Paris, CNRS, PSL Research University, Paris, France
| | - Jerome Sallet
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Matthew F S Rushworth
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK.,Wellcome Integrative Neuroimaging, Centre for Functional MRI of the Brain, Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
| |
Collapse
|
50
|
Beyond "incentive hope": Information sampling and learning under reward uncertainty. Behav Brain Sci 2019; 42:e56. [PMID: 30940240 DOI: 10.1017/s0140525x18001930] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Information seeking, especially when motivated by strategic learning and intrinsic curiosity, could render the new mechanism "incentive hope" proposed by Anselme & Güntürkün sufficient, but not necessary to explain how reward uncertainty promotes reward seeking and consumption. Naturalistic and foraging-like tasks can help parse motivational processes that bridge learning and foraging behaviors and identify their neural underpinnings.
Collapse
|