1
|
Anlló H, Bavard S, Benmarrakchi F, Bonagura D, Cerrotti F, Cicue M, Gueguen M, Guzmán EJ, Kadieva D, Kobayashi M, Lukumon G, Sartorio M, Yang J, Zinchenko O, Bahrami B, Silva Concha J, Hertz U, Konova AB, Li J, O'Madagain C, Navajas J, Reyes G, Sarabi-Jamab A, Shestakova A, Sukumaran B, Watanabe K, Palminteri S. Comparing experience- and description-based economic preferences across 11 countries. Nat Hum Behav 2024:10.1038/s41562-024-01894-9. [PMID: 38877287 DOI: 10.1038/s41562-024-01894-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Accepted: 04/19/2024] [Indexed: 06/16/2024]
Abstract
Recent evidence indicates that reward value encoding in humans is highly context dependent, leading to suboptimal decisions in some cases, but whether this computational constraint on valuation is a shared feature of human cognition remains unknown. Here we studied the behaviour of n = 561 individuals from 11 countries of markedly different socioeconomic and cultural makeup. Our findings show that context sensitivity was present in all 11 countries. Suboptimal decisions generated by context manipulation were not explained by risk aversion, as estimated through a separate description-based choice task (that is, lotteries) consisting of matched decision offers. Conversely, risk aversion significantly differed across countries. Overall, our findings suggest that context-dependent reward value encoding is a feature of human cognition that remains consistently present across different countries, as opposed to description-based decision-making, which is more permeable to cultural factors.
Collapse
Affiliation(s)
- Hernán Anlló
- Human Reinforcement Learning Team, Laboratory of Cognitive and Computational Neuroscience, Paris, France.
- Faculty of Science and Engineering, Waseda University, Tokyo, Japan.
- Intercultural Cognitive Network, Paris, France.
| | - Sophie Bavard
- Human Reinforcement Learning Team, Laboratory of Cognitive and Computational Neuroscience, Paris, France
- Intercultural Cognitive Network, Paris, France
- General Psychology Lab, Hamburg University, Hamburg, Germany
| | - FatimaEzzahra Benmarrakchi
- Intercultural Cognitive Network, Paris, France
- School of Collective Intelligence, Université Mohammed VI Polytechnique, Rabat, Morocco
| | - Darla Bonagura
- Intercultural Cognitive Network, Paris, France
- Department of Psychiatry, University Behavioral Health Care and Brain Health Institute, Rutgers University-New Brunswick, Piscataway, NJ, USA
| | - Fabien Cerrotti
- Human Reinforcement Learning Team, Laboratory of Cognitive and Computational Neuroscience, Paris, France
- Intercultural Cognitive Network, Paris, France
| | - Mirona Cicue
- Department of Cognitive Sciences, University of Haifa, Haifa, Israel
| | - Maelle Gueguen
- Intercultural Cognitive Network, Paris, France
- Department of Psychiatry, University Behavioral Health Care and Brain Health Institute, Rutgers University-New Brunswick, Piscataway, NJ, USA
| | - Eugenio José Guzmán
- Facultad de Psicología, Universidad del Desarrollo, Santiago de Chile, Chile
| | - Dzerassa Kadieva
- International Laboratory for Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Maiko Kobayashi
- Faculty of Science and Engineering, Waseda University, Tokyo, Japan
| | - Gafari Lukumon
- School of Collective Intelligence, Université Mohammed VI Polytechnique, Rabat, Morocco
| | - Marco Sartorio
- Laboratorio de Neurociencia, Universidad Torcuato Di Tella, Buenos Aires, Argentina
| | - Jiong Yang
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
| | - Oksana Zinchenko
- Intercultural Cognitive Network, Paris, France
- Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Bahador Bahrami
- Intercultural Cognitive Network, Paris, France
- Department of Psychology, Ludwig Maximilian University, Munich, Germany
| | - Jaime Silva Concha
- Intercultural Cognitive Network, Paris, France
- Facultad de Psicología, Universidad del Desarrollo, Santiago de Chile, Chile
| | - Uri Hertz
- Intercultural Cognitive Network, Paris, France
- Department of Cognitive Sciences, University of Haifa, Haifa, Israel
| | - Anna B Konova
- Intercultural Cognitive Network, Paris, France
- Department of Psychiatry, University Behavioral Health Care and Brain Health Institute, Rutgers University-New Brunswick, Piscataway, NJ, USA
| | - Jian Li
- Intercultural Cognitive Network, Paris, France
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
- IDG/McGovern Institute for Brain Research, Peking University, Beijing, China
| | - Cathal O'Madagain
- Intercultural Cognitive Network, Paris, France
- School of Collective Intelligence, Université Mohammed VI Polytechnique, Rabat, Morocco
| | - Joaquin Navajas
- Intercultural Cognitive Network, Paris, France
- Laboratorio de Neurociencia, Universidad Torcuato Di Tella, Buenos Aires, Argentina
- Escuela de Negocios, Universidad Torcuato Di Tella, Buenos Aires, Argentina
- Consejo Nacional de Investigaciones Científicas y Técnicas, Buenos Aires, Argentina
| | - Gabriel Reyes
- Intercultural Cognitive Network, Paris, France
- Facultad de Psicología, Universidad del Desarrollo, Santiago de Chile, Chile
| | - Atiye Sarabi-Jamab
- Intercultural Cognitive Network, Paris, France
- School of Cognitive Sciences, Institute for Research in Fundamental Sciences, Tehran, Iran
| | - Anna Shestakova
- Intercultural Cognitive Network, Paris, France
- Centre for Cognition and Decision Making, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Bhasi Sukumaran
- Intercultural Cognitive Network, Paris, France
- Department of Clinical Psychology, SRM Medical College Hospital and Research Centre, Chennai, India
| | - Katsumi Watanabe
- Faculty of Science and Engineering, Waseda University, Tokyo, Japan
- Intercultural Cognitive Network, Paris, France
| | - Stefano Palminteri
- Human Reinforcement Learning Team, Laboratory of Cognitive and Computational Neuroscience, Paris, France.
- Intercultural Cognitive Network, Paris, France.
- Departement d'études cognitives, Ecole normale supérieure, PSL Research University, Paris, France.
| |
Collapse
|
2
|
Olschewski S, Luckman A, Mason A, Ludvig EA, Konstantinidis E. The Future of Decisions From Experience: Connecting Real-World Decision Problems to Cognitive Processes. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2024; 19:82-102. [PMID: 37390328 PMCID: PMC10790535 DOI: 10.1177/17456916231179138] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/02/2023]
Abstract
In many important real-world decision domains, such as finance, the environment, and health, behavior is strongly influenced by experience. Renewed interest in studying this influence led to important advancements in the understanding of these decisions from experience (DfE) in the last 20 years. Building on this literature, we suggest ways the standard experimental design should be extended to better approach important real-world DfE. These extensions include, for example, introducing more complex choice situations, delaying feedback, and including social interactions. When acting upon experiences in these richer and more complicated environments, extensive cognitive processes go into making a decision. Therefore, we argue for integrating cognitive processes more explicitly into experimental research in DfE. These cognitive processes include attention to and perception of numeric and nonnumeric experiences, the influence of episodic and semantic memory, and the mental models involved in learning processes. Understanding these basic cognitive processes can advance the modeling, understanding and prediction of DfE in the laboratory and in the real world. We highlight the potential of experimental research in DfE for theory integration across the behavioral, decision, and cognitive sciences. Furthermore, this research could lead to new methodology that better informs decision-making and policy interventions.
Collapse
Affiliation(s)
- Sebastian Olschewski
- Department of Psychology, University of Basel
- Warwick Business School, University of Warwick
| | - Ashley Luckman
- Warwick Business School, University of Warwick
- University of Exeter Business School, University of Exeter
| | - Alice Mason
- Department of Psychology, University of Bath
- Department of Psychology, University of Warwick
| | | | | |
Collapse
|
3
|
Hayes WM, Wedell DH. Effects of blocked versus interleaved training on relative value learning. Psychon Bull Rev 2023; 30:1895-1907. [PMID: 37072667 DOI: 10.3758/s13423-023-02290-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/31/2023] [Indexed: 04/20/2023]
Abstract
In reinforcement learning tasks, people learn the values of options relative to other options in the local context. Prior research suggests that relative value learning is enhanced when choice contexts are temporally clustered in a blocked sequence compared to a randomly interleaved sequence. The present study was aimed at further investigating the effects of blocked versus interleaved training using a choice task that distinguishes among different contextual encoding models. Our results showed that the presentation format in which contexts are experienced can lead to qualitatively distinct forms of relative value learning. This conclusion was supported by a combination of model-free and model-based analyses. In the blocked condition, choice behavior was most consistent with a reference point model in which outcomes are encoded relative to a dynamic estimate of the contextual average reward. In contrast, the interleaved condition was best described by a range-frequency encoding model. We propose that blocked training makes it easier to track contextual outcome statistics, such as the average reward, which may then be used to relativize the values of experienced outcomes. When contexts are interleaved, range-frequency encoding may serve as a more efficient means of storing option values in memory for later retrieval.
Collapse
Affiliation(s)
- William M Hayes
- Department of Psychology, University of South Carolina, 1512 Pendleton St, Columbia, SC, 29208, USA.
| | - Douglas H Wedell
- Department of Psychology, University of South Carolina, 1512 Pendleton St, Columbia, SC, 29208, USA
| |
Collapse
|
4
|
Molinaro G, Collins AGE. Intrinsic rewards explain context-sensitive valuation in reinforcement learning. PLoS Biol 2023; 21:e3002201. [PMID: 37459394 PMCID: PMC10374061 DOI: 10.1371/journal.pbio.3002201] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2022] [Revised: 07/27/2023] [Accepted: 06/15/2023] [Indexed: 07/28/2023] Open
Abstract
When observing the outcome of a choice, people are sensitive to the choice's context, such that the experienced value of an option depends on the alternatives: getting $1 when the possibilities were 0 or 1 feels much better than when the possibilities were 1 or 10. Context-sensitive valuation has been documented within reinforcement learning (RL) tasks, in which values are learned from experience through trial and error. Range adaptation, wherein options are rescaled according to the range of values yielded by available options, has been proposed to account for this phenomenon. However, we propose that other mechanisms-reflecting a different theoretical viewpoint-may also explain this phenomenon. Specifically, we theorize that internally defined goals play a crucial role in shaping the subjective value attributed to any given option. Motivated by this theory, we develop a new "intrinsically enhanced" RL model, which combines extrinsically provided rewards with internally generated signals of goal achievement as a teaching signal. Across 7 different studies (including previously published data sets as well as a novel, preregistered experiment with replication and control studies), we show that the intrinsically enhanced model can explain context-sensitive valuation as well as, or better than, range adaptation. Our findings indicate a more prominent role of intrinsic, goal-dependent rewards than previously recognized within formal models of human RL. By integrating internally generated signals of reward, standard RL theories should better account for human behavior, including context-sensitive valuation and beyond.
Collapse
Affiliation(s)
- Gaia Molinaro
- Department of Psychology, University of California, Berkeley, Berkeley, California, United States of America
| | - Anne G E Collins
- Department of Psychology, University of California, Berkeley, Berkeley, California, United States of America
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California, United States of America
| |
Collapse
|