1
|
Dubinsky JM, Hamid AA. The neuroscience of active learning and direct instruction. Neurosci Biobehav Rev 2024; 163:105737. [PMID: 38796122 DOI: 10.1016/j.neubiorev.2024.105737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 05/13/2024] [Accepted: 05/20/2024] [Indexed: 05/28/2024]
Abstract
Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.
Collapse
Affiliation(s)
- Janet M Dubinsky
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA.
| | - Arif A Hamid
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
2
|
Kobayashi K, Kable JW. Neural mechanisms of information seeking. Neuron 2024; 112:1741-1756. [PMID: 38703774 DOI: 10.1016/j.neuron.2024.04.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Revised: 01/30/2024] [Accepted: 04/08/2024] [Indexed: 05/06/2024]
Abstract
We ubiquitously seek information to make better decisions. Particularly in the modern age, when more information is available at our fingertips than ever, the information we choose to collect determines the quality of our decisions. Decision neuroscience has long adopted empirical approaches where the information available to decision-makers is fully controlled by the researchers, leaving neural mechanisms of information seeking less understood. Although information seeking has long been studied in the context of the exploration-exploitation trade-off, recent studies have widened the scope to investigate more overt information seeking in a way distinct from other decision processes. Insights gained from these studies, accumulated over the last few years, raise the possibility that information seeking is driven by the reward system signaling the subjective value of information. In this piece, we review findings from the recent studies, highlighting the conceptual and empirical relationships between distinct literatures, and discuss future research directions necessary to establish a more comprehensive understanding of how individuals seek information as a part of value-based decision-making.
Collapse
Affiliation(s)
- Kenji Kobayashi
- Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA.
| | - Joseph W Kable
- Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA.
| |
Collapse
|
3
|
Zhou D, Bornstein AM. Expanding horizons in reinforcement learning for curious exploration and creative planning. Behav Brain Sci 2024; 47:e118. [PMID: 38770877 DOI: 10.1017/s0140525x23003394] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/22/2024]
Abstract
Curiosity and creativity are expressions of the trade-off between leveraging that with which we are familiar or seeking out novelty. Through the computational lens of reinforcement learning, we describe how formulating the value of information seeking and generation via their complementary effects on planning horizons formally captures a range of solutions to striking this balance.
Collapse
Affiliation(s)
- Dale Zhou
- Neurobiology and Behavior, 519 Biological Sciences Quad, University of California, Irvine, CA, USA ://dalezhou.com
- Center for the Neurobiology of Learning and Memory, Qureshey, Research Laboratory, University of California, Irvine, CA, USA ://aaron.bornstein.org/
| | - Aaron M Bornstein
- Center for the Neurobiology of Learning and Memory, Qureshey, Research Laboratory, University of California, Irvine, CA, USA ://aaron.bornstein.org/
- Department of Cognitive Sciences, 2318 Social & Behavioral Sciences Gateway, University of California, Irvine, CA, USA
| |
Collapse
|
4
|
Litovsky Y, Horn S, Olivola CY. Curiosity is more than novelty seeking. Behav Brain Sci 2024; 47:e107. [PMID: 38770854 DOI: 10.1017/s0140525x23003278] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/22/2024]
Abstract
The novelty-seeking model (NSM) does not offer a compelling unifying framework for understanding creativity and curiosity. It fails to explain important manifestations and features of curiosity. Moreover, the arguments offered to support a curiosity-creativity link - a shared association with a common core process and various superficial associations between them - are neither convincing nor do they yield useful predictions.
Collapse
Affiliation(s)
- Yana Litovsky
- Department of Banking and Finance, University of Innsbruck/Universität Innsbruck, Innsbruck, Austria ://www.yanalitovsky.com/
| | - Samantha Horn
- Department of Economics, University of Chicago, Chicago, IL, USA ://www.samihorn.com/
| | - Christopher Y Olivola
- Tepper School of Business, Carnegie Mellon University, Pittsburgh, PA, USA ://sites.google.com/site/chrisolivola/
- Department of Social and Decision Sciences, Carnegie Mellon University, Pittsburgh, PA, USA
| |
Collapse
|
5
|
Klaassen FH, de Voogd LD, Hulsman AM, O'Reilly JX, Klumpers F, Figner B, Roelofs K. The neurocomputational link between defensive cardiac states and approach-avoidance arbitration under threat. Commun Biol 2024; 7:576. [PMID: 38755409 PMCID: PMC11099143 DOI: 10.1038/s42003-024-06267-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2023] [Accepted: 04/30/2024] [Indexed: 05/18/2024] Open
Abstract
Avoidance, a hallmark of anxiety-related psychopathology, often comes at a cost; avoiding threat may forgo the possibility of a reward. Theories predict that optimal approach-avoidance arbitration depends on threat-induced psychophysiological states, like freezing-related bradycardia. Here we used model-based fMRI analyses to investigate whether and how bradycardia states are linked to the neurocomputational underpinnings of approach-avoidance arbitration under varying reward and threat magnitudes. We show that bradycardia states are associated with increased threat-induced avoidance and more pronounced reward-threat value comparison (i.e., a stronger tendency to approach vs. avoid when expected reward outweighs threat). An amygdala-striatal-prefrontal circuit supports approach-avoidance arbitration under threat, with specific involvement of the amygdala and dorsal anterior cingulate (dACC) in integrating reward-threat value and bradycardia states. These findings highlight the role of human freezing states in value-based decision making, relevant for optimal threat coping. They point to a specific role for amygdala/dACC in state-value integration under threat.
Collapse
Affiliation(s)
- Felix H Klaassen
- Radboud University, Donders Institute for Brain, Cognition, and Behaviour, Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands.
| | - Lycia D de Voogd
- Radboud University, Donders Institute for Brain, Cognition, and Behaviour, Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands
- Radboud University, Behavioural Science Institute (BSI), Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands
- Leiden University, Institute of Psychology and Leiden Institute for Brain and Cognition (LIBC), Rapenburg 70, 2311 EZ, Leiden, The Netherlands
| | - Anneloes M Hulsman
- Radboud University, Donders Institute for Brain, Cognition, and Behaviour, Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands
- Radboud University, Behavioural Science Institute (BSI), Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands
| | - Jill X O'Reilly
- Department of Experimental Psychology, University of Oxford, Woodstock Road, OX2 6GG, Oxford, UK
| | - Floris Klumpers
- Radboud University, Donders Institute for Brain, Cognition, and Behaviour, Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands
- Radboud University, Behavioural Science Institute (BSI), Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands
| | - Bernd Figner
- Radboud University, Donders Institute for Brain, Cognition, and Behaviour, Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands
- Radboud University, Behavioural Science Institute (BSI), Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands
| | - Karin Roelofs
- Radboud University, Donders Institute for Brain, Cognition, and Behaviour, Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands.
- Radboud University, Behavioural Science Institute (BSI), Thomas van Aquinostraat 4, 6525 GD, Nijmegen, The Netherlands.
| |
Collapse
|
6
|
Anderson BA. Trichotomy revisited: A monolithic theory of attentional control. Vision Res 2024; 217:108366. [PMID: 38387262 DOI: 10.1016/j.visres.2024.108366] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Revised: 02/05/2024] [Accepted: 02/09/2024] [Indexed: 02/24/2024]
Abstract
The control of attention was long held to reflect the influence of two competing mechanisms of assigning priority, one goal-directed and the other stimulus-driven. Learning-dependent influences on the control of attention that could not be attributed to either of those two established mechanisms of control gave rise to the concept of selection history and a corresponding third mechanism of attentional control. The trichotomy framework that ensued has come to dominate theories of attentional control over the past decade, replacing the historical dichotomy. In this theoretical review, I readily affirm that distinctions between the influence of goals, salience, and selection history are substantive and meaningful, and that abandoning the dichotomy between goal-directed and stimulus-driven mechanisms of control was appropriate. I do, however, question whether a theoretical trichotomy is the right answer to the problem posed by selection history. If we reframe the influence of goals and selection history as different flavors of memory-dependent modulations of attentional priority and if we characterize the influence of salience as a consequence of insufficient competition from such memory-dependent sources of priority, it is possible to account for a wide range of attention-related phenomena with only one mechanism of control. The monolithic framework for the control of attention that I propose offers several concrete advantages over a trichotomy framework, which I explore here.
Collapse
Affiliation(s)
- Brian A Anderson
- Texas A&M University, Department of Psychological & Brain Sciences, 4235 TAMU, College Station, TX 77843-4235, United States.
| |
Collapse
|
7
|
Algermissen J, Swart JC, Scheeringa R, Cools R, den Ouden HEM. Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases. Nat Commun 2024; 15:19. [PMID: 38168089 PMCID: PMC10762147 DOI: 10.1038/s41467-023-44632-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 12/22/2023] [Indexed: 01/05/2024] Open
Abstract
Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.
Collapse
Affiliation(s)
- Johannes Algermissen
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| | - Jennifer C Swart
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
| | - René Scheeringa
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Erwin L. Hahn Institute for Magnetic Resonance Imaging, University of Duisburg-Essen, Essen, Germany
| | - Roshan Cools
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands
- Department of Psychiatry, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Hanneke E M den Ouden
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
| |
Collapse
|
8
|
Hadeiy SK, Habtemariam S, Shankayi Z, Shahyad S, Sahraei H, Asghardoust Rezaei M, Bahrami F. Amelioration of pain and anxiety in sleep-deprived rats by intra-amygdala injection of cinnamaldehyde. Sleep Med X 2023. [DOI: 10.1016/j.sleepx.2023.100069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 04/03/2023] Open
|
9
|
Modirshanechi A, Kondrakiewicz K, Gerstner W, Haesler S. Curiosity-driven exploration: foundations in neuroscience and computational modeling. Trends Neurosci 2023; 46:1054-1066. [PMID: 37925342 DOI: 10.1016/j.tins.2023.10.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Revised: 09/28/2023] [Accepted: 10/04/2023] [Indexed: 11/06/2023]
Abstract
Curiosity refers to the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so. Thus far, no single, widely accepted definition or framework for curiosity has emerged, but there is growing consensus that curious behavior is not goal-directed but related to seeking or reacting to information. In this review, we take a phenomenological approach and group behavioral and neurophysiological studies which meet these criteria into three categories according to the type of information seeking observed. We then review recent computational models of curiosity from the field of machine learning and discuss how they enable integrating different types of information seeking into one theoretical framework. Combinations of behavioral and neurophysiological studies along with computational modeling will be instrumental in demystifying the notion of curiosity.
Collapse
Affiliation(s)
| | - Kacper Kondrakiewicz
- Neuroelectronics Research Flanders (NERF), Leuven, Belgium; VIB, Leuven, Belgium; Department of Neuroscience, KU Leuven, Leuven, Belgium
| | - Wulfram Gerstner
- École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland.
| | - Sebastian Haesler
- Neuroelectronics Research Flanders (NERF), Leuven, Belgium; VIB, Leuven, Belgium; Department of Neuroscience, KU Leuven, Leuven, Belgium; Leuven Brain Institute, Leuven, Belgium.
| |
Collapse
|
10
|
Nussenbaum K, Martin RE, Maulhardt S, Yang Y(J, Bizzell-Hatcher G, Bhatt NS, Koenig M, Rosenbaum GM, O'Doherty JP, Cockburn J, Hartley CA. Novelty and uncertainty differentially drive exploration across development. eLife 2023; 12:e84260. [PMID: 37585251 PMCID: PMC10431916 DOI: 10.7554/elife.84260] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Accepted: 08/07/2023] [Indexed: 08/17/2023] Open
Abstract
Across the lifespan, individuals frequently choose between exploiting known rewarding options or exploring unknown alternatives. A large body of work has suggested that children may explore more than adults. However, because novelty and reward uncertainty are often correlated, it is unclear how they differentially influence decision-making across development. Here, children, adolescents, and adults (ages 8-27 years, N = 122) completed an adapted version of a recently developed value-guided decision-making task that decouples novelty and uncertainty. In line with prior studies, we found that exploration decreased with increasing age. Critically, participants of all ages demonstrated a similar bias to select choice options with greater novelty, whereas aversion to reward uncertainty increased into adulthood. Computational modeling of participant choices revealed that whereas adolescents and adults demonstrated attenuated uncertainty aversion for more novel choice options, children's choices were not influenced by reward uncertainty.
Collapse
Affiliation(s)
| | | | - Sean Maulhardt
- New York UniversityNew YorkUnited States
- University of MarylandCollege ParkUnited States
| | - Yi (Jen) Yang
- New York UniversityNew YorkUnited States
- Temple UniversityPhiladelphiaUnited States
| | | | | | - Maximilian Koenig
- New York UniversityNew YorkUnited States
- Leiden UniversityLeidenNetherlands
| | - Gail M Rosenbaum
- New York UniversityNew YorkUnited States
- Geisinger Health SystemDanvilleUnited States
| | | | | | | |
Collapse
|
11
|
Philip VM, He H, Saul MC, Dickson PE, Bubier JA, Chesler EJ. Gene expression genetics of the striatum of Diversity Outbred mice. Sci Data 2023; 10:522. [PMID: 37543624 PMCID: PMC10404230 DOI: 10.1038/s41597-023-02426-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Accepted: 07/28/2023] [Indexed: 08/07/2023] Open
Abstract
Brain transcriptional variation is a heritable trait that mediates complex behaviors, including addiction. Expression quantitative trait locus (eQTL) mapping reveals genomic regions harboring genetic variants that influence transcript abundance. In this study, we profiled transcript abundance in the striatum of 386 Diversity Outbred (J:DO) mice of both sexes using RNA-Seq. All mice were characterized using a behavioral battery of widely-used exploratory and risk-taking assays prior to transcriptional profiling. We performed eQTL mapping, incorporated the results into a browser-based eQTL viewer, and deposited co-expression network members in GeneWeaver. The eQTL viewer allows researchers to query specific genes to obtain allelic effect plots, analyze SNP associations, assess gene expression correlations, and apply mediation analysis to evaluate whether the regulatory variant is acting through the expression of another gene. GeneWeaver allows multi-species comparison of gene sets using statistical and combinatorial tools. This data resource allows users to find genetic variants that regulate differentially expressed transcripts and place them in the context of other studies of striatal gene expression and function in addiction-related behavior.
Collapse
Affiliation(s)
- Vivek M Philip
- The Jackson Laboratory for Mammalian Genetics, Bar Harbor, ME, 04605, USA
| | - Hao He
- The Jackson Laboratory for Genomic Medicine, Farmington, CT, 06032, USA
| | - Michael C Saul
- The Jackson Laboratory for Mammalian Genetics, Bar Harbor, ME, 04605, USA
| | - Price E Dickson
- Department of Biomedical Sciences, Joan C. Edwards School of Medicine Marshall University, Huntington, WV, 25703, USA
| | - Jason A Bubier
- The Jackson Laboratory for Mammalian Genetics, Bar Harbor, ME, 04605, USA
| | - Elissa J Chesler
- The Jackson Laboratory for Mammalian Genetics, Bar Harbor, ME, 04605, USA.
| |
Collapse
|
12
|
Zhuang W, Niebaum J, Munakata Y. Changes in adaptation to time horizons across development. Dev Psychol 2023; 59:1532-1542. [PMID: 37166865 PMCID: PMC10524449 DOI: 10.1037/dev0001529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
When making decisions, the amount of time remaining matters. When time horizons are long, exploring unknown options can inform later decisions, but when time horizons are short, exploiting known options should be prioritized. While adults and adolescents adapt their exploration in this way, it is unclear when such adaptation emerges and how individuals behave when time horizons are ambiguous, as in many real-life situations. We examined these questions by having 5- to 6-year-olds (N = 43), 11- to 12-year-olds (N = 40), and adult college students (N = 49) in the United States complete a Simplified Horizons Task under short, long, and ambiguous time horizons. Adaptation to time horizons increased with age: older children and adults explored more when horizons were long than when short, and while some younger children adapted to time horizons, younger children overall did not show strong evidence of adapting. Under ambiguous horizons, older children and adults preferred to exploit over explore, while younger children did not show this preference. Thus, adaptation to time horizons is evident by ages 11-12 and may begin to emerge around 5-6 years, and children decrease their tendencies to explore under short and ambiguous time horizons with development. This developmental shift may lead to less learning but more adaptive decision making. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Collapse
Affiliation(s)
- Winnie Zhuang
- Department of Psychology and Center for Mind and Brain, University of California, Davis
- Department of Psychology, University of Colorado Boulder
| | - Jesse Niebaum
- Department of Psychology and Center for Mind and Brain, University of California, Davis
| | - Yuko Munakata
- Department of Psychology and Center for Mind and Brain, University of California, Davis
- Department of Psychology, University of Colorado Boulder
| |
Collapse
|
13
|
Campbell EM, Singh G, Claus ED, Witkiewitz K, Costa VD, Hogeveen J, Cavanagh JF. Electrophysiological Markers of Aberrant Cue-Specific Exploration in Hazardous Drinkers. COMPUTATIONAL PSYCHIATRY (CAMBRIDGE, MASS.) 2023; 7:47-59. [PMID: 38774639 PMCID: PMC11104413 DOI: 10.5334/cpsy.96] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/04/2023] [Accepted: 06/28/2023] [Indexed: 05/24/2024]
Abstract
Background Hazardous drinking is associated with maladaptive alcohol-related decision-making. Existing studies have often focused on how participants learn to exploit familiar cues based on prior reinforcement, but little is known about the mechanisms that drive hazardous drinkers to explore novel alcohol cues when their value is not known. Methods We investigated exploration of novel alcohol and non-alcohol cues in hazardous drinkers (N = 27) and control participants (N = 26) during electroencephalography (EEG). A normative computational model with two free parameters was fit to estimate participants' weighting of the future value of exploration and immediate value of exploitation. Results Hazardous drinkers demonstrated increased exploration of novel alcohol cues, and conversely, increased probability of exploiting familiar alternatives instead of exploring novel non-alcohol cues. The motivation to explore novel alcohol stimuli in hazardous drinkers was driven by an elevated relative future valuation of uncertain alcohol cues. P3a predicted more exploratory decision policies driven by an enhanced relative future valuation of novel alcohol cues. P3b did not predict choice behavior, but computational parameter estimates suggested that hazardous drinkers with enhanced P3b to alcohol cues were likely to learn to exploit their immediate expected value. Conclusions Hazardous drinkers did not display atypical choice behavior, different P3a/P3b amplitudes, or computational estimates to novel non-alcohol cues-diverging from previous studies in addiction showing atypical generalized explore-exploit decisions with non-drug-related cues. These findings reveal that cue-specific neural computations may drive aberrant alcohol-related decision-making in hazardous drinkers-highlighting the importance of drug-relevant cues in studies of decision-making in addiction.
Collapse
Affiliation(s)
- Ethan M. Campbell
- Department of Psychology & Psychology Clinical Neuroscience Center, University of New Mexico, US
| | - Garima Singh
- Department of Psychology & Psychology Clinical Neuroscience Center, University of New Mexico, US
| | - Eric D. Claus
- Department of Biobehavioral Health, Pennsylvania State University, US
| | - Katie Witkiewitz
- Department of Psychology & Psychology Clinical Neuroscience Center, University of New Mexico, US
| | - Vincent D. Costa
- Division of Neuroscience, Oregon National Primate Research Center, US
| | - Jeremy Hogeveen
- Department of Psychology & Psychology Clinical Neuroscience Center, University of New Mexico, US
| | - James F. Cavanagh
- Department of Psychology & Psychology Clinical Neuroscience Center, University of New Mexico, US
| |
Collapse
|
14
|
Aquino TG, Cockburn J, Mamelak AN, Rutishauser U, O'Doherty JP. Neurons in human pre-supplementary motor area encode key computations for value-based choice. Nat Hum Behav 2023; 7:970-985. [PMID: 36959327 PMCID: PMC10330469 DOI: 10.1038/s41562-023-01548-2] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Accepted: 02/02/2023] [Indexed: 03/25/2023]
Abstract
Adaptive behaviour in real-world environments requires that choices integrate several variables, including the novelty of the options under consideration, their expected value and uncertainty in value estimation. Here, to probe how integration over decision variables occurs during decision-making, we recorded neurons from the human pre-supplementary motor area (preSMA), ventromedial prefrontal cortex and dorsal anterior cingulate. Unlike the other areas, preSMA neurons not only represented separate pre-decision variables for each choice option but also encoded an integrated utility signal for each choice option and, subsequently, the decision itself. Post-decision encoding of variables for the chosen option was more widely distributed and especially prominent in the ventromedial prefrontal cortex. Our findings position the human preSMA as central to the implementation of value-based decisions.
Collapse
Affiliation(s)
- Tomas G Aquino
- Computation and Neural Systems, Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA.
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
| | - Jeffrey Cockburn
- Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - Adam N Mamelak
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Ueli Rutishauser
- Computation and Neural Systems, Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - John P O'Doherty
- Computation and Neural Systems, Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA
- Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| |
Collapse
|
15
|
Philip VM, He H, Saul MC, Dickson PE, Bubier JA, Chesler EJ. Gene expression genetics of the striatum of Diversity Outbred mice. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.11.540390. [PMID: 37214980 PMCID: PMC10197688 DOI: 10.1101/2023.05.11.540390] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
Brain transcriptional variation is a heritable trait that mediates complex behaviors, including addiction. Expression quantitative trait locus (eQTL) mapping reveals genomic regions harboring genetic variants that influence transcript abundance. In this study, we profiled transcript abundance in the striatum of 386 Diversity Outbred (J:DO) mice of both sexes using RNA-Seq. All mice were characterized using a behavioral battery of widely-used exploratory and risk-taking assays prior to transcriptional profiling. We performed eQTL mapping, incorporated the results into a browser-based eQTL viewer, and deposited co-expression network members in GeneWeaver. The eQTL viewer allows researchers to query specific genes to obtain allelic effect plots, analyze SNP associations, assess gene expression correlations, and apply mediation analysis to evaluate whether the regulatory variant is acting through the expression of another gene. GeneWeaver allows multi-species comparison of gene sets using statistical and combinatorial tools. This data resource allows users to find genetic variants that regulate differentially expressed transcripts and place them in the context of other studies of striatal gene expression and function in addiction-related behavior.
Collapse
Affiliation(s)
- Vivek M. Philip
- The Jackson Laboratory for Mammalian Genetics, Bar Harbor, ME 04605
| | - Hao He
- The Jackson Laboratory for Genomic Medicine, Farmington, CT 06032
| | - Michael C. Saul
- The Jackson Laboratory for Mammalian Genetics, Bar Harbor, ME 04605
| | - Price E. Dickson
- Department of Biomedical Sciences, Joan C. Edwards School of Medicine Marshall University, 1700 3rd Ave. Huntington, WV 25703
| | - Jason A. Bubier
- The Jackson Laboratory for Mammalian Genetics, Bar Harbor, ME 04605
| | | |
Collapse
|
16
|
Desch S, Schweinhardt P, Seymour B, Flor H, Becker S. Evidence for dopaminergic involvement in endogenous modulation of pain relief. eLife 2023; 12:e81436. [PMID: 36722857 PMCID: PMC9988263 DOI: 10.7554/elife.81436] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Accepted: 01/31/2023] [Indexed: 02/02/2023] Open
Abstract
Relief of ongoing pain is a potent motivator of behavior, directing actions to escape from or reduce potentially harmful stimuli. Whereas endogenous modulation of pain events is well characterized, relatively little is known about the modulation of pain relief and its corresponding neurochemical basis. Here, we studied pain modulation during a probabilistic relief-seeking task (a 'wheel of fortune' gambling task), in which people actively or passively received reduction of a tonic thermal pain stimulus. We found that relief perception was enhanced by active decisions and unpredictability, and greater in high novelty-seeking trait individuals, consistent with a model in which relief is tuned by its informational content. We then probed the roles of dopaminergic and opioidergic signaling, both of which are implicated in relief processing, by embedding the task in a double-blinded cross-over design with administration of the dopamine precursor levodopa and the opioid receptor antagonist naltrexone. We found that levodopa enhanced each of these information-specific aspects of relief modulation but no significant effects of the opioidergic manipulation. These results show that dopaminergic signaling has a key role in modulating the perception of pain relief to optimize motivation and behavior.
Collapse
Affiliation(s)
- Simon Desch
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg UniversityMannheimGermany
- Clinical Psychology, Department of Experimental Psychology, Heinrich Heine University DüsseldorfDüsseldorfGermany
| | - Petra Schweinhardt
- Integrative Spinal Research, Department of Chiropractic Medicine, Balgrist University Hospital, University of ZurichZurichSwitzerland
| | - Ben Seymour
- Wellcome Centre for Integrative Neuroimaging, John Radcliffe HospitalOxfordUnited Kingdom
| | - Herta Flor
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg UniversityMannheimGermany
| | - Susanne Becker
- Institute of Cognitive and Clinical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg UniversityMannheimGermany
- Clinical Psychology, Department of Experimental Psychology, Heinrich Heine University DüsseldorfDüsseldorfGermany
- Integrative Spinal Research, Department of Chiropractic Medicine, Balgrist University Hospital, University of ZurichZurichSwitzerland
| |
Collapse
|
17
|
Kenett YN, Humphries S, Chatterjee A. A Thirst for Knowledge: Grounding Curiosity, Creativity, and Aesthetics in Memory and Reward Neural Systems. CREATIVITY RESEARCH JOURNAL 2023. [DOI: 10.1080/10400419.2023.2165748] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/28/2023]
Affiliation(s)
- Yoed N. Kenett
- Technion - Faculty of Data and Decision Sciences, Israel Institute of Technology, Haifa, Israel
| | | | | |
Collapse
|
18
|
Irrelevant Threats Linger and Affect Behavior in High Anxiety. J Neurosci 2023; 43:656-671. [PMID: 36526373 PMCID: PMC9888506 DOI: 10.1523/jneurosci.1186-22.2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Revised: 11/18/2022] [Accepted: 11/24/2022] [Indexed: 12/23/2022] Open
Abstract
Threat-related information attracts attention and disrupts ongoing behavior, and particularly so for more anxious individuals. Yet, it is unknown how and to what extent threat-related information leave lingering influences on behavior (e.g., by impeding ongoing learning processes). Here, human male and female participants (N = 47) performed probabilistic reinforcement learning tasks where irrelevant distracting faces (neutral, happy, or fearful) were presented together with relevant monetary feedback. Behavioral modeling was combined with fMRI data (N = 27) to explore the neurocomputational bases of learning relevant and irrelevant information. In two separate studies, individuals with high trait anxiety showed increased avoidance of objects previously paired with the combination of neutral monetary feedback and fearful faces (but not neutral or happy faces). Behavioral modeling revealed that high anxiety increased the integration of fearful faces during feedback learning, and fMRI results (regarded as provisional, because of a relatively small sample size) further showed that variance in the prediction error signal, uniquely accounted for by fearful faces, correlated more strongly with activity in the right DLPFC for more anxious individuals. Behavioral and neuronal dissociations indicated that the threat-related distractors did not simply disrupt learning processes. By showing that irrelevant threats exert long-lasting influences on behavior, our results extend previous research that separately showed that anxiety increases learning from aversive feedbacks and distractibility by threat-related information. Our behavioral results, combined with the proposed neurocomputational mechanism, may help explain how increased exposure to irrelevant affective information contributes to the acquisition of maladaptive behaviors in more anxious individuals.SIGNIFICANCE STATEMENT In modern-day society, people are increasingly exposed to various types of irrelevant information (e.g., intruding social media announcements). Yet, the neurocomputational mechanisms influenced by irrelevant information during learning, and their interactions with increasingly distracted personality types are largely unknown. Using a reinforcement learning task, where relevant feedback is presented together with irrelevant distractors (emotional faces), we reveal an interaction between irrelevant threat-related information (fearful faces) and interindividual anxiety levels. fMRI shows provisional evidence for an interaction between anxiety levels and the coupling between activity in the DLPFC and learning signals specifically elicited by fearful faces. Our study reveals how irrelevant threat-related information may become entrenched in the anxious psyche and contribute to long-lasting abnormal behaviors.
Collapse
|
19
|
Zhang X, Liu Y, Wang L, Hu D, Liu L. A Curiosity-Based Autonomous Navigation Algorithm for Maze Robot. JOURNAL OF ADVANCED COMPUTATIONAL INTELLIGENCE AND INTELLIGENT INFORMATICS 2022. [DOI: 10.20965/jaciii.2022.p0893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
Abstract
The external reward plays an important role in the reinforcement learning process, and the quality of its design determines the final effect of the algorithm. However, in several real-world scenarios, rewards extrinsic to the agent are extremely sparse. This is particularly evident in mobile robot navigation. To solve this problem, this paper proposes a curiosity-based autonomous navigation algorithm that consists of a reinforcement learning framework and curiosity system. The curiosity system consists of three parts: prediction network, associative memory network, and curiosity rewards. The prediction network predicts the next state. An associative memory network was used to represent the world. Based on the associative memory network, an inference algorithm and distance calibration algorithm were designed. Curiosity rewards were combined with extrinsic rewards as complementary inputs to the Q-learning algorithm. The simulation results show that the algorithm helps the agent reduce repeated exploration of the environment during autonomous navigation. The algorithm also exhibits a better convergence effect.
Collapse
|
20
|
Saragosa-Harris NM, Cohen AO, Reneau TR, Villano WJ, Heller AS, Hartley CA. Real-World Exploration Increases Across Adolescence and Relates to Affect, Risk Taking, and Social Connectivity. Psychol Sci 2022; 33:1664-1679. [DOI: 10.1177/09567976221102070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Cross-species research suggests that exploratory behaviors increase during adolescence and relate to the social, affective, and risky behaviors characteristic of this developmental stage. However, how these typical adolescent behaviors manifest and relate in real-world settings remains unclear. Using geolocation tracking to quantify exploration—variability in daily movement patterns—over a 3-month period in 58 adolescents and adults (ages 13–27) in New York City, we investigated whether daily exploration varied with age and whether exploration related to social connectivity, risk taking, and momentary positive affect. In our cross-sectional sample, we found an association between daily exploration and age, with individuals near the transition to legal adulthood exhibiting the highest exploration levels. Days of higher exploration were associated with greater positive affect irrespective of age. Higher mean exploration was associated with greater social connectivity in all participants but was linked to higher risk taking selectively among adolescents. Our results highlight the interplay of exploration and socioemotional behaviors across development and suggest that societal norms may modulate their expression in naturalistic contexts.
Collapse
Affiliation(s)
| | | | | | | | | | - Catherine A. Hartley
- Department of Psychology, New York University
- Center for Neural Science, New York University
| |
Collapse
|
21
|
Hassall CD, Hunt LT. Novelty, uncertainty, and the looming horizon. Neuron 2022; 110:2521-2523. [PMID: 35981524 DOI: 10.1016/j.neuron.2022.07.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Novelty and uncertainty are powerful drivers of exploration that are often conflated. In this issue of Neuron, Cockburn and colleagues dissociate the two and report a key interaction: close to task termination, novel options appear much more attractive relative to uncertain options.
Collapse
Affiliation(s)
| | - Laurence T Hunt
- Department of Psychiatry, University of Oxford, Oxford OX3 7JX, UK.
| |
Collapse
|
22
|
Arigo D, Lobo AF, Ainsworth MC, Baga K, Pasko K. Development and Initial Testing of a Personalized, Adaptive, and Socially Focused Web Tool to Support Physical Activity Among Women in Midlife: Multidisciplinary and User-Centered Design Approach. JMIR Form Res 2022; 6:e36280. [PMID: 35881431 PMCID: PMC9364169 DOI: 10.2196/36280] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2022] [Revised: 03/18/2022] [Accepted: 04/10/2022] [Indexed: 01/02/2023] Open
Abstract
Background Women in midlife are vulnerable to developing cardiovascular disease, particularly those who have conditions such as hypertension. Physical activity (PA) can reduce risk, but efforts to promote PA in this population have been only modestly effective. More attention to social influences on PA behavior may be useful, particularly social support and social comparison processes. Activating these processes with digital tools can provide easy access that is flexible to the needs of women in midlife. Objective This paper describes the user-centered design processes of developing and conducting initial evaluation of a personalized and adaptive web application, tailored to the social needs of women in midlife. The goal was to gather feedback from the population of interest, before and during the design process. Methods This study was conducted in 4 stages. The first and second authors (DA and AFL) developed technical specifications, informed by their experience with the population of interest. We collected feedback on potential content for the web application with women in midlife using both interviews (5/10, 50%; mean age 47.4, SD 6.66 years; mean BMI 35.3, SD 9.55 kg/m2) and surveys (5/10, 50%; mean age 51, SD 6.60 years; mean BMI 32.7, SD 8.39 kg/m2). We used their feedback to inform support messages and peer profiles (ie, sources of social comparison information). Nine members of the behavioral science team and 3 testers unfamiliar with the web application completed internal testing. We conducted naturalistic functionality testing with a different group of women in midlife (n=5; mean age 50, SD 6.26 years; mean BMI 30.1, SD 5.83 kg/m2), who used the web application as intended for 4 days and provided feedback at the end of this period. Results Iterative storyboard development resulted in programming specifications for a prototype of the web application. We used content feedback to select and refine the support messages and peer profiles to be added. The following 2 rounds of internal testing identified bugs and other problems regarding the web application’s functioning and full data collection procedure. Problems were addressed or logged for future consideration. Naturalistic functionality testing revealed minimal further problems; findings showed preliminary acceptability of the web application and suggested that women may select different social content across days. Conclusions A multidisciplinary and user-centered design approach led to a personalized and adaptive web application, tailored to the social needs of women in midlife. Findings from testing with this population demonstrated the feasibility and acceptability of the new application and supported further development toward its use in daily life. We describe several potential uses of the web application and next steps for its development. We also discuss the lessons learned and offer recommendations for future collaborations between behavioral and computer scientists to develop similar tools.
Collapse
Affiliation(s)
- Danielle Arigo
- Department of Psychology, Rowan University, Glassboro, NJ, United States
- Department of Family Medicine, Rowan School of Osteopathic Medicine, Stratford, NJ, United States
| | - Andrea F Lobo
- Department of Computer Science, Rowan University, Glassboro, NJ, United States
| | - M Cole Ainsworth
- Department of Psychology, Rowan University, Glassboro, NJ, United States
| | - Kiri Baga
- Department of Psychology, Rowan University, Glassboro, NJ, United States
| | - Kristen Pasko
- Department of Psychology, Rowan University, Glassboro, NJ, United States
| |
Collapse
|
23
|
Cockburn J, Man V, Cunningham W, O'Doherty JP. Novelty and uncertainty regulate the balance between exploration and exploitation through distinct mechanisms in the human brain. Neuron 2022; 110:2691-2702.e8. [PMID: 35809575 DOI: 10.1016/j.neuron.2022.05.025] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2021] [Revised: 02/18/2022] [Accepted: 05/27/2022] [Indexed: 11/17/2022]
Abstract
Both novelty and uncertainty are potent features guiding exploration; however, they are often experimentally conflated, and an understanding of how they interact to regulate the balance between exploration and exploitation has proved elusive. Using a task designed to decouple the influence of novelty and uncertainty, we identify separable mechanisms through which exploration is directed. We show that uncertainty-directed exploration is sensitive to the prospective benefit offered by new information, whereas novelty-directed exploration is maintained regardless of its potential advantage. Using a computational framework in conjunction with fMRI, we show that uncertainty-directed choice is rooted in an adaptive bias indexing the prospective utility of exploration. In contrast, novelty persistently promotes exploration by optimistically inflating reward expectations while simultaneously dampening uncertainty signals. Our results identify separable neural substrates charged with balancing the explore/exploit trade-off to foster a manageable decomposition of an otherwise intractable problem.
Collapse
Affiliation(s)
- Jeffrey Cockburn
- Division of Humanities and Social Sciences, Caltech, Pasadena, CA, USA.
| | - Vincent Man
- Division of Humanities and Social Sciences, Caltech, Pasadena, CA, USA
| | | | - John P O'Doherty
- Division of Humanities and Social Sciences, Caltech, Pasadena, CA, USA
| |
Collapse
|
24
|
Fermin ASR, Friston K, Yamawaki S. An insula hierarchical network architecture for active interoceptive inference. ROYAL SOCIETY OPEN SCIENCE 2022; 9:220226. [PMID: 35774133 PMCID: PMC9240682 DOI: 10.1098/rsos.220226] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Accepted: 06/09/2022] [Indexed: 05/05/2023]
Abstract
In the brain, the insular cortex receives a vast amount of interoceptive information, ascending through deep brain structures, from multiple visceral organs. The unique hierarchical and modular architecture of the insula suggests specialization for processing interoceptive afferents. Yet, the biological significance of the insula's neuroanatomical architecture, in relation to deep brain structures, remains obscure. In this opinion piece, we propose the Insula Hierarchical Modular Adaptive Interoception Control (IMAC) model to suggest that insula modules (granular, dysgranular and agranular), forming parallel networks with the prefrontal cortex and striatum, are specialized to form higher order interoceptive representations. These interoceptive representations are recruited in a context-dependent manner to support habitual, model-based and exploratory control of visceral organs and physiological processes. We discuss how insula interoceptive representations may give rise to conscious feelings that best explain lower order deep brain interoceptive representations, and how the insula may serve to defend the body and mind against pathological depression.
Collapse
Affiliation(s)
- Alan S. R. Fermin
- Center for Brain, Mind and Kansei Sciences Research, Hiroshima University, Hiroshima, Japan
| | - Karl Friston
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, London, England
| | - Shigeto Yamawaki
- Center for Brain, Mind and Kansei Sciences Research, Hiroshima University, Hiroshima, Japan
| |
Collapse
|
25
|
Hogeveen J, Mullins TS, Romero JD, Eversole E, Rogge-Obando K, Mayer AR, Costa VD. The neurocomputational bases of explore-exploit decision-making. Neuron 2022; 110:1869-1879.e5. [PMID: 35390278 PMCID: PMC9167768 DOI: 10.1016/j.neuron.2022.03.014] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2021] [Revised: 12/11/2021] [Accepted: 03/10/2022] [Indexed: 02/04/2023]
Abstract
Flexible decision-making requires animals to forego immediate rewards (exploitation) and try novel choice options (exploration) to discover if they are preferable to familiar alternatives. Using the same task and a partially observable Markov decision process (POMDP) model to quantify the value of choices, we first determined that the computational basis for managing explore-exploit tradeoffs is conserved across monkeys and humans. We then used fMRI to identify where in the human brain the immediate value of exploitative choices and relative uncertainty about the value of exploratory choices were encoded. Consistent with prior neurophysiological evidence in monkeys, we observed divergent encoding of reward value and uncertainty in prefrontal and parietal regions, including frontopolar cortex, and parallel encoding of these computations in motivational regions including the amygdala, ventral striatum, and orbitofrontal cortex. These results clarify the interplay between prefrontal and motivational circuits that supports adaptive explore-exploit decisions in humans and nonhuman primates.
Collapse
Affiliation(s)
- Jeremy Hogeveen
- Department of Psychology, University of New Mexico, Albuquerque, NM 87131, USA; Psychology Clinical Neuroscience Center, University of New Mexico, Albuquerque, NM 87131, USA.
| | - Teagan S Mullins
- Department of Psychology, University of New Mexico, Albuquerque, NM 87131, USA; Psychology Clinical Neuroscience Center, University of New Mexico, Albuquerque, NM 87131, USA
| | - John D Romero
- Department of Psychology, University of New Mexico, Albuquerque, NM 87131, USA; Psychology Clinical Neuroscience Center, University of New Mexico, Albuquerque, NM 87131, USA
| | - Elizabeth Eversole
- Department of Psychology, University of New Mexico, Albuquerque, NM 87131, USA; Psychology Clinical Neuroscience Center, University of New Mexico, Albuquerque, NM 87131, USA
| | - Kimberly Rogge-Obando
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN 37235, USA
| | - Andrew R Mayer
- Department of Psychology, University of New Mexico, Albuquerque, NM 87131, USA; Department of Psychiatry & Behavioral Sciences, University of New Mexico School of Medicine, Albuquerque, NM 87131, USA; Department of Neurology, University of New Mexico School of Medicine, Albuquerque, NM 87131, USA; The Mind Research Network/Lovelace Biomedical Research Institute, Pete & Nancy Domenici Hall, Albuquerque, NM 87106, USA
| | - Vincent D Costa
- Department of Behavioral Neuroscience, Oregon Health and Science University, Portland, OR 97239, USA; Division of Neuroscience, Oregon National Primate Research Center, Beaverton, OR 97006, USA.
| |
Collapse
|
26
|
Novelty preferences depend on goals. Psychon Bull Rev 2022; 29:2293-2301. [PMID: 35618942 PMCID: PMC9135316 DOI: 10.3758/s13423-022-02118-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/03/2022] [Indexed: 11/26/2022]
Abstract
People are sometimes drawn to novel items, but other times prefer familiar ones. In the present research we show, though, that both children’s and adults’ preferences for novel versus familiar items depend on their goals. Across four experiments, we showed 4- to 7-year-olds (total N = 498) and adults (total N = 659) pairs of artifacts where one was familiar and the other was novel (e.g., a four-legged chair and ten-legged chair). In Experiment 1, children wanted to have familiar artifacts, but to learn about novel ones. Experiment 2 replicated this pattern using a simpler procedure, and found the same pattern in adults. In Experiment 3, 4- to 6-year-olds and adults more strongly preferred familiar items when choosing which they would rather have than when choosing which they would rather try using. Finally, Experiment 4 replicated adults’ preferences to have familiar items and learn about novel ones with an additional set of items. Together these findings show that preferences for novelty depend on people’s goals. We suggest these effects arise because children and adults are motivated both by the promise of information and the desire for safe options in high commitment decisions that entail risk.
Collapse
|
27
|
Poli F, Meyer M, Mars RB, Hunnius S. Contributions of expected learning progress and perceptual novelty to curiosity-driven exploration. Cognition 2022; 225:105119. [PMID: 35421742 PMCID: PMC9194910 DOI: 10.1016/j.cognition.2022.105119] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Revised: 03/31/2022] [Accepted: 04/01/2022] [Indexed: 11/30/2022]
Abstract
Exploration is curiosity-driven when it relies on the intrinsic motivation to know rather than on extrinsic rewards. Recent evidence shows that artificial agents perform better on a variety of tasks when their learning is curiosity-driven, and humans often engage in curiosity-driven learning when sampling information from the environment. However, which mechanisms underlie curiosity is still unclear. Here, we let participants freely explore different unknown environments that contained learnable sequences of events with varying degrees of noise and volatility. A hierarchical reinforcement learning model captured how participants were learning in these different kinds of unknown environments, and it also tracked the errors they expected to make and the learning opportunities they were planning to seek. With this computational approach, we show that participants' exploratory behavior is guided by learning progress and perceptual novelty. Moreover, we demonstrate an overall tendency of participants to avoid extreme forms of uncertainty. These findings elucidate the cognitive mechanisms that underlie curiosity-driven exploration of unknown environments. Implications of this novel way of quantifying curiosity within a reinforcement learning framework are discussed.
Collapse
Affiliation(s)
- Francesco Poli
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands.
| | - Marlene Meyer
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
| | - Rogier B Mars
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands; Wellcome Centre for Integrative Neuroimaging, Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
| | - Sabine Hunnius
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
| |
Collapse
|
28
|
Petzke TM, Schomaker J. A bias toward the unknown: individual and environmental factors influencing exploratory behavior. Ann N Y Acad Sci 2022; 1512:61-75. [PMID: 35218049 PMCID: PMC9306615 DOI: 10.1111/nyas.14757] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2021] [Accepted: 01/21/2022] [Indexed: 11/29/2022]
Abstract
With limited resources, exploring new opportunities is crucial for survival. Exploring novel options, however, comes at the cost of uncertainty. Therefore, there is a trade‐off between exploiting options with a known beneficial outcome and exploring novel options with a potentially higher gain. Computational models have suggested that novelty may promote exploratory behavior by inducing a so‐called novelty bonus through reward‐related processes. So far, few studies have provided behavioral evidence for such a novelty bonus. In this study, we aimed to investigate whether spatial novelty can stimulate exploratory behavior (Experiment 1), and whether age, novelty‐seeking, and reduced action radius or social interactions due to COVID‐19 restrictions influenced the exploration–exploitation trade‐off (Experiment 2). In both experiments, we employed a novel paradigm in which participants made binary decisions between food items, while on rare trials, a surprise option was presented. Results from Experiment 1 are in line with a novelty bonus, with spatial novelty promoting exploratory behavior. In Experiment 2, we found that exploratory behavior declined with age, high novelty seekers made more exploratory choices than low novelty seekers, and participants with a smaller action radius made fewer exploratory choices. These findings are consistent with previous findings in animals and predictions from computational models.
Collapse
Affiliation(s)
- Tara M Petzke
- Department of Health, Medical & Neuropsychology, Leiden University, Leiden, the Netherlands
| | - Judith Schomaker
- Department of Health, Medical & Neuropsychology, Leiden University, Leiden, the Netherlands.,Leiden Institute for Brain and Cognition, Leiden, the Netherlands
| |
Collapse
|
29
|
Karayanni M, Nelken I. Extrinsic rewards, intrinsic rewards, and non-optimal behavior. J Comput Neurosci 2022; 50:139-143. [PMID: 35122189 DOI: 10.1007/s10827-022-00813-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2021] [Revised: 01/20/2022] [Accepted: 01/25/2022] [Indexed: 10/19/2022]
Abstract
The optimality of behavior in experimental settings is usually determined with respect to an extrinsic reward defined by the experimenters. However, actions that do not lead to reward are ubiquitous in many species and in many experimental paradigms. Modern research on decision processes commonly treat non-optimal behaviors as noise, often excluding from analysis animals that do not reach behavioral performance criteria. However, non-optimal behaviors can be a window on important brain processes. Here we explore the evidence that non-optimal behaviors are the consequence of intrinsically motivated actions, related to drives that are different from that of obtaining extrinsic reward. One way of operationally characterizing these drives is by postulating intrinsic rewards associated with them. Behaviors that are apparently non-optimal can be interpreted as the consequence of optimal decisions whose goal is to optimize a combination of intrinsic and extrinsic rewards. We review intrinsic rewards that have been discussed in the literature, and suggest ways of testing their existence and role in shaping animal behavior.
Collapse
Affiliation(s)
- Mousa Karayanni
- Edmond and Lily Safra Center for Brain Sciences, Hebrew University, Jerusalem, Israel.
| | - Israel Nelken
- Edmond and Lily Safra Center for Brain Sciences, Hebrew University, Jerusalem, Israel.,Department of Neurobiology, Silberman Institute of Life Sciences, Hebrew University, Jerusalem, Israel
| |
Collapse
|
30
|
Differential coding of goals and actions in ventral and dorsal corticostriatal circuits during goal-directed behavior. Cell Rep 2022; 38:110198. [PMID: 34986350 PMCID: PMC9608360 DOI: 10.1016/j.celrep.2021.110198] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2021] [Revised: 07/08/2021] [Accepted: 12/10/2021] [Indexed: 02/04/2023] Open
Abstract
Goal-directed behavior requires identifying objects in the environment that can satisfy internal needs and executing actions to obtain those objects. The current study examines ventral and dorsal corticostriatal circuits that support complementary aspects of goal-directed behavior. We analyze activity from the amygdala, ventral striatum, orbitofrontal cortex, and lateral prefrontal cortex (LPFC) while monkeys perform a three-armed bandit task. Information about chosen stimuli and their value is primarily encoded in the amygdala, ventral striatum, and orbitofrontal cortex, while the spatial information is primarily encoded in the LPFC. Before the options are presented, information about the to-be-chosen stimulus is represented in the amygdala, ventral striatum, and orbitofrontal cortex; at the time of choice, the information is passed to the LPFC to direct a saccade. Thus, learned value information specifying behavioral goals is maintained throughout the ventral corticostriatal circuit, and it is routed through the dorsal circuit at the time actions are selected.
Collapse
|
31
|
Averbeck B, O'Doherty JP. Reinforcement-learning in fronto-striatal circuits. Neuropsychopharmacology 2022; 47:147-162. [PMID: 34354249 PMCID: PMC8616931 DOI: 10.1038/s41386-021-01108-0] [Citation(s) in RCA: 34] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Revised: 07/06/2021] [Accepted: 07/09/2021] [Indexed: 01/03/2023]
Abstract
We review the current state of knowledge on the computational and neural mechanisms of reinforcement-learning with a particular focus on fronto-striatal circuits. We divide the literature in this area into five broad research themes: the target of the learning-whether it be learning about the value of stimuli or about the value of actions; the nature and complexity of the algorithm used to drive the learning and inference process; how learned values get converted into choices and associated actions; the nature of state representations, and of other cognitive machinery that support the implementation of various reinforcement-learning operations. An emerging fifth area focuses on how the brain allocates or arbitrates control over different reinforcement-learning sub-systems or "experts". We will outline what is known about the role of the prefrontal cortex and striatum in implementing each of these functions. We then conclude by arguing that it will be necessary to build bridges from algorithmic level descriptions of computational reinforcement-learning to implementational level models to better understand how reinforcement-learning emerges from multiple distributed neural networks in the brain.
Collapse
Affiliation(s)
| | - John P O'Doherty
- Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA.
| |
Collapse
|
32
|
Anderson BA, Kim H, Kim AJ, Liao MR, Mrkonja L, Clement A, Grégoire L. The past, present, and future of selection history. Neurosci Biobehav Rev 2021; 130:326-350. [PMID: 34499927 PMCID: PMC8511179 DOI: 10.1016/j.neubiorev.2021.09.004] [Citation(s) in RCA: 49] [Impact Index Per Article: 16.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2021] [Revised: 07/08/2021] [Accepted: 09/02/2021] [Indexed: 01/22/2023]
Abstract
The last ten years of attention research have witnessed a revolution, replacing a theoretical dichotomy (top-down vs. bottom-up control) with a trichotomy (biased by current goals, physical salience, and selection history). This third new mechanism of attentional control, selection history, is multifaceted. Some aspects of selection history must be learned over time whereas others reflect much more transient influences. A variety of different learning experiences can shape the attention system, including reward, aversive outcomes, past experience searching for a target, target‒non-target relations, and more. In this review, we provide an overview of the historical forces that led to the proposal of selection history as a distinct mechanism of attentional control. We then propose a formal definition of selection history, with concrete criteria, and identify different components of experience-driven attention that fit within this definition. The bulk of the review is devoted to exploring how these different components relate to one another. We conclude by proposing an integrative account of selection history centered on underlying themes that emerge from our review.
Collapse
Affiliation(s)
- Brian A Anderson
- Texas A&M University, College Station, TX, 77843, United States.
| | - Haena Kim
- Texas A&M University, College Station, TX, 77843, United States
| | - Andy J Kim
- Texas A&M University, College Station, TX, 77843, United States
| | - Ming-Ray Liao
- Texas A&M University, College Station, TX, 77843, United States
| | - Lana Mrkonja
- Texas A&M University, College Station, TX, 77843, United States
| | - Andrew Clement
- Texas A&M University, College Station, TX, 77843, United States
| | | |
Collapse
|
33
|
Zhen S, Yaple ZA, Eickhoff SB, Yu R. To learn or to gain: neural signatures of exploration in human decision-making. Brain Struct Funct 2021; 227:63-76. [PMID: 34596757 DOI: 10.1007/s00429-021-02389-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2020] [Accepted: 09/19/2021] [Indexed: 11/26/2022]
Abstract
Individuals not only take actions to obtain immediate rewards but also to gain more information to guide future choices. An ideal exploration-exploitation balance is crucial for maximizing reward over the long run. However, the neural signatures of exploration in humans remain unclear. Using quantitative meta-analyses of functional magnetic resonance imaging experiments on exploratory behaviors, we sought to identify the concordant activity pertaining to exploration over a range of experiments. The results revealed that exploration activates concordant brain activity associated with risk (e.g., dorsal medial prefrontal cortex and anterior insula), cognitive control (e.g., dorsolateral prefrontal cortex and inferior frontal gyrus), and motor processing (e.g., premotor cortex). These stereotaxic maps of exploration may indicate that exploration is highly linked to risk processing, but is also specifically associated with regions involved in executive control processes. Although this explanation should be treated as exploratory, these findings support theories positing an important role for the prefrontal-insular-motor cortical network in exploration.
Collapse
Affiliation(s)
- Shanshan Zhen
- Department of Management, Hong Kong Baptist University, Hong Kong, China
| | - Zachary A Yaple
- Department of Psychology, Faculty of Health, York University, Toronto, ON, Canada
| | - Simon B Eickhoff
- Medical Faculty, Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
- Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany
| | - Rongjun Yu
- Department of Management, Hong Kong Baptist University, Hong Kong, China.
| |
Collapse
|
34
|
The interplay between domain-general and domain-specific mechanisms during the time-course of verbal associative learning: An event-related potential study. Neuroimage 2021; 242:118443. [PMID: 34352392 DOI: 10.1016/j.neuroimage.2021.118443] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2020] [Revised: 07/17/2021] [Accepted: 07/26/2021] [Indexed: 11/24/2022] Open
Abstract
Humans continuously learn new information. Here, we examined the temporal brain dynamics of explicit verbal associative learning between unfamiliar items. In the first experiment, 25 adults learned object-pseudoword associations during a 5-day training program allowing us to track the N400 dynamics across learning blocks within and across days. Successful learning was accompanied by an initial frontal N400 that decreased in amplitude across blocks during the first day and shifted to parietal sites during the last training day. In Experiment 2, we replicated our findings with 38 new participants randomly assigned to a consistent learning or an inconsistent learning group. The N400 amplitude modulations that we found, both within and between learning sessions, are taken to reflect the emergence of novel lexical traces even when learning concerns items for which no semantic information is provided. The shift in N400 topography suggests that different N400 neural generators may contribute to specific word learning steps through a balance between domain-general and language-specific mechanisms.
Collapse
|
35
|
Wolf C, Lappe M. Vision as oculomotor reward: cognitive contributions to the dynamic control of saccadic eye movements. Cogn Neurodyn 2021; 15:547-568. [PMID: 34367360 PMCID: PMC8286912 DOI: 10.1007/s11571-020-09661-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2020] [Revised: 12/12/2020] [Accepted: 12/28/2020] [Indexed: 01/08/2023] Open
Abstract
Humans and other primates are equipped with a foveated visual system. As a consequence, we reorient our fovea to objects and targets in the visual field that are conspicuous or that we consider relevant or worth looking at. These reorientations are achieved by means of saccadic eye movements. Where we saccade to depends on various low-level factors such as a targets' luminance but also crucially on high-level factors like the expected reward or a targets' relevance for perception and subsequent behavior. Here, we review recent findings how the control of saccadic eye movements is influenced by higher-level cognitive processes. We first describe the pathways by which cognitive contributions can influence the neural oculomotor circuit. Second, we summarize what saccade parameters reveal about cognitive mechanisms, particularly saccade latencies, saccade kinematics and changes in saccade gain. Finally, we review findings on what renders a saccade target valuable, as reflected in oculomotor behavior. We emphasize that foveal vision of the target after the saccade can constitute an internal reward for the visual system and that this is reflected in oculomotor dynamics that serve to quickly and accurately provide detailed foveal vision of relevant targets in the visual field.
Collapse
Affiliation(s)
- Christian Wolf
- Institute for Psychology, University of Muenster, Fliednerstrasse 21, 48149 Münster, Germany
| | - Markus Lappe
- Institute for Psychology, University of Muenster, Fliednerstrasse 21, 48149 Münster, Germany
| |
Collapse
|
36
|
Dezza IC, Noel X, Cleeremans A, Yu AJ. Distinct motivations to seek out information in healthy individuals and problem gamblers. Transl Psychiatry 2021; 11:408. [PMID: 34312367 PMCID: PMC8313706 DOI: 10.1038/s41398-021-01523-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Revised: 06/04/2021] [Accepted: 06/28/2021] [Indexed: 02/07/2023] Open
Abstract
As massive amounts of information are becoming available to people, understanding the mechanisms underlying information-seeking is more pertinent today than ever. In this study, we investigate the underlying motivations to seek out information in healthy and addicted individuals. We developed a novel decision-making task and a novel computational model which allows dissociating the relative contribution of two motivating factors to seek out information: a desire for novelty and a general desire for knowledge. To investigate whether/how the motivations to seek out information vary between healthy and addicted individuals, in addition to healthy controls we included a sample of individuals with gambling disorder-a form of addiction without the confound of substance consumption and characterized by compulsive gambling. Our results indicate that healthy subjects and problem gamblers adopt distinct information-seeking "modes". Healthy information-seeking behavior was mostly motivated by a desire for novelty. Problem gamblers, on the contrary, displayed reduced novelty-seeking and an increased desire for accumulating knowledge compared to healthy controls. Our findings not only shed new light on the motivations driving healthy and addicted individuals to seek out information, but they also have important implications for the treatment and diagnosis of behavioral addiction.
Collapse
Affiliation(s)
- Irene Cogliati Dezza
- grid.4989.c0000 0001 2348 0746Centre for Research in Cognition and Neurosciences, ULB Neuroscience Institute, Université Libre de Bruxelles, Bruxelles, Belgium ,grid.83440.3b0000000121901201Department of Experimental Psychology, Faculty of Brain Sciences, University College London, London, UK ,grid.83440.3b0000000121901201The Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK ,grid.5342.00000 0001 2069 7798Department of Experimental Psychology, Ghent University, Ghent, Belgium
| | - Xavier Noel
- grid.4989.c0000 0001 2348 0746Faculty of Medicine, Université Libre de Bruxelles, Bruxelles, Belgium
| | - Axel Cleeremans
- grid.4989.c0000 0001 2348 0746Centre for Research in Cognition and Neurosciences, ULB Neuroscience Institute, Université Libre de Bruxelles, Bruxelles, Belgium
| | - Angela J. Yu
- grid.266100.30000 0001 2107 4242Department of Cognitive Science, University of California San Diego, San Diego, USA
| |
Collapse
|
37
|
Wilson RC, Bonawitz E, Costa VD, Ebitz RB. Balancing exploration and exploitation with information and randomization. Curr Opin Behav Sci 2021; 38:49-56. [PMID: 33184605 PMCID: PMC7654823 DOI: 10.1016/j.cobeha.2020.10.001] [Citation(s) in RCA: 77] [Impact Index Per Article: 25.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Abstract
Explore-exploit decisions require us to trade off the benefits of exploring unknown options to learn more about them, with exploiting known options, for immediate reward. Such decisions are ubiquitous in nature, but from a computational perspective, they are notoriously hard. There is therefore much interest in how humans and animals make these decisions and recently there has been an explosion of research in this area. Here we provide a biased and incomplete snapshot of this field focusing on the major finding that many organisms use two distinct strategies to solve the explore-exploit dilemma: a bias for information ('directed exploration') and the randomization of choice ('random exploration'). We review evidence for the existence of these strategies, their computational properties, their neural implementations, as well as how directed and random exploration vary over the lifespan. We conclude by highlighting open questions in this field that are ripe to both explore and exploit.
Collapse
Affiliation(s)
- Robert C. Wilson
- Department of Psychology, University of Arizona, Tucson AZ USA
- Cognitive Science Program, University of Arizona, Tucson AZ USA
- Evelyn F. McKnight Brain Institute, University of Arizona, Tucson AZ USA
| | | | - Vincent D. Costa
- Department of Behavioral Neuroscience, Oregon Health and Science University, Portland OR USA
| | - R. Becket Ebitz
- Department of Neuroscience, University of Montréal, Montréal, Québec, Canada
| |
Collapse
|
38
|
Aloi J, Crum KI, Blair KS, Zhang R, Bashford-Largo J, Bajaj S, Schwartz A, Carollo E, Hwang S, Leiker E, Filbey FM, Averbeck BB, Dobbertin M, Blair RJR. Individual associations of adolescent alcohol use disorder versus cannabis use disorder symptoms in neural prediction error signaling and the response to novelty. Dev Cogn Neurosci 2021; 48:100944. [PMID: 33773241 PMCID: PMC8024914 DOI: 10.1016/j.dcn.2021.100944] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2020] [Revised: 03/13/2021] [Accepted: 03/14/2021] [Indexed: 12/20/2022] Open
Abstract
Two of the most commonly used illegal substances by adolescents are alcohol and cannabis. Alcohol use disorder (AUD) and cannabis use disorder (CUD) are associated with poorer decision-making in adolescents. In adolescents, level of AUD symptomatology has been negatively associated with striatal reward responsivity. However, little work has explored the relationship with striatal reward prediction error (RPE) representation and the extent to which any augmentation of RPE by novel stimuli is impacted. One-hundred fifty-one adolescents participated in the Novelty Task while undergoing functional magnetic resonance imaging (fMRI). In this task, participants learn to choose novel or non-novel stimuli to gain monetary reward. Level of AUD symptomatology was negatively associated with both optimal decision-making and BOLD response modulation by RPE within striatum and regions of prefrontal cortex. The neural alterations in RPE representation were particularly pronounced when participants were exploring novel stimuli. Level of CUD symptomatology moderated the relationship between novelty propensity and RPE representation within inferior parietal lobule and dorsomedial prefrontal cortex. These data expand on an emerging literature investigating individual associations of AUD symptomatology levels versus CUD symptomatology levels and RPE representation during reinforcement processing and provide insight on the role of neuro-computational processes underlying reinforcement learning/decision-making in adolescents.
Collapse
Affiliation(s)
- Joseph Aloi
- Center for Neurobehavioral Research in Children, Boys Town National Research Hospital, Boys Town, NE, United States; College of Medicine, University of Nebraska Medical Center, Omaha, NE, United States; Department of Psychiatry, Indiana University School of Medicine, Indianapolis, IN, United States.
| | - Kathleen I Crum
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis, IN, United States; Adolescent Behavioral Health Research Program, Indiana University School of Medicine, Indianapolis, IN, United States; Department of Neuroscience, Medical University of South Carolina, Charleston, SC, United States
| | - Karina S Blair
- Center for Neurobehavioral Research in Children, Boys Town National Research Hospital, Boys Town, NE, United States
| | - Ru Zhang
- Center for Neurobehavioral Research in Children, Boys Town National Research Hospital, Boys Town, NE, United States
| | - Johannah Bashford-Largo
- Center for Neurobehavioral Research in Children, Boys Town National Research Hospital, Boys Town, NE, United States
| | - Sahil Bajaj
- Center for Neurobehavioral Research in Children, Boys Town National Research Hospital, Boys Town, NE, United States
| | - Amanda Schwartz
- Center for Neurobehavioral Research in Children, Boys Town National Research Hospital, Boys Town, NE, United States
| | - Erin Carollo
- Center for Neurobehavioral Research in Children, Boys Town National Research Hospital, Boys Town, NE, United States
| | - Soonjo Hwang
- Department of Psychiatry, University of Nebraska Medical Center, Omaha, NE, United States
| | - Emily Leiker
- Department of Psychiatry, University of Pittsburgh Medical Center, Pittsburgh, PA, United States
| | - Francesca M Filbey
- Center for BrainHealth, School of Behavioral and Brain Sciences, University of Texas at Dallas, Dallas, TX, United States
| | - Bruno B Averbeck
- Section on Learning and Decision Making, Laboratory of Neuropsychology, National Institute of Mental Health, Bethesda, MD, United States
| | - Matthew Dobbertin
- Center for Neurobehavioral Research in Children, Boys Town National Research Hospital, Boys Town, NE, United States
| | - R James R Blair
- Center for Neurobehavioral Research in Children, Boys Town National Research Hospital, Boys Town, NE, United States
| |
Collapse
|
39
|
Rohan ML, Lowen SB, Rock A, Andersen SL. Novelty preferences and cocaine-associated cues influence regions associated with the salience network in juvenile female rats. Pharmacol Biochem Behav 2021; 203:173117. [PMID: 33561479 DOI: 10.1016/j.pbb.2021.173117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/26/2020] [Revised: 01/18/2021] [Accepted: 01/19/2021] [Indexed: 10/22/2022]
Abstract
Preferences for novel environments (novelty-seeking) is a risk factor for addiction, with little known about its underlying circuitry. Exposure to drug cues facilitates addiction maintenance, leading us to hypothesize that exposure to a novel environment activates a shared neural circuitry. Stimulation of the D1 receptor in the prelimbic cortex increases responsivity to drug-associated environments. Here, we use D1 receptor overexpression in the prelimbic cortex to probe brain responses to novelty-preferences (in a free-choice paradigm) and cocaine-associated odors following place conditioning. These same cocaine-conditioned odors were used to study neural circuitry with Blood Oxygen Level Dependent (BOLD) activity. D1 overexpressing females had deactivated BOLD signals related to novelty-preferences within the insula cortex and amygdala and activation in the frontal cortex and dopamine cell bodies. BOLD responses to cocaine cues were also sensitive to D1. Control females demonstrated a place preference for cocaine environments with no significant BOLD response, while D1 overexpressing females demonstrated a place aversion and weak BOLD responses to cocaine-conditioned odor cues within the insula cortex. For comparison, we provide data from an earlier study with juvenile males overexpressing D1 that show a strong preference for cocaine and elevated BOLD responses. The results support the use of a pharmacological manipulation (e.g., D1 overexpression) to probe the neural circuitry downstream from the prelimbic cortex.
Collapse
Affiliation(s)
- Michael L Rohan
- McLean Hospital, Department of Psychiatry, Harvard Medical School, 115 Mill Street, Belmont, MA 02478, United States of America
| | - Steven B Lowen
- McLean Hospital, Department of Psychiatry, Harvard Medical School, 115 Mill Street, Belmont, MA 02478, United States of America
| | - Anna Rock
- McLean Hospital, Department of Psychiatry, Harvard Medical School, 115 Mill Street, Belmont, MA 02478, United States of America
| | - Susan L Andersen
- McLean Hospital, Department of Psychiatry, Harvard Medical School, 115 Mill Street, Belmont, MA 02478, United States of America.
| |
Collapse
|
40
|
Dubois M, Habicht J, Michely J, Moran R, Dolan RJ, Hauser TU. Human complex exploration strategies are enriched by noradrenaline-modulated heuristics. eLife 2021; 10:e59907. [PMID: 33393461 PMCID: PMC7815309 DOI: 10.7554/elife.59907] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2020] [Accepted: 01/03/2021] [Indexed: 01/15/2023] Open
Abstract
An exploration-exploitation trade-off, the arbitration between sampling a lesser-known against a known rich option, is thought to be solved using computationally demanding exploration algorithms. Given known limitations in human cognitive resources, we hypothesised the presence of additional cheaper strategies. We examined for such heuristics in choice behaviour where we show this involves a value-free random exploration, that ignores all prior knowledge, and a novelty exploration that targets novel options alone. In a double-blind, placebo-controlled drug study, assessing contributions of dopamine (400 mg amisulpride) and noradrenaline (40 mg propranolol), we show that value-free random exploration is attenuated under the influence of propranolol, but not under amisulpride. Our findings demonstrate that humans deploy distinct computationally cheap exploration strategies and that value-free random exploration is under noradrenergic control.
Collapse
Affiliation(s)
- Magda Dubois
- Max Planck UCL Centre for Computational Psychiatry and Ageing ResearchLondonUnited Kingdom
- Wellcome Trust Centre for Neuroimaging, University College LondonLondonUnited Kingdom
| | - Johanna Habicht
- Max Planck UCL Centre for Computational Psychiatry and Ageing ResearchLondonUnited Kingdom
- Wellcome Trust Centre for Neuroimaging, University College LondonLondonUnited Kingdom
| | - Jochen Michely
- Max Planck UCL Centre for Computational Psychiatry and Ageing ResearchLondonUnited Kingdom
- Wellcome Trust Centre for Neuroimaging, University College LondonLondonUnited Kingdom
- Department of Psychiatry and Psychotherapy, Charité – Universitätsmedizin BerlinBerlinGermany
| | - Rani Moran
- Max Planck UCL Centre for Computational Psychiatry and Ageing ResearchLondonUnited Kingdom
- Wellcome Trust Centre for Neuroimaging, University College LondonLondonUnited Kingdom
| | - Ray J Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing ResearchLondonUnited Kingdom
- Wellcome Trust Centre for Neuroimaging, University College LondonLondonUnited Kingdom
| | - Tobias U Hauser
- Max Planck UCL Centre for Computational Psychiatry and Ageing ResearchLondonUnited Kingdom
- Wellcome Trust Centre for Neuroimaging, University College LondonLondonUnited Kingdom
| |
Collapse
|
41
|
Mosley PE, Akram H. Neuropsychiatric effects of subthalamic deep brain stimulation. THE HUMAN HYPOTHALAMUS - MIDDLE AND POSTERIOR REGION 2021; 180:417-431. [DOI: 10.1016/b978-0-12-820107-7.00026-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/11/2023]
|
42
|
Seymour B, Mancini F. Hierarchical models of pain: Inference, information-seeking, and adaptive control. Neuroimage 2020; 222:117212. [PMID: 32739554 DOI: 10.1016/j.neuroimage.2020.117212] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2020] [Revised: 07/21/2020] [Accepted: 07/25/2020] [Indexed: 11/26/2022] Open
Abstract
Computational models of pain consider how the brain processes nociceptive information and allow mapping neural circuits and networks to cognition and behaviour. To date, they have generally have assumed two largely independent processes: perceptual inference, typically modelled as an approximate Bayesian process, and action control, typically modelled as a reinforcement learning process. However, inference and control are intertwined in complex ways, challenging the clarity of this distinction. Here, we consider how they may comprise a parallel hierarchical architecture that combines inference, information-seeking, and adaptive value-based control. This sheds light on the complex neural architecture of the pain system, and takes us closer to understanding from where pain 'arises' in the brain.
Collapse
Affiliation(s)
- Ben Seymour
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, United Kingdom; Center for Information and Neural Networks, National Institute of Information and Communications Technology, Osaka, Japan.
| | - Flavia Mancini
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, United Kingdom.
| |
Collapse
|
43
|
Understanding exploration in humans and machines by formalizing the function of curiosity. Curr Opin Behav Sci 2020. [DOI: 10.1016/j.cobeha.2020.07.008] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
44
|
|
45
|
Ernst B, Steinhauser M. The effect of feedback novelty on neural correlates of feedback processing. Brain Cogn 2020; 144:105610. [PMID: 32777688 DOI: 10.1016/j.bandc.2020.105610] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2020] [Revised: 07/02/2020] [Accepted: 07/24/2020] [Indexed: 01/17/2023]
Abstract
It has been suggested that stimulus novelty itself can be rewarding and recent evidence suggests that novelty processing and reward processing share common neural mechanisms. For feedback processing, this can be beneficial as well as detrimental: If novelty lends a rewarding characteristic to a stimulus, then this should particularly decrease the impact of negative feedback. The present study investigated whether such an effect of feedback novelty on feedback processing is reflected in electrophysiological markers of reinforcement learning (feedback-related negativity, FRN) and feedback processing (feedback-P300) in a simple decision-making task. In this task, participants had to chose between two stimuli in a learning trial followed by a novel or a familiar feedback stimulus. Learning from feedback allowed them to optimize their payoff in a later test trial. As expected, we found that the FRN effect, i.e. the difference between the FRN amplitudes after negative and positive feedback, was reduced for novel compared to familiar feedback stimuli. In addition, the amplitude of the feedback-P300 was decreased by feedback novelty, both for the anterior P3a and the posterior P3b. Together, these results indicate that feedback novelty can affect feedback processing as reflected by feedback-related brain activity.
Collapse
|
46
|
Dipasquale O, Martins D, Sethi A, Veronese M, Hesse S, Rullmann M, Sabri O, Turkheimer F, Harrison NA, Mehta MA, Cercignani M. Unravelling the effects of methylphenidate on the dopaminergic and noradrenergic functional circuits. Neuropsychopharmacology 2020; 45:1482-1489. [PMID: 32473593 PMCID: PMC7360745 DOI: 10.1038/s41386-020-0724-x] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/09/2020] [Revised: 04/30/2020] [Accepted: 05/15/2020] [Indexed: 11/08/2022]
Abstract
Functional magnetic resonance imaging (fMRI) can be combined with drugs to investigate the system-level functional responses in the brain to such challenges. However, most psychoactive agents act on multiple neurotransmitters, limiting the ability of fMRI to identify functional effects related to actions on discrete pharmacological targets. We recently introduced a multimodal approach, REACT (Receptor-Enriched Analysis of functional Connectivity by Targets), which offers the opportunity to disentangle effects of drugs on different neurotransmitters and clarify the biological mechanisms driving clinical efficacy and side effects of a compound. Here, we focus on methylphenidate (MPH), which binds to the dopamine transporter (DAT) and the norepinephrine transporter (NET), to unravel its effects on dopaminergic and noradrenergic functional circuits in the healthy brain at rest. We then explored the relationship between these target-enriched resting state functional connectivity (FC) maps and inter-individual variability in behavioural responses to a reinforcement-learning task encompassing a novelty manipulation to disentangle the molecular systems underlying specific cognitive/behavioural effects. Our main analysis showed a significant MPH-induced FC increase in sensorimotor areas in the functional circuit associated with DAT. In our exploratory analysis, we found that MPH-induced regional variations in the DAT and NET-enriched FC maps were significantly correlated with some of the inter-individual differences on key behavioural responses associated with the reinforcement-learning task. Our findings show that main MPH-related FC changes at rest can be understood through the distribution of DAT in the brain. Furthermore, they suggest that when compounds have mixed pharmacological profiles, REACT may be able to capture regional functional effects that are underpinned by the same cognitive mechanism but are related to engagement of distinct molecular targets.
Collapse
Affiliation(s)
- Ottavia Dipasquale
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK.
| | - Daniel Martins
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Arjun Sethi
- Forensic & Neurodevelopmental Sciences, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Mattia Veronese
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Swen Hesse
- Department of Nuclear Medicine, University of Leipzig, Leipzig, Germany
- Integrated Research and Treatment Center (IFB) Adiposity Diseases, Leipzig University Medical Center, Leipzig, Germany
| | - Michael Rullmann
- Department of Nuclear Medicine, University of Leipzig, Leipzig, Germany
- Integrated Research and Treatment Center (IFB) Adiposity Diseases, Leipzig University Medical Center, Leipzig, Germany
| | - Osama Sabri
- Department of Nuclear Medicine, University of Leipzig, Leipzig, Germany
| | - Federico Turkheimer
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Neil A Harrison
- Cardiff University Brain Research Imaging Centre, Cardiff University, Cardiff, UK
| | - Mitul A Mehta
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Mara Cercignani
- Clinical Imaging Sciences Centre, Brighton and Sussex Medical School, Brighton, UK
| |
Collapse
|
47
|
Gregory DF, Ritchey M, Murty VP. Amygdala and ventral tegmental area differentially interact with hippocampus and cortical medial temporal lobe during rest in humans. Hippocampus 2020; 30:1073-1080. [PMID: 32485015 DOI: 10.1002/hipo.23216] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2019] [Revised: 12/29/2019] [Accepted: 04/25/2020] [Indexed: 12/15/2022]
Abstract
Neuromodulatory regions that detect salience, such as amygdala and ventral tegmental area (VTA), have distinct effects on memory. Yet, questions remain about how these modulatory regions target subregions across the hippocampus and medial temporal lobe (MTL) cortex. Here, we sought to characterize how VTA and amygdala subregions (i.e., basolateral amygdala and central-medial amygdala) interact with hippocampus head, body, and tail, as well as cortical MTL areas of perirhinal cortex and parahippocampal cortex in a task-free state. To quantify these interactions, we used high-resolution resting state fMRI and characterized pair-wise, partial correlations across regions-of-interest. We found that basolateral amygdala showed greater functional coupling with hippocampus head, hippocampus tail, and perirhinal cortex when compared to either VTA or central-medial amygdala. Furthermore, the VTA showed greater functional coupling with hippocampus tail when compared to central-medial amygdala. There were no significant differences in functional coupling with hippocampus body and parahippocampal cortex. These results support a framework by which neuromodulatory regions do not indiscriminately influence all MTL subregions equally, but rather bias information processing to discrete MTL targets. These findings provide a more specified model of the intrinsic properties of systems underlying MTL neuromodulation. This emphasizes the need to consider heterogeneity both across and within neuromodulatory systems to better understand affective memory.
Collapse
Affiliation(s)
- David F Gregory
- Department of Psychology, Temple University, Philadelphia, Pennsylvania, USA
| | - Maureen Ritchey
- Department of Psychology and Neuroscience, Boston College, Chestnut Hill, Massachusetts, USA
| | - Vishnu P Murty
- Department of Psychology, Temple University, Philadelphia, Pennsylvania, USA
| |
Collapse
|
48
|
Reichardt R, Polner B, Simor P. Novelty Manipulations, Memory Performance, and Predictive Coding: the Role of Unexpectedness. Front Hum Neurosci 2020; 14:152. [PMID: 32410975 PMCID: PMC7201021 DOI: 10.3389/fnhum.2020.00152] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2020] [Accepted: 04/06/2020] [Indexed: 12/15/2022] Open
Abstract
Novelty is central to the study of memory, but the wide range of experimental manipulations aimed to reveal its effects on learning produced inconsistent results. The novelty/encoding hypothesis suggests that novel information undergoes enhanced encoding and thus leads to benefits in memory, especially in recognition performance; however, recent studies cast doubts on this assumption. On the other hand, data from animal studies provided evidence on the robust effects of novelty manipulations on the neurophysiological correlates of memory processes. Conceptualizations and operationalizations of novelty are remarkably variable and were categorized into different subtypes, such as stimulus, context, associative or spatial novelty. Here, we summarize previous findings about the effects of novelty on memory and suggest that predictive coding theories provide a framework that could shed light on the differential influence of novelty manipulations on memory performance. In line with predictive coding theories, we emphasize the role of unexpectedness as a crucial property mediating the behavioral and neural effects of novelty manipulations.
Collapse
Affiliation(s)
- Richárd Reichardt
- Department of Cognitive Science, Budapest University of Technology and Economics, Budapest, Hungary.,Institute of Psychology, Eötvös Lóránd University, Budapest, Hungary
| | - Bertalan Polner
- Department of Cognitive Science, Budapest University of Technology and Economics, Budapest, Hungary
| | - Péter Simor
- Institute of Psychology, Eötvös Lóránd University, Budapest, Hungary.,Institute of Behavioural Sciences, Semmelweis University, Budapest, Hungary.,UR2NF, Neuropsychology and Functional Neuroimaging Research Unit, CRCN-Center for Research in Cognition and Neurosciences and UNI-ULB Neurosciences Institute, Université Libre de Bruxelles (ULB), Brussels, Belgium
| |
Collapse
|
49
|
Primate Orbitofrontal Cortex Codes Information Relevant for Managing Explore-Exploit Tradeoffs. J Neurosci 2020; 40:2553-2561. [PMID: 32060169 DOI: 10.1523/jneurosci.2355-19.2020] [Citation(s) in RCA: 38] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2019] [Revised: 01/26/2020] [Accepted: 02/09/2020] [Indexed: 11/21/2022] Open
Abstract
Reinforcement learning (RL) refers to the behavioral process of learning to obtain reward and avoid punishment. An important component of RL is managing explore-exploit tradeoffs, which refers to the problem of choosing between exploiting options with known values and exploring unfamiliar options. We examined correlates of this tradeoff, as well as other RL related variables, in orbitofrontal cortex (OFC) while three male monkeys performed a three-armed bandit learning task. During the task, novel choice options periodically replaced familiar options. The values of the novel options were unknown, and the monkeys had to explore them to see if they were better than other currently available options. The identity of the chosen stimulus and the reward outcome were strongly encoded in the responses of single OFC neurons. These two variables define the states and state transitions in our model that are relevant to decision-making. The chosen value of the option and the relative value of exploring that option were encoded at intermediate levels. We also found that OFC value coding was stimulus specific, as opposed to coding value independent of the identity of the option. The location of the option and the value of the current environment were encoded at low levels. Therefore, we found encoding of the variables relevant to learning and managing explore-exploit tradeoffs in OFC. These results are consistent with findings in the ventral striatum and amygdala and show that this monosynaptically connected network plays an important role in learning based on the immediate and future consequences of choices.SIGNIFICANCE STATEMENT Orbitofrontal cortex (OFC) has been implicated in representing the expected values of choices. Here we extend these results and show that OFC also encodes information relevant to managing explore-exploit tradeoffs. Specifically, OFC encodes an exploration bonus, which characterizes the relative value of exploring novel choice options. OFC also strongly encodes the identity of the chosen stimulus, and reward outcomes, which are necessary for computing the value of novel and familiar options.
Collapse
|
50
|
Rigoli F, Martinelli C, Shergill SS. The role of expecting feedback during decision-making under risk. Neuroimage 2019; 202:116079. [PMID: 31400531 DOI: 10.1016/j.neuroimage.2019.116079] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2019] [Revised: 07/29/2019] [Accepted: 08/06/2019] [Indexed: 11/19/2022] Open
Abstract
Sometimes choice is followed by outcome feedback and other times it is not. It remains unknown whether humans prefer gambling when they expect feedback to be revealed. Regarding this question, decision-making theories make alternative predictions. Some theories have proposed that choice is influenced by whether one expects to be disappointed in the future. Given that feedback is sometimes disappointing, these theories predict increased aversion towards gambling when feedback is expected compared to when feedback is not expected. The opposite effect is predicted by theories of curiosity, which postulate reduction of uncertainty as an important behavioural drive. Given that feedback reduces uncertainty, these theories predict that gambling will be favoured when feedback is expected. To examine whether expecting feedback influences gambling behaviour, we recorded functional neuroimaging data while participants performed a novel decision-making task requiring to chose between a sure option and a gamble. Crucially, participants expected to receive feedback in some trials but not in other trials. Consistent with theories of curiosity, we found that expecting feedback increased gambling propensity. At the neural level, at option presentation the increased value of gambling during feedback was reflected in activity in the ventral striatum. This suggests that, together with its established role in signalling reward, the ventral striatum also processes a form of epistemic value. Our study demonstrates that gambling becomes more attractive when feedback is expected and suggests that striatal activity could signal the value of feedback information.
Collapse
Affiliation(s)
- Francesco Rigoli
- City, University of London, Northampton Square, London, EC1V 0HB, UK; The Wellcome Trust Centre for Neuroimaging, UCL, 12 Queen Square, London, WC1N 3BG, UK.
| | - Cristina Martinelli
- Institute of Psychiatry, Psychology & Neuroscience, King's College London, 16 De Crespigny Park Road, London, SE5 8AF, UK; Kingston University, Penrhyn Road, Kingston Upon Thames, Surrey, KT1 2EE, UK
| | - Sukhwinder S Shergill
- Institute of Psychiatry, Psychology & Neuroscience, King's College London, 16 De Crespigny Park Road, London, SE5 8AF, UK
| |
Collapse
|