1
|
Man V, Cockburn J, Flouty O, Gander PE, Sawada M, Kovach CK, Kawasaki H, Oya H, Howard Iii MA, O'Doherty JP. Temporally organized representations of reward and risk in the human brain. Nat Commun 2024; 15:2162. [PMID: 38461343 PMCID: PMC10924934 DOI: 10.1038/s41467-024-46094-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Accepted: 02/13/2024] [Indexed: 03/11/2024] Open
Abstract
The value and uncertainty associated with choice alternatives constitute critical features relevant for decisions. However, the manner in which reward and risk representations are temporally organized in the brain remains elusive. Here we leverage the spatiotemporal precision of intracranial electroencephalography, along with a simple card game designed to elicit the unfolding computation of a set of reward and risk variables, to uncover this temporal organization. Reward outcome representations across wide-spread regions follow a sequential order along the anteroposterior axis of the brain. In contrast, expected value can be decoded from multiple regions at the same time, and error signals in both reward and risk domains reflect a mixture of sequential and parallel encoding. We further highlight the role of the anterior insula in generalizing between reward prediction error and risk prediction error codes. Together our results emphasize the importance of neural dynamics for understanding value-based decisions under uncertainty.
Collapse
Affiliation(s)
- Vincent Man
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, 91125, USA.
| | - Jeffrey Cockburn
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, 91125, USA
| | - Oliver Flouty
- Department of Neurosurgery and Brain Repair, University of South Florida, Tampa, FL, 33606, USA
| | - Phillip E Gander
- Department of Neurosurgery, University of Iowa Hospitals and Clinics, Iowa City, IA, 52242, USA
- Department of Radiology, University of Iowa Hospitals and Clinics, Iowa City, IA, 52242, USA
- Iowa Neuroscience Institute, University of Iowa Carver College of Medicine, Iowa City, IA, 52242, USA
| | - Masahiro Sawada
- Department of Neurosurgery, University of Iowa Hospitals and Clinics, Iowa City, IA, 52242, USA
| | - Christopher K Kovach
- Department of Neurosurgery, University of Iowa Hospitals and Clinics, Iowa City, IA, 52242, USA
- Department of Neurosurgery, University of Nebraska Medical Center, Omaha, NE, 68198, USA
| | - Hiroto Kawasaki
- Department of Neurosurgery, University of Iowa Hospitals and Clinics, Iowa City, IA, 52242, USA
| | - Hiroyuki Oya
- Department of Neurosurgery, University of Iowa Hospitals and Clinics, Iowa City, IA, 52242, USA
- Iowa Neuroscience Institute, University of Iowa Carver College of Medicine, Iowa City, IA, 52242, USA
| | - Matthew A Howard Iii
- Department of Neurosurgery, University of Iowa Hospitals and Clinics, Iowa City, IA, 52242, USA
- Iowa Neuroscience Institute, University of Iowa Carver College of Medicine, Iowa City, IA, 52242, USA
| | - John P O'Doherty
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, 91125, USA
- Computation and Neural Systems, California Institute of Technology, Pasadena, CA, 91125, USA
| |
Collapse
|
2
|
Çakar T, Son-Turan S, Girişken Y, Sayar A, Ertuğrul S, Filiz G, Tuna E. Unlocking the neural mechanisms of consumer loan evaluations: an fNIRS and ML-based consumer neuroscience study. Front Hum Neurosci 2024; 18:1286918. [PMID: 38375365 PMCID: PMC10875049 DOI: 10.3389/fnhum.2024.1286918] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Accepted: 01/11/2024] [Indexed: 02/21/2024] Open
Abstract
Introduction This study conducts a comprehensive exploration of the neurocognitive processes underlying consumer credit decision-making using cutting-edge techniques from neuroscience and machine learning (ML). Employing functional Near-Infrared Spectroscopy (fNIRS), the research examines the hemodynamic responses of participants while evaluating diverse credit offers. Methods The experimental phase of this study investigates the hemodynamic responses collected from 39 healthy participants with respect to different loan offers. This study integrates fNIRS data with advanced ML algorithms, specifically Extreme Gradient Boosting, CatBoost, Extra Tree Classifier, and Light Gradient Boosted Machine, to predict participants' credit decisions based on prefrontal cortex (PFC) activation patterns. Results Findings reveal distinctive PFC regions correlating with credit behaviors, including the dorsolateral prefrontal cortex (dlPFC) associated with strategic decision-making, the orbitofrontal cortex (OFC) linked to emotional valuations, and the ventromedial prefrontal cortex (vmPFC) reflecting brand integration and reward processing. Notably, the right dorsomedial prefrontal cortex (dmPFC) and the right vmPFC contribute to positive credit preferences. Discussion This interdisciplinary approach bridges neuroscience, machine learning and finance, offering unprecedented insights into the neural mechanisms guiding financial choices regarding different loan offers. The study's predictive model holds promise for refining financial services and illuminating human financial behavior within the burgeoning field of neurofinance. The work exemplifies the potential of interdisciplinary research to enhance our understanding of human financial decision-making.
Collapse
Affiliation(s)
- Tuna Çakar
- Department of Computer Engineering, MEF University, Istanbul, Türkiye
| | - Semen Son-Turan
- Department of Business Administration, MEF University, Maslak, Türkiye
| | - Yener Girişken
- Faculty of Economics and Administrative Sciences, Final International University, Istanbul, Türkiye
| | - Alperen Sayar
- Informatics Technologies Master Program, MEF University, Istanbul, Türkiye
| | - Seyit Ertuğrul
- Informatics Technologies Master Program, MEF University, Istanbul, Türkiye
| | - Gözde Filiz
- Computer Science and Engineering Ph.D. Program, MEF University, Istanbul, Türkiye
| | - Esin Tuna
- Department of Psychology, MEF University, Istanbul, Türkiye
| |
Collapse
|
3
|
De Filippo R, Schmitz D. Synthetic surprise as the foundation of the psychedelic experience. Neurosci Biobehav Rev 2024; 157:105538. [PMID: 38220035 PMCID: PMC10839673 DOI: 10.1016/j.neubiorev.2024.105538] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 01/03/2024] [Accepted: 01/05/2024] [Indexed: 01/16/2024]
Abstract
Psychedelic agents, such as LSD and psilocybin, induce marked alterations in consciousness via activation of the 5-HT2A receptor (5-HT2ARs). We hypothesize that psychedelics enforce a state of synthetic surprise through the biased activation of the 5-HTRs system. This idea is informed by recent insights into the role of 5-HT in signaling surprise. The effects on consciousness, explained by the cognitive penetrability of perception, can be described within the predictive coding framework where surprise corresponds to prediction error, the mismatch between predictions and actual sensory input. Crucially, the precision afforded to the prediction error determines its effect on priors, enabling a dynamic interaction between top-down expectations and incoming sensory data. By integrating recent findings on predictive coding circuitry and 5-HT2ARs transcriptomic data, we propose a biological implementation with emphasis on the role of inhibitory interneurons. Implications arise for the clinical use of psychedelics, which may rely primarily on their inherent capacity to induce surprise in order to disrupt maladaptive patterns.
Collapse
Affiliation(s)
- Roberto De Filippo
- Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität Berlin, and Berlin Institute of Health, Neuroscience Research Center, 10117 Berlin, Germany.
| | - Dietmar Schmitz
- Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität Berlin, and Berlin Institute of Health, Neuroscience Research Center, 10117 Berlin, Germany; German Center for Neurodegenerative Diseases (DZNE) Berlin, 10117 Berlin, Germany; Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität Berlin, and Berlin Institute of Health, Einstein Center for Neuroscience, 10117 Berlin, Germany; Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität Berlin, and Berlin Institute of Health, NeuroCure Cluster of Excellence, 10117 Berlin, Germany; Humboldt-Universität zu Berlin, Bernstein Center for Computational Neuroscience, Philippstr. 13, 10115 Berlin, Germany
| |
Collapse
|
4
|
Hallock HL, Adiraju SS, Miranda-Barrientos J, McInerney JM, Oh S, DeBrosse AC, Li Y, Carr GV, Martinowich K. Electrophysiological correlates of attention in the locus coeruleus-prelimbic cortex circuit during the rodent continuous performance test. Neuropsychopharmacology 2024; 49:521-531. [PMID: 37563281 PMCID: PMC10789747 DOI: 10.1038/s41386-023-01692-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Revised: 07/19/2023] [Accepted: 07/24/2023] [Indexed: 08/12/2023]
Abstract
Sustained attention, the ability to focus on an activity or stimulus over time, is significantly impaired in many psychiatric disorders, and there remains a major unmet need in treating impaired attention. Continuous performance tests (CPTs) were developed to measure sustained attention in humans, non-human primates, rats, and mice, and similar neural circuits are engaged across species during CPT performance, supporting their use in translational studies to identify novel therapeutics. Here, we identified electrophysiological correlates of attentional performance in a touchscreen-based rodent CPT (rCPT) in the locus coeruleus (LC) and prelimbic cortex (PrL), two inter-connected regions that are implicated in attentional processes. We used viral labeling and molecular techniques to demonstrate that neural activity is recruited in LC-PrL projections during the rCPT, and that this recruitment increases with cognitive demand. We implanted male mice with depth electrodes within the LC and PrL for local field potential (LFP) recordings during rCPT training, and identified an increase in PrL delta and theta power, and an increase in LC delta power during correct responses in the rCPT. We also found that the LC leads the PrL in theta frequencies during correct responses while the PrL leads the LC in gamma frequencies during incorrect responses. These findings may represent translational biomarkers that can be used to screen novel therapeutics for drug discovery in attention.
Collapse
Affiliation(s)
- Henry L Hallock
- Neuroscience Program, Lafayette College, Easton, PA, 18042, USA
| | - Suhaas S Adiraju
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA
| | | | - Jessica M McInerney
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA
| | - Seyun Oh
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA
| | - Adrienne C DeBrosse
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA
| | - Ye Li
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA
| | - Gregory V Carr
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA.
- Department of Pharmacology and Molecular Sciences, Johns Hopkins University School of Medicine, Baltimore, MD, 21205, USA.
| | - Keri Martinowich
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA.
- Department of Psychiatry and Behavioral Sciences, Johns Hopkins School of Medicine, Baltimore, MD, 21205, USA.
- Department of Neuroscience, Johns Hopkins School of Medicine, Baltimore, MD, 21205, USA.
- The Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD, 21205, USA.
| |
Collapse
|
5
|
Prat-Carrabin A, Meyniel F, Azeredo da Silveira R. Resource-rational account of sequential effects in human prediction. eLife 2024; 13:e81256. [PMID: 38224341 PMCID: PMC10789490 DOI: 10.7554/elife.81256] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Accepted: 12/11/2023] [Indexed: 01/16/2024] Open
Abstract
An abundant literature reports on 'sequential effects' observed when humans make predictions on the basis of stochastic sequences of stimuli. Such sequential effects represent departures from an optimal, Bayesian process. A prominent explanation posits that humans are adapted to changing environments, and erroneously assume non-stationarity of the environment, even if the latter is static. As a result, their predictions fluctuate over time. We propose a different explanation in which sub-optimal and fluctuating predictions result from cognitive constraints (or costs), under which humans however behave rationally. We devise a framework of costly inference, in which we develop two classes of models that differ by the nature of the constraints at play: in one case the precision of beliefs comes at a cost, resulting in an exponential forgetting of past observations, while in the other beliefs with high predictive power are favored. To compare model predictions to human behavior, we carry out a prediction task that uses binary random stimuli, with probabilities ranging from 0.05 to 0.95. Although in this task the environment is static and the Bayesian belief converges, subjects' predictions fluctuate and are biased toward the recent stimulus history. Both classes of models capture this 'attractive effect', but they depart in their characterization of higher-order effects. Only the precision-cost model reproduces a 'repulsive effect', observed in the data, in which predictions are biased away from stimuli presented in more distant trials. Our experimental results reveal systematic modulations in sequential effects, which our theoretical approach accounts for in terms of rationality under cognitive constraints.
Collapse
Affiliation(s)
- Arthur Prat-Carrabin
- Department of Economics, Columbia UniversityNew YorkUnited States
- Laboratoire de Physique de l’École Normale Supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université de ParisParisFrance
| | - Florent Meyniel
- Cognitive Neuroimaging Unit, Institut National de la Santé et de la Recherche Médicale, Commissariat à l’Energie Atomique et aux Energies Alternatives, Centre National de la Recherche Scientifique, Université Paris-Saclay, NeuroSpin centerGif-sur-YvetteFrance
- Institut de neuromodulation, GHU Paris, Psychiatrie et Neurosciences, Centre Hospitalier Sainte-Anne, Pôle Hospitalo-Universitaire 15, Université Paris CitéParisFrance
| | - Rava Azeredo da Silveira
- Laboratoire de Physique de l’École Normale Supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université de ParisParisFrance
- Institute of Molecular and Clinical Ophthalmology BaselBaselSwitzerland
- Faculty of Science, University of BaselBaselSwitzerland
| |
Collapse
|
6
|
Valdebenito-Oyarzo G, Martínez-Molina MP, Soto-Icaza P, Zamorano F, Figueroa-Vargas A, Larraín-Valenzuela J, Stecher X, Salinas C, Bastin J, Valero-Cabré A, Polania R, Billeke P. The parietal cortex has a causal role in ambiguity computations in humans. PLoS Biol 2024; 22:e3002452. [PMID: 38198502 PMCID: PMC10824459 DOI: 10.1371/journal.pbio.3002452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Revised: 01/23/2024] [Accepted: 11/28/2023] [Indexed: 01/12/2024] Open
Abstract
Humans often face the challenge of making decisions between ambiguous options. The level of ambiguity in decision-making has been linked to activity in the parietal cortex, but its exact computational role remains elusive. To test the hypothesis that the parietal cortex plays a causal role in computing ambiguous probabilities, we conducted consecutive fMRI and TMS-EEG studies. We found that participants assigned unknown probabilities to objective probabilities, elevating the uncertainty of their decisions. Parietal cortex activity correlated with the objective degree of ambiguity and with a process that underestimates the uncertainty during decision-making. Conversely, the midcingulate cortex (MCC) encodes prediction errors and increases its connectivity with the parietal cortex during outcome processing. Disruption of the parietal activity increased the uncertainty evaluation of the options, decreasing cingulate cortex oscillations during outcome evaluation and lateral frontal oscillations related to value ambiguous probability. These results provide evidence for a causal role of the parietal cortex in computing uncertainty during ambiguous decisions made by humans.
Collapse
Affiliation(s)
- Gabriela Valdebenito-Oyarzo
- Laboratorio de Neurociencia Social y Neuromodulación, Centro de Investigación en Complejidad Social, (neuroCICS), Facultad de Gobierno, Universidad del Desarrollo, Santiago, Chile
| | - María Paz Martínez-Molina
- Laboratorio de Neurociencia Social y Neuromodulación, Centro de Investigación en Complejidad Social, (neuroCICS), Facultad de Gobierno, Universidad del Desarrollo, Santiago, Chile
| | - Patricia Soto-Icaza
- Laboratorio de Neurociencia Social y Neuromodulación, Centro de Investigación en Complejidad Social, (neuroCICS), Facultad de Gobierno, Universidad del Desarrollo, Santiago, Chile
| | - Francisco Zamorano
- Unidad de Neuroimágenes Cuantitativas avanzadas (UNICA), Departamento de Imágenes, Clínica Alemana de Santiago, Santiago, Chile
- Facultad de Ciencias para el Cuidado de la Salud, Campus Los Leones, Universidad San Sebastián, Santiago, Chile
| | - Alejandra Figueroa-Vargas
- Laboratorio de Neurociencia Social y Neuromodulación, Centro de Investigación en Complejidad Social, (neuroCICS), Facultad de Gobierno, Universidad del Desarrollo, Santiago, Chile
| | - Josefina Larraín-Valenzuela
- Laboratorio de Neurociencia Social y Neuromodulación, Centro de Investigación en Complejidad Social, (neuroCICS), Facultad de Gobierno, Universidad del Desarrollo, Santiago, Chile
| | - Ximena Stecher
- Unidad de Neuroimágenes Cuantitativas avanzadas (UNICA), Departamento de Imágenes, Clínica Alemana de Santiago, Santiago, Chile
| | - César Salinas
- Unidad de Neuroimágenes Cuantitativas avanzadas (UNICA), Departamento de Imágenes, Clínica Alemana de Santiago, Santiago, Chile
| | - Julien Bastin
- Univ. Grenoble Alpes, Inserm, U1216, Grenoble Institut Neurosciences, Grenoble, France
| | - Antoni Valero-Cabré
- Causal Dynamics, Plasticity and Rehabilitation Group, FRONTLAB team, Institut du Cerveau et de la Moelle Epinière (ICM), CNRS UMR 7225, INSERM U 1127 and Sorbonne Université, Paris, France
- Cognitive Neuroscience and Information Technology Research Program, Open University of Catalonia (UOC), Barcelona, Spain
- Laboratory for Cerebral Dynamics Plasticity and Rehabilitation, Boston University, School of Medicine, Boston, Massachusetts, United States of America
| | - Rafael Polania
- Decision Neuroscience Lab, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland
| | - Pablo Billeke
- Laboratorio de Neurociencia Social y Neuromodulación, Centro de Investigación en Complejidad Social, (neuroCICS), Facultad de Gobierno, Universidad del Desarrollo, Santiago, Chile
| |
Collapse
|
7
|
Carvalheiro J, Philiastides MG. Distinct spatiotemporal brainstem pathways of outcome valence during reward- and punishment-based learning. Cell Rep 2023; 42:113589. [PMID: 38100353 DOI: 10.1016/j.celrep.2023.113589] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Revised: 10/05/2023] [Accepted: 11/30/2023] [Indexed: 12/17/2023] Open
Abstract
Learning to seek rewards and avoid punishments, based on positive and negative choice outcomes, is essential for human survival. Yet, the neural underpinnings of outcome valence in the human brainstem and the extent to which they differ in reward and punishment learning contexts remain largely elusive. Here, using simultaneously acquired electroencephalography and functional magnetic resonance imaging data, we show that during reward learning the substantia nigra (SN)/ventral tegmental area (VTA) and locus coeruleus are initially activated following negative outcomes, while the VTA subsequently re-engages exhibiting greater responses for positive than negative outcomes, consistent with an early arousal/avoidance response and a later value-updating process, respectively. During punishment learning, we show that distinct raphe nucleus and SN subregions are activated only by negative outcomes with a sustained post-outcome activity across time, supporting the involvement of these brainstem subregions in avoidance behavior. Finally, we demonstrate that the coupling of these brainstem structures with other subcortical and cortical areas helps to shape participants' serial choice behavior in each context.
Collapse
Affiliation(s)
- Joana Carvalheiro
- School of Psychology and Neuroscience, University of Glasgow, Glasgow G12 8QB, UK; Centre for Cognitive Neuroimaging, University of Glasgow, Glasgow G12 8QB, UK.
| | - Marios G Philiastides
- School of Psychology and Neuroscience, University of Glasgow, Glasgow G12 8QB, UK; Centre for Cognitive Neuroimaging, University of Glasgow, Glasgow G12 8QB, UK.
| |
Collapse
|
8
|
Narasimhan S, Schriver BJ, Wang Q. Adaptive decision-making depends on pupil-linked arousal in rats performing tactile discrimination tasks. J Neurophysiol 2023; 130:1541-1551. [PMID: 37964751 PMCID: PMC11068411 DOI: 10.1152/jn.00309.2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2022] [Revised: 11/08/2023] [Accepted: 11/09/2023] [Indexed: 11/16/2023] Open
Abstract
Perceptual decision-making is a dynamic cognitive process and is shaped by many factors, including behavioral state, reward contingency, and sensory environment. To understand the extent to which adaptive behavior in decision-making is dependent on pupil-linked arousal, we trained head-fixed rats to perform perceptual decision-making tasks and systematically manipulated the probability of Go and No-go stimuli while simultaneously measuring their pupil size in the tasks. Our data demonstrated that the animals adaptively modified their behavior in response to the changes in the sensory environment. The response probability to both Go and No-go stimuli decreased as the probability of the Go stimulus being presented decreased. Analyses within the signal detection theory framework showed that while the animals' perceptual sensitivity was invariant, their decision criterion increased as the probability of the Go stimulus decreased. Simulation results indicated that the adaptive increase in the decision criterion will increase possible water rewards during the task. Moreover, the adaptive decision-making is dependent on pupil-linked arousal as the increase in the decision criterion was the largest during low pupil-linked arousal periods. Taken together, our results demonstrated that the rats were able to adjust their decision-making to maximize rewards in the tasks, and that adaptive behavior in perceptual decision-making is dependent on pupil-linked arousal.NEW & NOTEWORTHY Perceptual decision-making is a dynamic cognitive process and is shaped by many factors. However, the extent to which changes in sensory environment result in adaptive decision-making remains poorly understood. Our data provided new experimental evidence demonstrating that the rats were able to adaptively modify their decision criterion to maximize water reward in response to changes in the statistics of the sensory environment. Furthermore, the adaptive decision-making is dependent on pupil-linked arousal.
Collapse
Affiliation(s)
- Shreya Narasimhan
- Department of Biomedical Engineering, Columbia University, New York City, New York, United States
| | - Brian J Schriver
- Department of Biomedical Engineering, Columbia University, New York City, New York, United States
| | - Qi Wang
- Department of Biomedical Engineering, Columbia University, New York City, New York, United States
| |
Collapse
|
9
|
Walker EY, Pohl S, Denison RN, Barack DL, Lee J, Block N, Ma WJ, Meyniel F. Studying the neural representations of uncertainty. Nat Neurosci 2023; 26:1857-1867. [PMID: 37814025 DOI: 10.1038/s41593-023-01444-y] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Accepted: 08/30/2023] [Indexed: 10/11/2023]
Abstract
The study of the brain's representations of uncertainty is a central topic in neuroscience. Unlike most quantities of which the neural representation is studied, uncertainty is a property of an observer's beliefs about the world, which poses specific methodological challenges. We analyze how the literature on the neural representations of uncertainty addresses those challenges and distinguish between 'code-driven' and 'correlational' approaches. Code-driven approaches make assumptions about the neural code for representing world states and the associated uncertainty. By contrast, correlational approaches search for relationships between uncertainty and neural activity without constraints on the neural representation of the world state that this uncertainty accompanies. To compare these two approaches, we apply several criteria for neural representations: sensitivity, specificity, invariance and functionality. Our analysis reveals that the two approaches lead to different but complementary findings, shaping new research questions and guiding future experiments.
Collapse
Affiliation(s)
- Edgar Y Walker
- Department of Physiology and Biophysics, Computational Neuroscience Center, University of Washington, Seattle, WA, USA
| | - Stephan Pohl
- Department of Philosophy, New York University, New York, NY, USA
| | - Rachel N Denison
- Department of Psychological & Brain Sciences, Boston University, Boston, MA, USA
| | - David L Barack
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA, USA
- Department of Philosophy, University of Pennsylvania, Philadelphia, PA, USA
| | - Jennifer Lee
- Center for Neural Science, New York University, New York, NY, USA
| | - Ned Block
- Department of Philosophy, New York University, New York, NY, USA
| | - Wei Ji Ma
- Center for Neural Science, New York University, New York, NY, USA
- Department of Psychology, New York University, New York, NY, USA
| | - Florent Meyniel
- Cognitive Neuroimaging Unit, INSERM, CEA, CNRS, Université Paris-Saclay, NeuroSpin center, Gif-sur-Yvette, France.
| |
Collapse
|
10
|
Nussenbaum K, Martin RE, Maulhardt S, Yang Y(J, Bizzell-Hatcher G, Bhatt NS, Koenig M, Rosenbaum GM, O'Doherty JP, Cockburn J, Hartley CA. Novelty and uncertainty differentially drive exploration across development. eLife 2023; 12:e84260. [PMID: 37585251 PMCID: PMC10431916 DOI: 10.7554/elife.84260] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Accepted: 08/07/2023] [Indexed: 08/17/2023] Open
Abstract
Across the lifespan, individuals frequently choose between exploiting known rewarding options or exploring unknown alternatives. A large body of work has suggested that children may explore more than adults. However, because novelty and reward uncertainty are often correlated, it is unclear how they differentially influence decision-making across development. Here, children, adolescents, and adults (ages 8-27 years, N = 122) completed an adapted version of a recently developed value-guided decision-making task that decouples novelty and uncertainty. In line with prior studies, we found that exploration decreased with increasing age. Critically, participants of all ages demonstrated a similar bias to select choice options with greater novelty, whereas aversion to reward uncertainty increased into adulthood. Computational modeling of participant choices revealed that whereas adolescents and adults demonstrated attenuated uncertainty aversion for more novel choice options, children's choices were not influenced by reward uncertainty.
Collapse
Affiliation(s)
| | | | - Sean Maulhardt
- New York UniversityNew YorkUnited States
- University of MarylandCollege ParkUnited States
| | - Yi (Jen) Yang
- New York UniversityNew YorkUnited States
- Temple UniversityPhiladelphiaUnited States
| | | | | | - Maximilian Koenig
- New York UniversityNew YorkUnited States
- Leiden UniversityLeidenNetherlands
| | - Gail M Rosenbaum
- New York UniversityNew YorkUnited States
- Geisinger Health SystemDanvilleUnited States
| | | | | | | |
Collapse
|
11
|
Sapey-Triomphe LA, Pattyn L, Weilnhammer V, Sterzer P, Wagemans J. Neural correlates of hierarchical predictive processes in autistic adults. Nat Commun 2023; 14:3640. [PMID: 37336874 DOI: 10.1038/s41467-023-38580-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Accepted: 05/08/2023] [Indexed: 06/21/2023] Open
Abstract
Bayesian theories of autism spectrum disorders (ASD) suggest that atypical predictive mechanisms could underlie the autistic symptomatology, but little is known about their neural correlates. Twenty-six neurotypical (NT) and 26 autistic adults participated in an fMRI study where they performed an associative learning task in a volatile environment. By inverting a model of perceptual inference, we characterized the neural correlates of hierarchically structured predictions and prediction errors in ASD. Behaviorally, the predictive abilities of autistic adults were intact. Neurally, predictions were encoded hierarchically in both NT and ASD participants and biased their percepts. High-level predictions were following activity levels in a set of regions more closely in ASD than NT. Prediction errors yielded activation in shared regions in NT and ASD, but group differences were found in the anterior cingulate cortex and putamen. This study sheds light on the neural specificities of ASD that might underlie atypical predictive processing.
Collapse
Affiliation(s)
- Laurie-Anne Sapey-Triomphe
- Department of Brain and Cognition, Leuven Brain Institute, KU Leuven, 3000, Leuven, Belgium.
- Leuven Autism Research (LAuRes), KU Leuven, 3000, Leuven, Belgium.
| | - Lauren Pattyn
- Department of Brain and Cognition, Leuven Brain Institute, KU Leuven, 3000, Leuven, Belgium
| | - Veith Weilnhammer
- Department of Psychiatry, Charité-Universitätsmedizin Berlin, 10117, Berlin, Germany
- Berlin Institute of Health, Charité-Universitätsmedizin Berlin, 10178, Berlin, Germany
| | - Philipp Sterzer
- Department of Psychiatry, Charité-Universitätsmedizin Berlin, 10117, Berlin, Germany
- Berlin Institute of Health, Charité-Universitätsmedizin Berlin, 10178, Berlin, Germany
| | - Johan Wagemans
- Department of Brain and Cognition, Leuven Brain Institute, KU Leuven, 3000, Leuven, Belgium
- Leuven Autism Research (LAuRes), KU Leuven, 3000, Leuven, Belgium
| |
Collapse
|
12
|
Woo JH, Aguirre CG, Bari BA, Tsutsui KI, Grabenhorst F, Cohen JY, Schultz W, Izquierdo A, Soltani A. Mechanisms of adjustments to different types of uncertainty in the reward environment across mice and monkeys. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2023; 23:600-619. [PMID: 36823249 PMCID: PMC10444905 DOI: 10.3758/s13415-022-01059-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 12/22/2022] [Indexed: 02/25/2023]
Abstract
Despite being unpredictable and uncertain, reward environments often exhibit certain regularities, and animals navigating these environments try to detect and utilize such regularities to adapt their behavior. However, successful learning requires that animals also adjust to uncertainty associated with those regularities. Here, we analyzed choice data from two comparable dynamic foraging tasks in mice and monkeys to investigate mechanisms underlying adjustments to different types of uncertainty. In these tasks, animals selected between two choice options that delivered reward probabilistically, while baseline reward probabilities changed after a variable number (block) of trials without any cues to the animals. To measure adjustments in behavior, we applied multiple metrics based on information theory that quantify consistency in behavior, and fit choice data using reinforcement learning models. We found that in both species, learning and choice were affected by uncertainty about reward outcomes (in terms of determining the better option) and by expectation about when the environment may change. However, these effects were mediated through different mechanisms. First, more uncertainty about the better option resulted in slower learning and forgetting in mice, whereas it had no significant effect in monkeys. Second, expectation of block switches accompanied slower learning, faster forgetting, and increased stochasticity in choice in mice, whereas it only reduced learning rates in monkeys. Overall, while demonstrating the usefulness of metrics based on information theory in examining adaptive behavior, our study provides evidence for multiple types of adjustments in learning and choice behavior according to uncertainty in the reward environment.
Collapse
Affiliation(s)
- Jae Hyung Woo
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Claudia G Aguirre
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Bilal A Bari
- Department of Psychiatry, Massachusetts General Hospital, Boston, MA, USA
| | - Ken-Ichiro Tsutsui
- Department of Physiology, Development & Neuroscience, University of Cambridge, Cambridge, UK
- Laboratory of Systems Neuroscience, Tohoku University Graduate School of Life Sciences, Sendai, Japan
| | - Fabian Grabenhorst
- Department of Physiology, Development & Neuroscience, University of Cambridge, Cambridge, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Jeremiah Y Cohen
- The Solomon H. Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Allen Institute for Neural Dynamics, Seattle, WA, USA
| | - Wolfram Schultz
- Department of Physiology, Development & Neuroscience, University of Cambridge, Cambridge, UK
| | - Alicia Izquierdo
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
- The Brain Research Institute, University of California, Los Angeles, Los Angeles, CA, USA
| | - Alireza Soltani
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA.
| |
Collapse
|
13
|
Topel S, Ma I, Sleutels J, van Steenbergen H, de Bruijn ERA, van Duijvenvoorde ACK. Expecting the unexpected: a review of learning under uncertainty across development. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2023:10.3758/s13415-023-01098-0. [PMID: 37237092 PMCID: PMC10390612 DOI: 10.3758/s13415-023-01098-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 03/28/2023] [Indexed: 05/28/2023]
Abstract
Many of our decisions take place under uncertainty. To successfully navigate the environment, individuals need to estimate the degree of uncertainty and adapt their behaviors accordingly by learning from experiences. However, uncertainty is a broad construct and distinct types of uncertainty may differentially influence our learning. We provide a semi-systematic review to illustrate cognitive and neurobiological processes involved in learning under two types of uncertainty: learning in environments with stochastic outcomes, and with volatile outcomes. We specifically reviewed studies (N = 26 studies) that included an adolescent population, because adolescence is a period in life characterized by heightened exploration and learning, as well as heightened uncertainty due to experiencing many new, often social, environments. Until now, reviews have not comprehensively compared learning under distinct types of uncertainties in this age range. Our main findings show that although the overall developmental patterns were mixed, most studies indicate that learning from stochastic outcomes, as indicated by increased accuracy in performance, improved with age. We also found that adolescents tended to have an advantage compared with adults and children when learning from volatile outcomes. We discuss potential mechanisms explaining these age-related differences and conclude by outlining future research directions.
Collapse
Affiliation(s)
- Selin Topel
- Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333, AK, Leiden, The Netherlands.
- Leiden Institute for Brain and Cognition, Leiden, The Netherlands.
| | - Ili Ma
- Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333, AK, Leiden, The Netherlands
- Leiden Institute for Brain and Cognition, Leiden, The Netherlands
| | - Jan Sleutels
- Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333, AK, Leiden, The Netherlands
- Leiden University, Institute for Philosophy, Leiden, The Netherlands
| | - Henk van Steenbergen
- Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333, AK, Leiden, The Netherlands
- Leiden Institute for Brain and Cognition, Leiden, The Netherlands
| | - Ellen R A de Bruijn
- Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333, AK, Leiden, The Netherlands
- Leiden Institute for Brain and Cognition, Leiden, The Netherlands
| | - Anna C K van Duijvenvoorde
- Leiden University, Institute of Psychology, Wassenaarseweg 52, 2333, AK, Leiden, The Netherlands
- Leiden Institute for Brain and Cognition, Leiden, The Netherlands
| |
Collapse
|
14
|
Man V, Cockburn J, Flouty O, Gander PE, Sawada M, Kovach CK, Kawasaki H, Oya H, Howard MA, O'Doherty JP. Temporally organized representations of reward and risk in the human brain. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.09.539916. [PMID: 37214975 PMCID: PMC10197553 DOI: 10.1101/2023.05.09.539916] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
The value and uncertainty associated with choice alternatives constitute critical features along which decisions are made. While the neural substrates supporting reward and risk processing have been investigated, the temporal organization by which these computations are encoded remains elusive. Here we leverage the high spatiotemporal precision of intracranial electroencephalography (iEEG) to uncover how representations of decision-related computations unfold in time. We present evidence of locally distributed representations of reward and risk variables that are temporally organized across multiple regions of interest. Reward outcome representations across wide-spread regions follow a temporally cascading order along the anteroposterior axis of the brain. In contrast, expected value can be decoded from multiple regions at the same time, and error signals in both reward and risk domains reflect a mixture of sequential and parallel encoding. We highlight the role of the anterior insula in generalizing between reward prediction error (RePE) and risk prediction error (RiPE), within which the encoding of RePE in the distributed iEEG signal predicts RiPE. Together our results emphasize the utility of uncovering temporal dynamics in the human brain for understanding how computational processes critical for value-based decisions under uncertainty unfold.
Collapse
|
15
|
Wang Z, Nan T, Goerlich KS, Li Y, Aleman A, Luo Y, Xu P. Neurocomputational mechanisms underlying fear-biased adaptation learning in changing environments. PLoS Biol 2023; 21:e3001724. [PMID: 37126501 PMCID: PMC10174591 DOI: 10.1371/journal.pbio.3001724] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2022] [Revised: 05/11/2023] [Accepted: 03/31/2023] [Indexed: 05/02/2023] Open
Abstract
Humans are able to adapt to the fast-changing world by estimating statistical regularities of the environment. Although fear can profoundly impact adaptive behaviors, the computational and neural mechanisms underlying this phenomenon remain elusive. Here, we conducted a behavioral experiment (n = 21) and a functional magnetic resonance imaging experiment (n = 37) with a novel cue-biased adaptation learning task, during which we simultaneously manipulated emotional valence (fearful/neutral expressions of the cue) and environmental volatility (frequent/infrequent reversals of reward probabilities). Across 2 experiments, computational modeling consistently revealed a higher learning rate for the environment with frequent versus infrequent reversals following neutral cues. In contrast, this flexible adjustment was absent in the environment with fearful cues, suggesting a suppressive role of fear in adaptation to environmental volatility. This suppressive effect was underpinned by activity of the ventral striatum, hippocampus, and dorsal anterior cingulate cortex (dACC) as well as increased functional connectivity between the dACC and temporal-parietal junction (TPJ) for fear with environmental volatility. Dynamic causal modeling identified that the driving effect was located in the TPJ and was associated with dACC activation, suggesting that the suppression of fear on adaptive behaviors occurs at the early stage of bottom-up processing. These findings provide a neuro-computational account of how fear interferes with adaptation to volatility during dynamic environments.
Collapse
Affiliation(s)
- Zhihao Wang
- Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education (BNU), Faculty of Psychology, Beijing Normal University, Beijing, China
- CNRS-Centre d'Economie de la Sorbonne, Panthéon-Sorbonne University, France
| | - Tian Nan
- School of Psychology, Sichuan Center of Applied Psychology, Chengdu Medical College, Chengdu, China
| | - Katharina S Goerlich
- University of Groningen, Department of Biomedical Sciences of Cells & Systems, Section Cognitive Neuroscience, University Medical Center Groningen, Groningen, the Netherlands
| | - Yiman Li
- Shenzhen Key Laboratory of Affective and Social Neuroscience, Magnetic Resonance Imaging, Center for Brain Disorders and Cognitive Sciences, Shenzhen University, Shenzhen, China
| | - André Aleman
- University of Groningen, Department of Biomedical Sciences of Cells & Systems, Section Cognitive Neuroscience, University Medical Center Groningen, Groningen, the Netherlands
| | - Yuejia Luo
- School of Psychology, Sichuan Center of Applied Psychology, Chengdu Medical College, Chengdu, China
- Shenzhen Key Laboratory of Affective and Social Neuroscience, Magnetic Resonance Imaging, Center for Brain Disorders and Cognitive Sciences, Shenzhen University, Shenzhen, China
- The State Key Lab of Cognitive and Learning, Faculty of Psychology, Beijing Normal University, Beijing, China
| | - Pengfei Xu
- Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education (BNU), Faculty of Psychology, Beijing Normal University, Beijing, China
- Center for Neuroimaging, Shenzhen Institute of Neuroscience, Shenzhen, China
| |
Collapse
|
16
|
Hallock HL, Adiraju S, Miranda-Barrientos J, McInerney JM, Oh S, DeBrosse AC, Li Y, Carr GV, Martinowich K. Electrophysiological correlates of attention in the locus coeruleus - anterior cingulate cortex circuit during the rodent continuous performance test. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.04.19.537406. [PMID: 37131757 PMCID: PMC10153204 DOI: 10.1101/2023.04.19.537406] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Sustained attention, the ability to focus on an activity or stimulus over time, is significantly impaired in many psychiatric disorders, and there remains a major unmet need in treating impaired attention. Continuous performance tests (CPTs) were developed to measure sustained attention in humans, non-human primates, rats, and mice, and similar neural circuits are engaged across species during CPT performance, supporting their use in translational studies to identify novel therapeutics. Here, we identified electrophysiological correlates of attentional performance in a touchscreen-based rodent CPT (rCPT) in the locus coeruleus (LC) and anterior cingulate cortex (ACC), two inter-connected regions that are implicated in attentional processes. We used viral labeling and molecular techniques to demonstrate that neural activity is recruited in LC-ACC projections during the rCPT, and that this recruitment increases with cognitive demand. We implanted male mice with depth electrodes within the LC and ACC for local field potential (LFP) recordings during rCPT training, and identified an increase in ACC delta and theta power, and an increase in LC delta power during correct responses in the rCPT. We also found that the LC leads the ACC in theta frequencies during correct responses while the ACC leads the LC in gamma frequencies during incorrect responses. These findings may represent translational biomarkers that can be used to screen novel therapeutics for drug discovery in attention.
Collapse
Affiliation(s)
| | - Suhaas Adiraju
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA
| | | | - Jessica M. McInerney
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA
| | - Seyun Oh
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA
| | - Adrienne C. DeBrosse
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA
| | - Ye Li
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA
| | - Gregory V. Carr
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA
- Department of Pharmacology and Molecular Sciences, Johns Hopkins University School of Medicine, Baltimore, MD, 21205, USA
| | - Keri Martinowich
- Lieber Institute for Brain Development, Johns Hopkins Medical Campus, Baltimore, MD, 21205, USA
- Department of Psychiatry and Behavioral Sciences, Johns Hopkins School of Medicine, Baltimore, MD, 21205, USA
- Department of Neuroscience, Johns Hopkins School of Medicine, Baltimore, MD, 21205, USA
- The Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD, 21205, USA
| |
Collapse
|
17
|
Bounmy T, Eger E, Meyniel F. A characterization of the neural representation of confidence during probabilistic learning. Neuroimage 2023; 268:119849. [PMID: 36640947 DOI: 10.1016/j.neuroimage.2022.119849] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2022] [Revised: 12/09/2022] [Accepted: 12/29/2022] [Indexed: 01/13/2023] Open
Abstract
Learning in a stochastic and changing environment is a difficult task. Models of learning typically postulate that observations that deviate from the learned predictions are surprising and used to update those predictions. Bayesian accounts further posit the existence of a confidence-weighting mechanism: learning should be modulated by the confidence level that accompanies those predictions. However, the neural bases of this confidence are much less known than the ones of surprise. Here, we used a dynamic probability learning task and high-field MRI to identify putative cortical regions involved in the representation of confidence about predictions during human learning. We devised a stringent test based on the conjunction of four criteria. We localized several regions in parietal and frontal cortices whose activity is sensitive to the confidence of an ideal observer, specifically so with respect to potential confounds (surprise and predictability), and in a way that is invariant to which item is predicted. We also tested for functionality in two ways. First, we localized regions whose activity patterns at the subject level showed an effect of both confidence and surprise in qualitative agreement with the confidence-weighting principle. Second, we found neural representations of ideal confidence that also accounted for subjective confidence. Taken together, those results identify a set of cortical regions potentially implicated in the confidence-weighting of learning.
Collapse
Affiliation(s)
- Tiffany Bounmy
- Cognitive Neuroimaging Unit, CEA DRF/Joliot, INSERM, Université Paris-Saclay, NeuroSpin Center, Gif-sur-Yvette, France; Université de Paris, Paris, France.
| | - Evelyn Eger
- Cognitive Neuroimaging Unit, CEA DRF/Joliot, INSERM, Université Paris-Saclay, NeuroSpin Center, Gif-sur-Yvette, France
| | - Florent Meyniel
- Cognitive Neuroimaging Unit, CEA DRF/Joliot, INSERM, Université Paris-Saclay, NeuroSpin Center, Gif-sur-Yvette, France.
| |
Collapse
|
18
|
Berkay D, Jenkins AC. A Role for Uncertainty in the Neural Distinction Between Social and Nonsocial Thought. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2023; 18:491-502. [PMID: 36170572 DOI: 10.1177/17456916221112077] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Neuroimaging research has identified a network of brain regions that is consistently more engaged when people think about the minds of other people than when they engage in nonsocial tasks. Activations in this "mentalizing network" are sometimes interpreted as evidence for the domain-specificity of cognitive processes supporting social thought. Here, we examine the alternative possibility that at least some activations in the mentalizing network may be explained by uncertainty. A reconsideration of findings from existing functional MRI studies in light of new data from independent raters suggests that (a) social tasks used in past studies have higher levels of uncertainty than their nonsocial comparison tasks and (b) activation in a key brain region associated with social cognition, the dorsomedial prefrontal cortex (DMPFC), may track with the degree of uncertainty surrounding both social and nonsocial inferences. These observations suggest that the preferential DMPFC response observed consistently in social scenarios may reflect the engagement of domain-general processes of uncertainty reduction, which points to avenues for future research into the core cognitive mechanisms supporting typical and atypical social thought.
Collapse
Affiliation(s)
- Dilara Berkay
- Department of Psychology, University of Pennsylvania
| | | |
Collapse
|
19
|
Bakst L, McGuire JT. Experience-driven recalibration of learning from surprising events. Cognition 2023; 232:105343. [PMID: 36481590 PMCID: PMC9851993 DOI: 10.1016/j.cognition.2022.105343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2022] [Revised: 10/13/2022] [Accepted: 11/21/2022] [Indexed: 12/12/2022]
Abstract
Different environments favor different patterns of adaptive learning. A surprising event that in one context would accelerate belief updating might, in another context, be downweighted as a meaningless outlier. Here, we investigated whether people would spontaneously regulate the influence of surprise on learning in response to event-by-event experiential feedback. Across two experiments, we examined whether participants performing a perceptual judgment task under spatial uncertainty (n = 29, n = 63) adapted their patterns of predictive gaze according to the informativeness or uninformativeness of surprising events in their current environment. Uninstructed predictive eye movements exhibited a form of metalearning in which surprise came to modulate event-by-event learning rates in opposite directions across contexts. Participants later appropriately readjusted their patterns of adaptive learning when the statistics of the environment underwent an unsignaled reversal. Although significant adjustments occurred in both directions, performance was consistently superior in environments in which surprising events reflected meaningful change, potentially reflecting a bias towards interpreting surprise as informative and/or difficulty ignoring salient outliers. Our results provide evidence for spontaneous, context-appropriate recalibration of the role of surprise in adaptive learning.
Collapse
Affiliation(s)
- Leah Bakst
- Department of Psychological & Brain Sciences, Boston University, 64 Cummington Mall, Boston, MA 02215, USA; Center for Systems Neuroscience, Boston University, 610 Commonwealth Avenue, Boston, MA 02215, USA.
| | - Joseph T McGuire
- Department of Psychological & Brain Sciences, Boston University, 64 Cummington Mall, Boston, MA 02215, USA; Center for Systems Neuroscience, Boston University, 610 Commonwealth Avenue, Boston, MA 02215, USA.
| |
Collapse
|
20
|
Cockburn J, Man V, Cunningham W, O'Doherty JP. Novelty and uncertainty regulate the balance between exploration and exploitation through distinct mechanisms in the human brain. Neuron 2022; 110:2691-2702.e8. [PMID: 35809575 DOI: 10.1016/j.neuron.2022.05.025] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2021] [Revised: 02/18/2022] [Accepted: 05/27/2022] [Indexed: 11/17/2022]
Abstract
Both novelty and uncertainty are potent features guiding exploration; however, they are often experimentally conflated, and an understanding of how they interact to regulate the balance between exploration and exploitation has proved elusive. Using a task designed to decouple the influence of novelty and uncertainty, we identify separable mechanisms through which exploration is directed. We show that uncertainty-directed exploration is sensitive to the prospective benefit offered by new information, whereas novelty-directed exploration is maintained regardless of its potential advantage. Using a computational framework in conjunction with fMRI, we show that uncertainty-directed choice is rooted in an adaptive bias indexing the prospective utility of exploration. In contrast, novelty persistently promotes exploration by optimistically inflating reward expectations while simultaneously dampening uncertainty signals. Our results identify separable neural substrates charged with balancing the explore/exploit trade-off to foster a manageable decomposition of an otherwise intractable problem.
Collapse
Affiliation(s)
- Jeffrey Cockburn
- Division of Humanities and Social Sciences, Caltech, Pasadena, CA, USA.
| | - Vincent Man
- Division of Humanities and Social Sciences, Caltech, Pasadena, CA, USA
| | | | - John P O'Doherty
- Division of Humanities and Social Sciences, Caltech, Pasadena, CA, USA
| |
Collapse
|
21
|
Martinez-Saito M, Gorina E. Learning under social versus nonsocial uncertainty: A meta-analytic approach. Hum Brain Mapp 2022; 43:4185-4206. [PMID: 35620870 PMCID: PMC9374892 DOI: 10.1002/hbm.25948] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2021] [Revised: 04/08/2022] [Accepted: 05/04/2022] [Indexed: 01/10/2023] Open
Abstract
Much of the uncertainty that clouds our understanding of the world springs from the covert values and intentions held by other people. Thus, it is plausible that specialized mechanisms that compute learning signals under uncertainty of exclusively social origin operate in the brain. To test this hypothesis, we scoured academic databases for neuroimaging studies involving learning under uncertainty, and performed a meta‐analysis of brain activation maps that compared learning in the face of social versus nonsocial uncertainty. Although most of the brain activations associated with learning error signals were shared between social and nonsocial conditions, we found some evidence for functional segregation of error signals of exclusively social origin during learning in limited regions of ventrolateral prefrontal cortex and insula. This suggests that most behavioral adaptations to navigate social environments are reused from frontal and subcortical areas processing generic value representation and learning, but that a specialized circuitry might have evolved in prefrontal regions to deal with social context representation and strategic action.
Collapse
Affiliation(s)
| | - Elena Gorina
- Department of Cognitive and Brain Sciences, Hebrew University of Jerusalem, Jerusalem, Israel
| |
Collapse
|
22
|
Zha R, Li P, Liu Y, Alarefi A, Zhang X, Li J. The orbitofrontal cortex represents advantageous choice in the Iowa gambling task. Hum Brain Mapp 2022; 43:3840-3856. [PMID: 35476367 PMCID: PMC9294296 DOI: 10.1002/hbm.25887] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2021] [Revised: 02/19/2022] [Accepted: 03/18/2022] [Indexed: 01/26/2023] Open
Abstract
A good‐based model, the central neurobiological model of economic decision‐making, proposes that the orbitofrontal cortex (OFC) represents binary choice outcome, that is, the chosen good. A good is defined by a group of determinants characterizing the conditions in which the commodity is offered, including commodity type, cost, risk, time delay, and ambiguity. Previous studies have found that the OFC represents the binary choice outcome in decision‐making tasks involving commodity type, cost, risk, and delay. Real‐life decisions are often complex and involve uncertainty, rewards, and penalties; however, whether the OFC represents binary choice outcomes in a complex decision‐making situation, for example, Iowa gambling task (IGT), remains unclear. Here, we propose that the OFC represents binary choice outcome, that is, advantageous choice versus disadvantageous choice, in the IGT. We propose two hypotheses: first, the activity pattern in the human OFC represents an advantageous choice; and second, choice induces an OFC‐related functional network. Using functional magnetic resonance imaging and advanced machine‐learning tools, we found that the OFC represented an advantageous choice in the IGT. The OFC representation of advantageous choice was related to decision‐making performance. Choice modulated the functional connectivity between the OFC and the superior medial gyrus. In conclusion, the OFC represents an advantageous choice during the IGT. In the framework of a good‐based model, the results extend the role of the OFC to complex decision‐making situation when making a binary choice.
Collapse
Affiliation(s)
- Rujing Zha
- Department of Radiology, the First Affiliated Hospital of USTC, Department of Psychology, School of Humanities & Social Science, Division of Life Science and Medicine, University of Science & Technology of China, Hefei, Anhui, China
| | - Peng Li
- Department of Automation, School of Information Science and Technology, University of Science and Technology of China, Hefei, Anhui, China
| | - Ying Liu
- Department of Radiology, the First Affiliated Hospital of USTC, Department of Psychology, School of Humanities & Social Science, Division of Life Science and Medicine, University of Science & Technology of China, Hefei, Anhui, China
| | - Abdulqawi Alarefi
- Department of Radiology, the First Affiliated Hospital of USTC, Department of Psychology, School of Humanities & Social Science, Division of Life Science and Medicine, University of Science & Technology of China, Hefei, Anhui, China
| | - Xiaochu Zhang
- Department of Radiology, the First Affiliated Hospital of USTC, Department of Psychology, School of Humanities & Social Science, Division of Life Science and Medicine, University of Science & Technology of China, Hefei, Anhui, China.,Application Technology Center of Physical Therapy to Brain Disorders, Institute of Advanced Technology, University of Science & Technology of China, Hefei, Anhui, China.,Hefei Medical Research Center on Alcohol Addiction, Affiliated Psychological Hospital of Anhui Medical University, Hefei Fourth People's Hospital, Anhui Mental Health Center, Hefei, Anhui, China.,Biomedical Sciences and Health Laboratory of Anhui Province, University of Science & Technology of China, Hefei, Anhui, China
| | - Jun Li
- Department of Automation, University of Science and Technology of China, Hefei, China
| |
Collapse
|
23
|
Grossman CD, Bari BA, Cohen JY. Serotonin neurons modulate learning rate through uncertainty. Curr Biol 2022; 32:586-599.e7. [PMID: 34936883 PMCID: PMC8825708 DOI: 10.1016/j.cub.2021.12.006] [Citation(s) in RCA: 40] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2020] [Revised: 10/11/2021] [Accepted: 12/03/2021] [Indexed: 12/20/2022]
Abstract
Regulating how fast to learn is critical for flexible behavior. Learning about the consequences of actions should be slow in stable environments, but accelerate when that environment changes. Recognizing stability and detecting change are difficult in environments with noisy relationships between actions and outcomes. Under these conditions, theories propose that uncertainty can be used to modulate learning rates ("meta-learning"). We show that mice behaving in a dynamic foraging task exhibit choice behavior that varied as a function of two forms of uncertainty estimated from a meta-learning model. The activity of dorsal raphe serotonin neurons tracked both types of uncertainty in the foraging task as well as in a dynamic Pavlovian task. Reversible inhibition of serotonin neurons in the foraging task reproduced changes in learning predicted by a simulated lesion of meta-learning in the model. We thus provide a quantitative link between serotonin neuron activity, learning, and decision making.
Collapse
Affiliation(s)
- Cooper D Grossman
- The Solomon H. Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, 725 N. Wolfe Street, Baltimore, MD 21205, USA
| | - Bilal A Bari
- The Solomon H. Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, 725 N. Wolfe Street, Baltimore, MD 21205, USA
| | - Jeremiah Y Cohen
- The Solomon H. Snyder Department of Neuroscience, Brain Science Institute, Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, 725 N. Wolfe Street, Baltimore, MD 21205, USA.
| |
Collapse
|
24
|
Habicht J, Dubois M, Michely J, Hauser TU. Do propranolol and amisulpride modulate confidence in risk-taking? Wellcome Open Res 2022. [DOI: 10.12688/wellcomeopenres.17423.1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
Background: Making rational choices and being able to consciously reflect on the goodness of these choices is important for successfully navigating the world. Value-based decisions have been extensively studied, but we know little about the factors that influence our confidence in value-based choice. Particularly, we know very little about the neurotransmitters that may mediate these processes. Methods: In this double-blind, placebo-controlled study design involving 61 healthy human subjects (30 female), we assessed the contributions of dopamine (400 mg amisulpride) and noradrenaline (40 mg propranolol) to value-based decision making and the subjective confidence therein in a monetary risky gambling task. Results: We did not find any significant effect of either of the two pharmacological manipulations, neither on value-based decision making, nor on subjective confidence. Conclusion: We discuss these (null) findings, and release all relevant data and code. This will allow researchers to further interrogate the data, to counteract publication biases in favour of significant findings, and to use our study as a source for balanced meta-analyses.
Collapse
|
25
|
Locus Coeruleus in Non-Mammalian Vertebrates. Brain Sci 2022; 12:brainsci12020134. [PMID: 35203898 PMCID: PMC8870555 DOI: 10.3390/brainsci12020134] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2021] [Revised: 01/08/2022] [Accepted: 01/15/2022] [Indexed: 11/30/2022] Open
Abstract
The locus coeruleus (LC) is a vertebrate-specific nucleus and the primary source of norepinephrine (NE) in the brain. This nucleus has conserved properties across species: highly homogeneous cell types, a small number of cells but extensive axonal projections, and potent influence on brain states. Comparative studies on LC benefit greatly from its homogeneity in cell types and modularity in projection patterns, and thoroughly understanding the LC-NE system could shed new light on the organization principles of other more complex modulatory systems. Although studies on LC are mainly focused on mammals, many of the fundamental properties and functions of LC are readily observable in other vertebrate models and could inform mammalian studies. Here, we summarize anatomical and functional studies of LC in non-mammalian vertebrate classes, fish, amphibians, reptiles, and birds, on topics including axonal projections, gene expressions, homeostatic control, and modulation of sensorimotor transformation. Thus, this review complements mammalian studies on the role of LC in the brain.
Collapse
|
26
|
Matsumoto M, Abe H, Tanaka K, Matsumoto K. Different types of uncertainty distinguished by monkey prefrontal neurons. Cereb Cortex Commun 2022; 3:tgac002. [PMID: 35169710 PMCID: PMC8842276 DOI: 10.1093/texcom/tgac002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2021] [Revised: 12/28/2021] [Accepted: 12/29/2021] [Indexed: 11/15/2022] Open
Abstract
To adapt one’s behavior, in a timely manner, to an environment that changes in many different aspects, one must be sensitive to uncertainty about each aspect of the environment. Although the medial prefrontal cortex has been implicated in the representation and reduction of a variety of uncertainties, it is unknown whether different types of uncertainty are distinguished by distinct neuronal populations. To investigate how the prefrontal cortex distinguishes between different types of uncertainty, we recorded neuronal activities from the medial and lateral prefrontal cortices of monkeys performing a visual feedback-based action-learning task in which uncertainty of coming feedback and that of context change varied asynchronously. We found that the activities of two groups of prefrontal cells represented the two different types of uncertainty. These results suggest that different types of uncertainty are represented by distinct neural populations in the prefrontal cortex.
Collapse
Affiliation(s)
- Madoka Matsumoto
- Department of Preventive Intervention for Psychiatric Disorders, National Institute of Mental Health, National Center of Neurology and Psychiatry, 4-1-1 Ogawahigashi-cho, Kodaira, Tokyo 187-8553, Japan
- Tamagawa University Brain Science Institute, 6-1-1 Tamagawa-gakuen, Machida, Tokyo 194-8610, Japan
- Laboratory for Molecular Analysis of Higher Brain Function, Center for Brain Science, RIKEN, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan
- Laboratory for Cognitive Brain Mapping, Center for Brain Science, RIKEN, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan
| | - Hiroshi Abe
- Laboratory for Molecular Analysis of Higher Brain Function, Center for Brain Science, RIKEN, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan
- Laboratory for Cognitive Brain Mapping, Center for Brain Science, RIKEN, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan
| | - Keiji Tanaka
- Laboratory for Cognitive Brain Mapping, Center for Brain Science, RIKEN, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan
| | - Kenji Matsumoto
- Tamagawa University Brain Science Institute, 6-1-1 Tamagawa-gakuen, Machida, Tokyo 194-8610, Japan
- Laboratory for Cognitive Brain Mapping, Center for Brain Science, RIKEN, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan
| |
Collapse
|
27
|
Kim D, Jeong J, Lee SW. Prefrontal solution to the bias-variance tradeoff during reinforcement learning. Cell Rep 2021; 37:110185. [PMID: 34965420 DOI: 10.1016/j.celrep.2021.110185] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2021] [Revised: 08/09/2021] [Accepted: 12/07/2021] [Indexed: 11/17/2022] Open
Abstract
Evidence that the brain combines different value learning strategies to minimize prediction error is accumulating. However, the tradeoff between bias and variance error, which imposes different constraints on each learning strategy's performance, poses a challenge for value learning. While this tradeoff specifies the requirements for optimal learning, little has been known about how the brain deals with this issue. Here, we hypothesize that the brain adaptively resolves the bias-variance tradeoff during reinforcement learning. Our theory suggests that the solution necessitates baseline correction for prediction error, which offsets the adverse effects of irreducible error on value learning. We show behavioral evidence of adaptive control using a Markov decision task with context changes. The prediction error baseline seemingly signals context changes to improve adaptability. Critically, we identify multiplexed representations of prediction error baseline within the ventrolateral and ventromedial prefrontal cortex, key brain regions known to guide model-based and model-free reinforcement learning.
Collapse
Affiliation(s)
- Dongjae Kim
- Center for Neural Science, New York University, New York, NY, USA; Department of Psychology, New York University, New York, NY, USA
| | - Jaeseung Jeong
- Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST), 34141 Daejeon, Republic of Korea; Program of Brain and Cognitive Engineering, Korea Advanced Institute of Science and Technology (KAIST), 34141 Daejeon, Republic of Korea
| | - Sang Wan Lee
- Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST), 34141 Daejeon, Republic of Korea; Program of Brain and Cognitive Engineering, Korea Advanced Institute of Science and Technology (KAIST), 34141 Daejeon, Republic of Korea; KAIST Center for Neuroscience-inspired AI, Korea Advanced Institute of Science and Technology (KAIST), 34141 Daejeon, Republic of Korea; KI for Health Science and Technology, Korea Advanced Institute of Science and Technology (KAIST), 34141 Daejeon, Republic of Korea; KI for Artificial Intelligence, Korea Advanced Institute of Science and Technology (KAIST), 34141 Daejeon, Republic of Korea.
| |
Collapse
|
28
|
Fiore VG, Gu X. Similar network compositions, but distinct neural dynamics underlying belief updating in environments with and without explicit outcomes. Neuroimage 2021; 247:118821. [PMID: 34920087 PMCID: PMC8823284 DOI: 10.1016/j.neuroimage.2021.118821] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2021] [Revised: 12/10/2021] [Accepted: 12/13/2021] [Indexed: 11/19/2022] Open
Abstract
Classic decision theories typically assume the presence of explicit value-based outcomes after action selections to update beliefs about action-outcome contingencies. However, ecological environments are often opaque, and it remains unclear whether the neural dynamics underlying belief updating vary under conditions characterized by the presence or absence of such explicit value-based information, after each choice selection. We investigated this question in healthy humans (n = 28) using Bayesian inference and two multi-option fMRI tasks: a multi-armed bandit task, and a probabilistic perceptual task, respectively with and without explicit value-based feedback after choice selections. Model-based fMRI analysis revealed a network encoding belief updating which did not change depending on the task. More precisely, we found a confidence-building network that included anterior hippocampus, amygdala, and medial prefrontal cortex (mPFC), which became more active as beliefs about action-outcome probabilities were confirmed by newly acquired information. Despite these consistent responses across tasks, dynamic causal modeling estimated that the network dynamics changed depending on the presence or absence of trial-by-trial value-based outcomes. In the task deprived of immediate feedback, the hippocampus increased its influence towards both amygdala and mPFC, in association with increased strength in the confidence signal. However, the opposite causal relations were found (i.e., from both mPFC and amygdala towards the hippocampus), in presence of immediate outcomes. This finding revealed an asymmetric relationship between decision confidence computations, which were based on similar computational models across tasks, and neural implementation, which varied depending on the availability of outcomes after choice selections.
Collapse
Affiliation(s)
- Vincenzo G Fiore
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY 10029, United States; Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY 10027, United States.
| | - Xiaosi Gu
- Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY 10029, United States; Center for Computational Psychiatry, Icahn School of Medicine at Mount Sinai, New York, NY 10027, United States; Nash Family Department of Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY 10029, United States.
| |
Collapse
|
29
|
Foucault C, Meyniel F. Gated recurrence enables simple and accurate sequence prediction in stochastic, changing, and structured environments. eLife 2021; 10:71801. [PMID: 34854377 PMCID: PMC8735865 DOI: 10.7554/elife.71801] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2021] [Accepted: 12/01/2021] [Indexed: 11/13/2022] Open
Abstract
From decision making to perception to language, predicting what is coming next is crucial. It is also challenging in stochastic, changing, and structured environments; yet the brain makes accurate predictions in many situations. What computational architecture could enable this feat? Bayesian inference makes optimal predictions but is prohibitively difficult to compute. Here, we show that a specific recurrent neural network architecture enables simple and accurate solutions in several environments. This architecture relies on three mechanisms: gating, lateral connections, and recurrent weight training. Like the optimal solution and the human brain, such networks develop internal representations of their changing environment (including estimates of the environment’s latent variables and the precision of these estimates), leverage multiple levels of latent structure, and adapt their effective learning rate to changes without changing their connection weights. Being ubiquitous in the brain, gated recurrence could therefore serve as a generic building block to predict in real-life environments.
Collapse
Affiliation(s)
- Cédric Foucault
- INSERM, CEA, Université Paris-Saclay, Gif sur Yvette, France
| | | |
Collapse
|
30
|
Zhen S, Yaple ZA, Eickhoff SB, Yu R. To learn or to gain: neural signatures of exploration in human decision-making. Brain Struct Funct 2021; 227:63-76. [PMID: 34596757 DOI: 10.1007/s00429-021-02389-3] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2020] [Accepted: 09/19/2021] [Indexed: 11/26/2022]
Abstract
Individuals not only take actions to obtain immediate rewards but also to gain more information to guide future choices. An ideal exploration-exploitation balance is crucial for maximizing reward over the long run. However, the neural signatures of exploration in humans remain unclear. Using quantitative meta-analyses of functional magnetic resonance imaging experiments on exploratory behaviors, we sought to identify the concordant activity pertaining to exploration over a range of experiments. The results revealed that exploration activates concordant brain activity associated with risk (e.g., dorsal medial prefrontal cortex and anterior insula), cognitive control (e.g., dorsolateral prefrontal cortex and inferior frontal gyrus), and motor processing (e.g., premotor cortex). These stereotaxic maps of exploration may indicate that exploration is highly linked to risk processing, but is also specifically associated with regions involved in executive control processes. Although this explanation should be treated as exploratory, these findings support theories positing an important role for the prefrontal-insular-motor cortical network in exploration.
Collapse
Affiliation(s)
- Shanshan Zhen
- Department of Management, Hong Kong Baptist University, Hong Kong, China
| | - Zachary A Yaple
- Department of Psychology, Faculty of Health, York University, Toronto, ON, Canada
| | - Simon B Eickhoff
- Medical Faculty, Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
- Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany
| | - Rongjun Yu
- Department of Management, Hong Kong Baptist University, Hong Kong, China.
| |
Collapse
|
31
|
Kujala T, Lappi O. Inattention and Uncertainty in the Predictive Brain. FRONTIERS IN NEUROERGONOMICS 2021; 2:718699. [PMID: 38235221 PMCID: PMC10790892 DOI: 10.3389/fnrgo.2021.718699] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/01/2021] [Accepted: 09/02/2021] [Indexed: 01/19/2024]
Abstract
Negative effects of inattention on task performance can be seen in many contexts of society and human behavior, such as traffic, work, and sports. In traffic, inattention is one of the most frequently cited causal factors in accidents. In order to identify inattention and mitigate its negative effects, there is a need for quantifying attentional demands of dynamic tasks, with a credible basis in cognitive modeling and neuroscience. Recent developments in cognitive science have led to theories of cognition suggesting that brains are an advanced prediction engine. The function of this prediction engine is to support perception and action by continuously matching incoming sensory input with top-down predictions of the input, generated by hierarchical models of the statistical regularities and causal relationships in the world. Based on the capacity of this predictive processing framework to explain various mental phenomena and neural data, we suggest it also provides a plausible theoretical and neural basis for modeling attentional demand and attentional capacity "in the wild" in terms of uncertainty and prediction error. We outline a predictive processing approach to the study of attentional demand and inattention in driving, based on neurologically-inspired theories of uncertainty processing and experimental research combining brain imaging, visual occlusion and computational modeling. A proper understanding of uncertainty processing would enable comparison of driver's uncertainty to a normative level of appropriate uncertainty, and thereby improve definition and detection of inattentive driving. This is the necessary first step toward applications such as attention monitoring systems for conventional and semi-automated driving.
Collapse
Affiliation(s)
- Tuomo Kujala
- Cognitive Science, Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland
| | - Otto Lappi
- Cognitive Science, Traffic Research Unit, Faculty of Arts, University of Helsinki, Helsinki, Finland
| |
Collapse
|
32
|
Bossaerts P. How Neurobiology Elucidates the Role of Emotions in Financial Decision-Making. Front Psychol 2021; 12:697375. [PMID: 34349708 PMCID: PMC8326835 DOI: 10.3389/fpsyg.2021.697375] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2021] [Accepted: 06/09/2021] [Indexed: 11/13/2022] Open
Abstract
Over the last 15 years, a revolution has been taking place in neuroscience, whereby models and methods of economics have led to deeper insights into the neurobiological foundations of human decision-making. These have revealed a number of widespread mis-conceptions, among others, about the role of emotions. Furthermore, the findings suggest that a purely behavior-based approach to studying decisions may miss crucial features of human choice long appreciated in biology, such as Pavlovian approach. The findings could help economists formalize elusive concepts such as intuition, as I show here for financial “trading intuition.”
Collapse
Affiliation(s)
- Peter Bossaerts
- Faculty of Business and Economics, University of Melbourne, Parkville, VIC, Australia.,Faculty of Economics, University of Cambridge, Cambridge, United Kingdom.,Geneva School of Economics and Management, University of Geneva, Geneva, Switzerland
| |
Collapse
|
33
|
Moutoussis M, Garzón B, Neufeld S, Bach DR, Rigoli F, Goodyer I, Bullmore E, Guitart-Masip M, Dolan RJ. Decision-making ability, psychopathology, and brain connectivity. Neuron 2021; 109:2025-2040.e7. [PMID: 34019810 PMCID: PMC8221811 DOI: 10.1016/j.neuron.2021.04.019] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2020] [Revised: 02/16/2021] [Accepted: 04/19/2021] [Indexed: 12/11/2022]
Abstract
Decision-making is a cognitive process of central importance for the quality of our lives. Here, we ask whether a common factor underpins our diverse decision-making abilities. We obtained 32 decision-making measures from 830 young people and identified a common factor that we call "decision acuity," which was distinct from IQ and reflected a generic decision-making ability. Decision acuity was decreased in those with aberrant thinking and low general social functioning. Crucially, decision acuity and IQ had dissociable brain signatures, in terms of their associated neural networks of resting-state functional connectivity. Decision acuity was reliably measured, and its relationship with functional connectivity was also stable when measured in the same individuals 18 months later. Thus, our behavioral and brain data identify a new cognitive construct that underpins decision-making ability across multiple domains. This construct may be important for understanding mental health, particularly regarding poor social function and aberrant thought patterns.
Collapse
Affiliation(s)
- Michael Moutoussis
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK.
| | - Benjamín Garzón
- Aging Research Centre, Karolinska Institute, Stockholm, Sweden
| | - Sharon Neufeld
- Department of Psychiatry, University of Cambridge, Cambridge CB2 0SZ, UK
| | - Dominik R Bach
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK; Computational Psychiatry Research, Department of Psychiatry, Psychotherapy, and Psychosomatics, Psychiatric Hospital, University of Zurich, 8032 Zurich, Switzerland
| | | | - Ian Goodyer
- Department of Psychiatry, University of Cambridge, Cambridge CB2 0SZ, UK
| | - Edward Bullmore
- Department of Psychiatry, University of Cambridge, Cambridge CB2 0SZ, UK
| | - Marc Guitart-Masip
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK; Aging Research Centre, Karolinska Institute, Stockholm, Sweden
| | - Raymond J Dolan
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
| |
Collapse
|
34
|
Nassar MR, Troiani V. The stability flexibility tradeoff and the dark side of detail. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2021; 21:607-623. [PMID: 33236296 PMCID: PMC8141540 DOI: 10.3758/s13415-020-00848-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/04/2020] [Indexed: 12/26/2022]
Abstract
Learning in dynamic environments requires integrating over stable fluctuations to minimize the impact of noise (stability) but rapidly responding in the face of fundamental changes (flexibility). Achieving one of these goals often requires sacrificing the other to some degree, producing a stability-flexibility tradeoff. Individuals navigate this tradeoff in different ways; some people learn rapidly (emphasizing flexibility) and others rely more heavily on historical information (emphasizing stability). Despite the prominence of such individual differences in learning tasks, the degree to which they relate to broader characteristics of real-world behavior or pathologies has not been well explored. We relate individual differences in learning behavior to self-report measures thought to capture collectively the characteristics of the Autism spectrum. We show that young adults who learn most slowly tend to integrate more effective samples into their beliefs about the world making them more robust to noise (more stability) but are more likely to integrate information from previous contexts (less flexibility). We show that individuals who report paying more attention to detail tend to use high flexibility and low stability information processing strategies. We demonstrate the robustness of this inverse relationship between attention to detail and formation of stable beliefs in a heterogeneous population of children that includes a high proportion of Autism diagnoses. Together, our results highlight that attention to detail reflects an information processing policy that comes with a substantial downside, namely the ability to integrate data to overcome environmental noise.
Collapse
Affiliation(s)
- Matthew R Nassar
- Department of Neuroscience; Carney Institute for Brain Science, Brown University, Providence, RI, 02912-1821, USA.
| | - Vanessa Troiani
- Geisinger-Bucknell Autism & Developmental Medicine Institute, Lewisburg, PA, USA
| |
Collapse
|
35
|
Better the devil you know than the devil you don't: Neural processing of risk and ambiguity. Neuroimage 2021; 236:118109. [PMID: 33940147 DOI: 10.1016/j.neuroimage.2021.118109] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2020] [Revised: 04/14/2021] [Accepted: 04/18/2021] [Indexed: 11/23/2022] Open
Abstract
Risk and ambiguity are inherent in virtually all human decision-making. Risk refers to a situation in which we know the precise probability of potential outcomes of each option, whereas ambiguity refers to a situation in which outcome probabilities are not known. A large body of research has shown that individuals prefer known risks to ambiguity, a phenomenon known as ambiguity aversion. One heated debate concerns whether risky and ambiguous decisions rely on the same or distinct neural circuits. In the current meta-analyses, we integrated the results of neuroimaging research on decision-making under risk (n = 69) and ambiguity (n = 31). Our results showed that both processing of risk and ambiguity showed convergence in anterior insula, indicating a key role of anterior insula in encoding uncertainty. Risk additionally engaged dorsomedial prefrontal cortex (dmPFC) and ventral striatum, whereas ambiguity specifically recruited the dorsolateral prefrontal cortex (dlPFC), inferior parietal lobe (IPL) and right anterior insula. Our findings demonstrate overlapping and distinct neural substrates underlying different types of uncertainty, guiding future neuroimaging research on risk-taking and ambiguity aversion.
Collapse
|
36
|
Grueschow M, Stenz N, Thörn H, Ehlert U, Breckwoldt J, Brodmann Maeder M, Exadaktylos AK, Bingisser R, Ruff CC, Kleim B. Real-world stress resilience is associated with the responsivity of the locus coeruleus. Nat Commun 2021; 12:2275. [PMID: 33859187 PMCID: PMC8050280 DOI: 10.1038/s41467-021-22509-1] [Citation(s) in RCA: 38] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2019] [Accepted: 03/04/2021] [Indexed: 02/02/2023] Open
Abstract
Individuals may show different responses to stressful events. Here, we investigate the neurobiological basis of stress resilience, by showing that neural responsitivity of the noradrenergic locus coeruleus (LC-NE) and associated pupil responses are related to the subsequent change in measures of anxiety and depression in response to prolonged real-life stress. We acquired fMRI and pupillometry data during an emotional-conflict task in medical residents before they underwent stressful emergency-room internships known to be a risk factor for anxiety and depression. The LC-NE conflict response and its functional coupling with the amygdala was associated with stress-related symptom changes in response to the internship. A similar relationship was found for pupil-dilation, a potential marker of LC-NE firing. Our results provide insights into the noradrenergic basis of conflict generation, adaptation and stress resilience.
Collapse
Affiliation(s)
- Marcus Grueschow
- Zurich Center for Neuroeconomics (ZNE), Department of Economics, University of Zurich, Zurich, Switzerland.
| | - Nico Stenz
- Division of Experimental Psychopathology and Psychotherapy, Dept of Psychology, University of Zurich, Zurich, Switzerland
- Department of Psychiatry, Psychotherapy and Psychosomatics, University of Zurich, Zurich, Switzerland
| | - Hanna Thörn
- Division of Experimental Psychopathology and Psychotherapy, Dept of Psychology, University of Zurich, Zurich, Switzerland
- Department of Psychiatry, Psychotherapy and Psychosomatics, University of Zurich, Zurich, Switzerland
- Division of Clinical Psychology and Psychotherapy, Dept of Psychology, University of Zurich, Zurich, Switzerland
| | - Ulrike Ehlert
- Division of Clinical Psychology and Psychotherapy, Dept of Psychology, University of Zurich, Zurich, Switzerland
| | - Jan Breckwoldt
- Medical School, Deanery, University of Zurich, Zurich, Switzerland
| | | | | | - Roland Bingisser
- Department of Emergency Medicine, University Hospital Basel, Basel, Switzerland
| | - Christian C Ruff
- Zurich Center for Neuroeconomics (ZNE), Department of Economics, University of Zurich, Zurich, Switzerland
| | - Birgit Kleim
- Division of Experimental Psychopathology and Psychotherapy, Dept of Psychology, University of Zurich, Zurich, Switzerland.
- Department of Psychiatry, Psychotherapy and Psychosomatics, University of Zurich, Zurich, Switzerland.
| |
Collapse
|
37
|
Payzan-LeNestour E, Pradier L, Doran J, Nave G, Balleine B. Impact of ambient sound on risk perception in humans: neuroeconomic investigations. Sci Rep 2021; 11:5392. [PMID: 33686093 PMCID: PMC7940636 DOI: 10.1038/s41598-021-84359-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2020] [Accepted: 02/12/2021] [Indexed: 11/09/2022] Open
Abstract
Research in the field of multisensory perception shows that what we hear can influence what we see in a wide range of perceptual tasks. It is however unknown whether this extends to the visual perception of risk, despite the importance of the question in many applied domains where properly assessing risk is crucial, starting with financial trading. To fill this knowledge gap, we ran interviews with professional traders and conducted three laboratory studies using judgments of financial asset risk as a testbed. We provide evidence that the presence of ambient sound impacts risk perception, possibly due to the combination of facilitatory and synesthetic effects of general relevance to the perception of risk in many species as well as humans. We discuss the implications of our findings for various applied domains (e.g., financial, medical, and military decision-making), and raise new questions for future research.
Collapse
Affiliation(s)
| | - Lionnel Pradier
- University of New South Wales Business School, Sydney, Australia
| | - James Doran
- University of New South Wales Business School, Sydney, Australia
| | - Gideon Nave
- The Wharton School of the University of Pennsylvania, Philadelphia, PA, USA
| | - Bernard Balleine
- School of Psychology, University of New South Wales, Kensington, Australia
| |
Collapse
|
38
|
Inglis JB, Valentin VV, Ashby FG. Modulation of Dopamine for Adaptive Learning: A Neurocomputational Model. COMPUTATIONAL BRAIN & BEHAVIOR 2021; 4:34-52. [PMID: 34151186 PMCID: PMC8210637 DOI: 10.1007/s42113-020-00083-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
There have been many proposals that learning rates in the brain are adaptive, in the sense that they increase or decrease depending on environmental conditions. The majority of these models are abstract and make no attempt to describe the neural circuitry that implements the proposed computations. This article describes a biologically detailed computational model that overcomes this shortcoming. Specifically, we propose a neural circuit that implements adaptive learning rates by modulating the gain on the dopamine response to reward prediction errors, and we model activity within this circuit at the level of spiking neurons. The model generates a dopamine signal that depends on the size of the tonically active dopamine neuron population and the phasic spike rate. The model was tested successfully against results from two single-neuron recording studies and a fast-scan cyclic voltammetry study. We conclude by discussing the general applicability of the model to dopamine mediated tasks that transcend the experimental phenomena it was initially designed to address.
Collapse
Affiliation(s)
- Jeffrey B Inglis
- Interdepartmental Graduate Program in Dynamical Neuroscience, University of California, Santa Barbara
| | - Vivian V Valentin
- Department of Psychological & Brain Sciences, University of California, Santa Barbara
| | - F Gregory Ashby
- Department of Psychological & Brain Sciences, University of California, Santa Barbara
| |
Collapse
|
39
|
Affiliation(s)
- Markus Ullsperger
- Otto-von-Guericke University Magdeburg, Magdeburg, Germany. .,Center for Behavioral Brain Sciences, Magdeburg, Germany.
| |
Collapse
|
40
|
Abstract
A large body of work has linked dopaminergic signaling to learning and reward processing. It stresses the role of dopamine in reward prediction error signaling, a key neural signal that allows us to learn from past experiences, and that facilitates optimal choice behavior. Latterly, it has become clear that dopamine does not merely code prediction error size but also signals the difference between the expected value of rewards, and the value of rewards actually received, which is obtained through the integration of reward attributes such as the type, amount, probability and delay. More recent work has posited a role of dopamine in learning beyond rewards. These theories suggest that dopamine codes absolute or unsigned prediction errors, playing a key role in how the brain models associative regularities within its environment, while incorporating critical information about the reliability of those regularities. Work is emerging supporting this perspective and, it has inspired theoretical models of how certain forms of mental pathology may emerge in relation to dopamine function. Such pathology is frequently related to disturbed inferences leading to altered internal models of the environment. Thus, it is critical to understand the role of dopamine in error-related learning and inference.
Collapse
Affiliation(s)
- Kelly M. J. Diederen
- Department of Psychosis Studies,
Institute of Psychiatry, Psychology and Neuroscience, King’s College London,
London, UK
| | - Paul C. Fletcher
- Department of Psychiatry,
University of Cambridge, Cambridge, UK
- Cambridgeshire and Peterborough
NHS Foundation Trust, Cambridge, UK
- Wellcome Trust MRC Institute of
Metabolic Science, University of Cambridge, Cambridge Biomedical Campus,
Cambridge, UK
| |
Collapse
|
41
|
Levy I, Schiller D. Neural Computations of Threat. Trends Cogn Sci 2021; 25:151-171. [PMID: 33384214 PMCID: PMC8084636 DOI: 10.1016/j.tics.2020.11.007] [Citation(s) in RCA: 36] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2020] [Revised: 11/16/2020] [Accepted: 11/18/2020] [Indexed: 12/26/2022]
Abstract
A host of learning, memory, and decision-making processes form the individual's response to threat and may be disrupted in anxiety and post-trauma psychopathology. Here we review the neural computations of threat, from the first encounter with a dangerous situation, through learning, storing, and updating cues that predict it, to making decisions about the optimal course of action. The overview highlights the interconnected nature of these processes and their reliance on shared neural and computational mechanisms. We propose an integrative approach to the study of threat-related processes, in which specific computations are studied across the various stages of threat experience rather than in isolation. This approach can generate new insights about the evolution, diagnosis, and treatment of threat-related psychopathology.
Collapse
Affiliation(s)
- Ifat Levy
- Departments of Comparative Medicine, Neuroscience, and Psychology, Yale University, New Haven, CT, USA.
| | - Daniela Schiller
- Department of Psychiatry, Department of Neuroscience, and Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
| |
Collapse
|
42
|
Gagne C, Zika O, Dayan P, Bishop SJ. Impaired adaptation of learning to contingency volatility in internalizing psychopathology. eLife 2020; 9:e61387. [PMID: 33350387 PMCID: PMC7755392 DOI: 10.7554/elife.61387] [Citation(s) in RCA: 33] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2020] [Accepted: 11/26/2020] [Indexed: 11/13/2022] Open
Abstract
Using a contingency volatility manipulation, we tested the hypothesis that difficulty adapting probabilistic decision-making to second-order uncertainty might reflect a core deficit that cuts across anxiety and depression and holds regardless of whether outcomes are aversive or involve reward gain or loss. We used bifactor modeling of internalizing symptoms to separate symptom variance common to both anxiety and depression from that unique to each. Across two experiments, we modeled performance on a probabilistic decision-making under volatility task using a hierarchical Bayesian framework. Elevated scores on the common internalizing factor, with high loadings across anxiety and depression items, were linked to impoverished adjustment of learning to volatility regardless of whether outcomes involved reward gain, electrical stimulation, or reward loss. In particular, high common factor scores were linked to dampened learning following better-than-expected outcomes in volatile environments. No such relationships were observed for anxiety- or depression-specific symptom factors.
Collapse
Affiliation(s)
- Christopher Gagne
- Department of Psychology, UC BerkeleyBerkeleyUnited States
- Max Planck Institute for Biological CyberneticsTübingenGermany
| | - Ondrej Zika
- Max Planck Institute for Human DevelopmentBerlinGermany
| | - Peter Dayan
- Max Planck Institute for Biological CyberneticsTübingenGermany
- University of TübingenTübingenGermany
| | - Sonia J Bishop
- Department of Psychology, UC BerkeleyBerkeleyUnited States
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, FMRIB, John Radcliffe HospitalOxfordUnited Kingdom
- Helen Wills Neuroscience Institute, UC BerkeleyBerkeleyUnited States
| |
Collapse
|
43
|
Iglesias S, Kasper L, Harrison SJ, Manka R, Mathys C, Stephan KE. Cholinergic and dopaminergic effects on prediction error and uncertainty responses during sensory associative learning. Neuroimage 2020; 226:117590. [PMID: 33285332 DOI: 10.1016/j.neuroimage.2020.117590] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2020] [Revised: 10/20/2020] [Accepted: 11/19/2020] [Indexed: 01/11/2023] Open
Abstract
Navigating the physical world requires learning probabilistic associations between sensory events and their change in time (volatility). Bayesian accounts of this learning process rest on hierarchical prediction errors (PEs) that are weighted by estimates of uncertainty (or its inverse, precision). In a previous fMRI study we found that low-level precision-weighted PEs about visual outcomes (that update beliefs about associations) activated the putative dopaminergic midbrain; by contrast, precision-weighted PEs about cue-outcome associations (that update beliefs about volatility) activated the cholinergic basal forebrain. These findings suggested selective dopaminergic and cholinergic influences on precision-weighted PEs at different hierarchical levels. Here, we tested this hypothesis, repeating our fMRI study under pharmacological manipulations in healthy participants. Specifically, we performed two pharmacological fMRI studies with a between-subject double-blind placebo-controlled design: study 1 used antagonists of dopaminergic (amisulpride) and muscarinic (biperiden) receptors, study 2 used enhancing drugs of dopaminergic (levodopa) and cholinergic (galantamine) modulation. Pooled across all pharmacological conditions of study 1 and study 2, respectively, we found that low-level precision-weighted PEs activated the midbrain and high-level precision-weighted PEs the basal forebrain as in our previous study. However, we found pharmacological effects on brain activity associated with these computational quantities only when splitting the precision-weighted PEs into their PE and precision components: in a brainstem region putatively containing cholinergic (pedunculopontine and laterodorsal tegmental) nuclei, biperiden (compared to placebo) enhanced low-level PE responses and attenuated high-level PE activity, while amisulpride reduced high-level PE responses. Additionally, in the putative dopaminergic midbrain, galantamine compared to placebo enhanced low-level PE responses (in a body-weight dependent manner) and amisulpride enhanced high-level precision activity. Task behaviour was not affected by any of the drugs. These results do not support our hypothesis of a clear-cut dichotomy between different hierarchical inference levels and neurotransmitter systems, but suggest a more complex interaction between these neuromodulatory systems and hierarchical Bayesian quantities. However, our present results may have been affected by confounds inherent to pharmacological fMRI. We discuss these confounds and outline improved experimental tests for the future.
Collapse
Affiliation(s)
- Sandra Iglesias
- Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich & Swiss Federal Institute of Technology (ETH Zurich), Wilfriedstr. 6, 8032 Zurich, Switzerland.
| | - Lars Kasper
- Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich & Swiss Federal Institute of Technology (ETH Zurich), Wilfriedstr. 6, 8032 Zurich, Switzerland; Institute for Biomedical Engineering, ETH Zurich and University of Zurich, Switzerland
| | - Samuel J Harrison
- Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich & Swiss Federal Institute of Technology (ETH Zurich), Wilfriedstr. 6, 8032 Zurich, Switzerland
| | - Robert Manka
- Department of Cardiology, University Hospital Zurich, Switzerland
| | - Christoph Mathys
- Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich & Swiss Federal Institute of Technology (ETH Zurich), Wilfriedstr. 6, 8032 Zurich, Switzerland; Interacting Minds Centre, Aarhus University, Aarhus, Denmark
| | - Klaas E Stephan
- Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich & Swiss Federal Institute of Technology (ETH Zurich), Wilfriedstr. 6, 8032 Zurich, Switzerland; Max Planck Institute for Metabolism Research, Cologne, Germany
| |
Collapse
|
44
|
Wang D, Si W, Luo Y. A Biologically Inspired Behavior Control for the Unexpected Uncertainty With Motivated Developmental Network. IEEE Trans Cogn Dev Syst 2020. [DOI: 10.1109/tcds.2019.2953944] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
45
|
Ryali CK, Goffin S, Winkielman P, Yu AJ. From likely to likable: The role of statistical typicality in human social assessment of faces. Proc Natl Acad Sci U S A 2020; 117:29371-29380. [PMID: 33229540 PMCID: PMC7703555 DOI: 10.1073/pnas.1912343117] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022] Open
Abstract
Humans readily form social impressions, such as attractiveness and trustworthiness, from a stranger's facial features. Understanding the provenance of these impressions has clear scientific importance and societal implications. Motivated by the efficient coding hypothesis of brain representation, as well as Claude Shannon's theoretical result that maximally efficient representational systems assign shorter codes to statistically more typical data (quantified as log likelihood), we suggest that social "liking" of faces increases with statistical typicality. Combining human behavioral data and computational modeling, we show that perceived attractiveness, trustworthiness, dominance, and valence of a face image linearly increase with its statistical typicality (log likelihood). We also show that statistical typicality can at least partially explain the role of symmetry in attractiveness perception. Additionally, by assuming that the brain focuses on a task-relevant subset of facial features and assessing log likelihood of a face using those features, our model can explain the "ugliness-in-averageness" effect found in social psychology, whereby otherwise attractive, intercategory faces diminish in attractiveness during a categorization task.
Collapse
Affiliation(s)
- Chaitanya K Ryali
- Department of Computer Science and Engineering, University of California San Diego, La Jolla, CA 92093
| | - Stanny Goffin
- Department of Psychology, University of California San Diego, La Jolla, CA 92093
- Department of Cognitive Neuroscience, Maastricht University, 6229 ER Maastricht, The Netherlands
| | - Piotr Winkielman
- Department of Psychology, University of California San Diego, La Jolla, CA 92093
- Faculty of Psychology, SWPS University of Social Sciences and Humanities, 03-815 Warsaw, Poland
| | - Angela J Yu
- Department of Cognitive Science, University of California San Diego, La Jolla, CA 92093;
- Department of Cognitive Science, University of California San Diego, La Jolla, CA 92093
| |
Collapse
|
46
|
Imprecise neural computations as a source of adaptive behaviour in volatile environments. Nat Hum Behav 2020; 5:99-112. [PMID: 33168951 DOI: 10.1038/s41562-020-00971-z] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2019] [Accepted: 09/18/2020] [Indexed: 02/01/2023]
Abstract
In everyday life, humans face environments that feature uncertain and volatile or changing situations. Efficient adaptive behaviour must take into account uncertainty and volatility. Previous models of adaptive behaviour involve inferences about volatility that rely on complex and often intractable computations. Because such computations are presumably implausible biologically, it is unclear how humans develop efficient adaptive behaviours in such environments. Here, we demonstrate a counterintuitive result: simple, low-level inferences confined to uncertainty can produce near-optimal adaptive behaviour, regardless of the environmental volatility, assuming imprecisions in computation that conform to the psychophysical Weber law. We further show empirically that this Weber-imprecision model explains human behaviour in volatile environments better than optimal adaptive models that rely on high-level inferences about volatility, even when considering biologically plausible approximations of such models, as well as non-inferential models like adaptive reinforcement learning.
Collapse
|
47
|
Kao CH, Lee S, Gold JI, Kable JW. Neural encoding of task-dependent errors during adaptive learning. eLife 2020; 9:58809. [PMID: 33074104 PMCID: PMC7584453 DOI: 10.7554/elife.58809] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2020] [Accepted: 10/15/2020] [Indexed: 01/20/2023] Open
Abstract
Effective learning requires using errors in a task-dependent manner, for example adjusting to errors that result from unpredicted environmental changes but ignoring errors that result from environmental stochasticity. Where and how the brain represents errors in a task-dependent manner and uses them to guide behavior are not well understood. We imaged the brains of human participants performing a predictive-inference task with two conditions that had different sources of errors. Their performance was sensitive to this difference, including more choice switches after fundamental changes versus stochastic fluctuations in reward contingencies. Using multi-voxel pattern classification, we identified task-dependent representations of error magnitude and past errors in posterior parietal cortex. These representations were distinct from representations of the resulting behavioral adjustments in dorsomedial frontal, anterior cingulate, and orbitofrontal cortex. The results provide new insights into how the human brain represents errors in a task-dependent manner and guides subsequent adaptive behavior.
Collapse
Affiliation(s)
- Chang-Hao Kao
- Department of Psychology, University of Pennsylvania, Philadelphia, United States
| | - Sangil Lee
- Department of Psychology, University of Pennsylvania, Philadelphia, United States
| | - Joshua I Gold
- Department of Neuroscience, University of Pennsylvania, Philadelphia, United States
| | - Joseph W Kable
- Department of Psychology, University of Pennsylvania, Philadelphia, United States
| |
Collapse
|
48
|
Bakst L, McGuire JT. Eye movements reflect adaptive predictions and predictive precision. J Exp Psychol Gen 2020; 150:915-929. [PMID: 33048566 DOI: 10.1037/xge0000977] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Successful decision-making depends on the ability to form predictions about uncertain future events. Existing evidence suggests predictive representations are not limited to point estimates but also include information about the associated level of predictive uncertainty. Estimates of predictive uncertainty have an important role in governing the rate at which beliefs are updated in response to new observations. It is not yet known, however, whether the same form of uncertainty-modulated learning occurs naturally and spontaneously when there is no task requirement to express predictions explicitly. Here, we used a gaze-based predictive inference paradigm to show that (a) predictive inference manifested in spontaneous gaze dynamics, (b) feedback-driven updating of spontaneous gaze-based predictions reflected adaptation to environmental statistics, and (c) anticipatory gaze variability tracked predictive uncertainty in an event-by-event manner. Our results demonstrate that sophisticated predictive inference can occur spontaneously and that oculomotor behavior can provide a multidimensional readout of internal predictive beliefs. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
Collapse
Affiliation(s)
- Leah Bakst
- Department of Psychological and Brain Sciences, Boston University
| | - Joseph T McGuire
- Department of Psychological and Brain Sciences, Boston University
| |
Collapse
|
49
|
Henco L, Brandi ML, Lahnakoski JM, Diaconescu AO, Mathys C, Schilbach L. Bayesian modelling captures inter-individual differences in social belief computations in the putamen and insula. Cortex 2020; 131:221-236. [DOI: 10.1016/j.cortex.2020.02.024] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2019] [Revised: 12/21/2019] [Accepted: 02/14/2020] [Indexed: 02/07/2023]
|
50
|
Uncertainty-driven regulation of learning and exploration in adolescents: A computational account. PLoS Comput Biol 2020; 16:e1008276. [PMID: 32997659 PMCID: PMC7549782 DOI: 10.1371/journal.pcbi.1008276] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2019] [Revised: 10/12/2020] [Accepted: 08/20/2020] [Indexed: 01/31/2023] Open
Abstract
Healthy adults flexibly adapt their learning strategies to ongoing changes in uncertainty, a key feature of adaptive behaviour. However, the developmental trajectory of this ability is yet unknown, as developmental studies have not incorporated trial-to-trial variation in uncertainty in their analyses or models. To address this issue, we compared adolescents’ and adults’ trial-to-trial dynamics of uncertainty, learning rate, and exploration in two tasks that assess learning in noisy but otherwise stable environments. In an estimation task—which provides direct indices of trial-specific learning rate—both age groups reduced their learning rate over time, as self-reported uncertainty decreased. Accordingly, the estimation data in both groups was better explained by a Bayesian model with dynamic learning rate (Kalman filter) than by conventional reinforcement-learning models. Furthermore, adolescents’ learning rates asymptoted at a higher level, reflecting an over-weighting of the most recent outcome, and the estimated Kalman-filter parameters suggested that this was due to an overestimation of environmental volatility. In a choice task, both age groups became more likely to choose the higher-valued option over time, but this increase in choice accuracy was smaller in the adolescents. In contrast to the estimation task, we found no evidence for a Bayesian expectation-updating process in the choice task, suggesting that estimation and choice tasks engage different learning processes. However, our modeling results of the choice task suggested that both age groups reduced their degree of exploration over time, and that the adolescents explored overall more than the adults. Finally, age-related differences in exploration parameters from fits to the choice data were mediated by participants’ volatility parameter from fits to the estimation data. Together, these results suggest that adolescents overestimate the rate of environmental change, resulting in elevated learning rates and increased exploration, which may help understand developmental changes in learning and decision-making. To successfully learn the value of stimuli and actions, people should take into account their current (un)certainty about these values: Learning rates and exploration should be high when one’s value estimates are highly uncertain (in the beginning of learning), and decrease over time as evidence accumulates and uncertainty decreases. Recent studies have shown that healthy adults flexibly adapt their learning strategies based on ongoing changes in uncertainty, consistent with normative learning. However, the development of this ability prior to adulthood is yet unknown, as developmental learning studies have not considered trial-to-trial changes in uncertainty. Here, we show that adolescents, as compared to adults, showed a smaller decrease in both learning rate and exploration over time. Computational modeling revealed that both of these effects were due to adolescents overestimating the amount of environmental volatility, which made them more sensitive to recent relative to older evidence. The overestimation of volatility during adolescence may represent the rapidly changing environmental demands during this developmental period, and can help understand the surge in real-life risk taking and exploratory behaviours characteristic of adolescents.
Collapse
|