1
|
Dubinsky JM, Hamid AA. The neuroscience of active learning and direct instruction. Neurosci Biobehav Rev 2024; 163:105737. [PMID: 38796122 DOI: 10.1016/j.neubiorev.2024.105737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 05/13/2024] [Accepted: 05/20/2024] [Indexed: 05/28/2024]
Abstract
Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.
Collapse
Affiliation(s)
- Janet M Dubinsky
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA.
| | - Arif A Hamid
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
2
|
Shen B, Wilson J, Nguyen D, Glimcher PW, Louie K. Origins of noise in both improving and degrading decision making. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.03.26.586597. [PMID: 38915616 PMCID: PMC11195060 DOI: 10.1101/2024.03.26.586597] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/26/2024]
Abstract
Noise is a fundamental problem for information processing in neural systems. In decision-making, noise is assumed to have a primary role in errors and stochastic choice behavior. However, little is known about how noise arising from different sources contributes to value coding and choice behaviors, especially when it interacts with neural computation. Here we examine how noise arising early versus late in the choice process differentially impacts context-dependent choice behavior. We found in model simulations that early and late noise predict opposing context effects: under early noise, contextual information enhances choice accuracy; while under late noise, context degrades choice accuracy. Furthermore, we verified these opposing predictions in experimental human choice behavior. Manipulating early and late noise - by inducing uncertainty in option values and controlling time pressure - produced dissociable positive and negative context effects. These findings reconcile controversial experimental findings in the literature reporting either context-driven impairments or improvements in choice performance, suggesting a unified mechanism for context-dependent choice. More broadly, these findings highlight how different sources of noise can interact with neural computations to differentially modulate behavior. Significance The current study addresses the role of noise origin in decision-making, reconciling controversies around how decision-making is impacted by context. We demonstrate that different types of noise - either arising early during evaluation or late during option comparison - leads to distinct results: with early noise, context enhances choice accuracy, while with late noise, context impairs it. Understanding these dynamics offers potential strategies for improving decision-making in noisy environments and refining existing neural computation models. Overall, our findings advance our understanding of how neural systems handle noise in essential cognitive tasks, suggest a beneficial role for contextual modulation under certain conditions, and highlight the profound implications of noise structure in decision-making.
Collapse
|
3
|
Goris RLT, Coen-Cagli R, Miller KD, Priebe NJ, Lengyel M. Response sub-additivity and variability quenching in visual cortex. Nat Rev Neurosci 2024; 25:237-252. [PMID: 38374462 DOI: 10.1038/s41583-024-00795-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/24/2024] [Indexed: 02/21/2024]
Abstract
Sub-additivity and variability are ubiquitous response motifs in the primary visual cortex (V1). Response sub-additivity enables the construction of useful interpretations of the visual environment, whereas response variability indicates the factors that limit the precision with which the brain can do this. There is increasing evidence that experimental manipulations that elicit response sub-additivity often also quench response variability. Here, we provide an overview of these phenomena and suggest that they may have common origins. We discuss empirical findings and recent model-based insights into the functional operations, computational objectives and circuit mechanisms underlying V1 activity. These different modelling approaches all predict that response sub-additivity and variability quenching often co-occur. The phenomenology of these two response motifs, as well as many of the insights obtained about them in V1, generalize to other cortical areas. Thus, the connection between response sub-additivity and variability quenching may be a canonical motif across the cortex.
Collapse
Affiliation(s)
- Robbe L T Goris
- Center for Perceptual Systems, University of Texas at Austin, Austin, TX, USA.
| | - Ruben Coen-Cagli
- Department of Systems and Computational Biology, Albert Einstein College of Medicine, Bronx, NY, USA
- Dominick P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, USA
- Department of Ophthalmology and Visual Sciences, Albert Einstein College of Medicine, Bronx, NY, USA
| | - Kenneth D Miller
- Center for Theoretical Neuroscience, Columbia University, New York, NY, USA
- Kavli Institute for Brain Science, Columbia University, New York, NY, USA
- Dept. of Neuroscience, College of Physicians and Surgeons, Columbia University, New York, NY, USA
- Morton B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
- Swartz Program in Theoretical Neuroscience, Columbia University, New York, NY, USA
| | - Nicholas J Priebe
- Center for Learning and Memory, University of Texas at Austin, Austin, TX, USA
| | - Máté Lengyel
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK
- Center for Cognitive Computation, Department of Cognitive Science, Central European University, Budapest, Hungary
| |
Collapse
|
4
|
Shih WY, Yu HY, Lee CC, Chou CC, Chen C, Glimcher PW, Wu SW. Electrophysiological population dynamics reveal context dependencies during decision making in human frontal cortex. Nat Commun 2023; 14:7821. [PMID: 38016973 PMCID: PMC10684521 DOI: 10.1038/s41467-023-42092-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 09/28/2023] [Indexed: 11/30/2023] Open
Abstract
Evidence from monkeys and humans suggests that the orbitofrontal cortex (OFC) encodes the subjective value of options under consideration during choice. Data from non-human primates suggests that these value signals are context-dependent, representing subjective value in a way influenced by the decision makers' recent experience. Using electrodes distributed throughout cortical and subcortical structures, human epilepsy patients performed an auction task where they repeatedly reported the subjective values they placed on snack food items. High-gamma activity in many cortical and subcortical sites including the OFC positively correlated with subjective value. Other OFC sites showed signals contextually modulated by the subjective value of previously offered goods-a context dependency predicted by theory but not previously observed in humans. These results suggest that value and value-context signals are simultaneously present but separately represented in human frontal cortical activity.
Collapse
Affiliation(s)
- Wan-Yu Shih
- Institute of Neuroscience, College of Life Sciences, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC.
| | - Hsiang-Yu Yu
- College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
- Department of Epilepsy, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
| | - Cheng-Chia Lee
- Department of Epilepsy, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
- Department of Neurosurgery, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
| | - Chien-Chen Chou
- College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
- Department of Epilepsy, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
| | - Chien Chen
- College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
- Department of Epilepsy, Neurological Institute, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
| | - Paul W Glimcher
- Neuroscience Institute, NYU Grossman School of Medicine, New York, NY, USA.
- Department of Neuroscience and Physiology, NYU Grossman School of Medicine, New York, NY, USA.
| | - Shih-Wei Wu
- Institute of Neuroscience, College of Life Sciences, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC.
- Brain Research Center, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC.
| |
Collapse
|
5
|
Phangwiwat T, Punchongham P, Wongsawat Y, Chatnuntawech I, Wang S, Chunharas C, Sprague T, Woodman GF, Itthipuripat S. Sustained attention operates via dissociable neural mechanisms across different eccentric locations. RESEARCH SQUARE 2023:rs.3.rs-3562186. [PMID: 37986807 PMCID: PMC10659535 DOI: 10.21203/rs.3.rs-3562186/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/22/2023]
Abstract
In primates, foveal and peripheral vision have distinct neural architectures and functions. However, it has been debated if selective attention operates via the same or different neural mechanisms across eccentricities. We tested these alternative accounts by examining the effects of selective attention on the steady-state visually evoked potential (SSVEP) and the fronto-parietal signal measured via EEG from human subjects performing a sustained visuospatial attention task. With a negligible level of eye movements, both SSVEP and SND exhibited the heterogeneous patterns of attentional modulations across eccentricities. Specifically, the attentional modulations of these signals peaked at the parafoveal locations and such modulations wore off as visual stimuli appeared closer to the fovea or further away towards the periphery. However, with a relatively higher level of eye movements, the heterogeneous patterns of attentional modulations of these neural signals were less robust. These data demonstrate that the top-down influence of covert visuospatial attention on early sensory processing in human cortex depends on eccentricity and the level of saccadic responses. Taken together, the results suggest that sustained visuospatial attention operates differently across different eccentric locations, providing new understanding of how attention augments sensory representations regardless of where the attended stimulus appears.
Collapse
Affiliation(s)
- Tanagrit Phangwiwat
- Department of Computer Engineering, King Mongkut's University of Technology Thonburi
| | - Phond Punchongham
- Department of Computer Engineering, King Mongkut's University of Technology Thonburi
| | - Yodchanan Wongsawat
- Department of Biomedical Engineering, Faculty of Engineering, Mahidol University
| | - Itthi Chatnuntawech
- National Nanotechnology Center, National Science and Technology Development Agency
| | - Sisi Wang
- Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam
| | - Chaipat Chunharas
- Chula Neuroscience Center, King Chulalongkorn Memorial Hospital, Thai Red Cross Society
| | - Thomas Sprague
- Psychological and Brain Science, 251, University of California Santa Barbara
| | | | - Sirawaj Itthipuripat
- Neuroscience Center for Research and Innovation (NX), Learning Institute, King Mongkut's University of Technology Thonburi
| |
Collapse
|
6
|
Antono JE, Dang S, Auksztulewicz R, Pooresmaeili A. Distinct Patterns of Connectivity between Brain Regions Underlie the Intra-Modal and Cross-Modal Value-Driven Modulations of the Visual Cortex. J Neurosci 2023; 43:7361-7375. [PMID: 37684031 PMCID: PMC10621764 DOI: 10.1523/jneurosci.0355-23.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Revised: 07/30/2023] [Accepted: 08/26/2023] [Indexed: 09/10/2023] Open
Abstract
Past reward associations may be signaled from different sensory modalities; however, it remains unclear how different types of reward-associated stimuli modulate sensory perception. In this human fMRI study (female and male participants), a visual target was simultaneously presented with either an intra- (visual) or a cross-modal (auditory) cue that was previously associated with rewards. We hypothesized that, depending on the sensory modality of the cues, distinct neural mechanisms underlie the value-driven modulation of visual processing. Using a multivariate approach, we confirmed that reward-associated cues enhanced the target representation in early visual areas and identified the brain valuation regions. Then, using an effective connectivity analysis, we tested three possible patterns of connectivity that could underlie the modulation of the visual cortex: a direct pathway from the frontal valuation areas to the visual areas, a mediated pathway through the attention-related areas, and a mediated pathway that additionally involved sensory association areas. We found evidence for the third model demonstrating that the reward-related information in both sensory modalities is communicated across the valuation and attention-related brain regions. Additionally, the superior temporal areas were recruited when reward was cued cross-modally. The strongest dissociation between the intra- and cross-modal reward-driven effects was observed at the level of the feedforward and feedback connections of the visual cortex estimated from the winning model. These results suggest that, in the presence of previously rewarded stimuli from different sensory modalities, a combination of domain-general and domain-specific mechanisms are recruited across the brain to adjust the visual perception.SIGNIFICANCE STATEMENT Reward has a profound effect on perception, but it is not known whether shared or disparate mechanisms underlie the reward-driven effects across sensory modalities. In this human fMRI study, we examined the reward-driven modulation of the visual cortex by visual (intra-modal) and auditory (cross-modal) reward-associated cues. Using a model-based approach to identify the most plausible pattern of inter-regional effective connectivity, we found that higher-order areas involved in the valuation and attentional processing were recruited by both types of rewards. However, the pattern of connectivity between these areas and the early visual cortex was distinct between the intra- and cross-modal rewards. This evidence suggests that, to effectively adapt to the environment, reward signals may recruit both domain-general and domain-specific mechanisms.
Collapse
Affiliation(s)
- Jessica Emily Antono
- Perception and Cognition Lab, European Neuroscience Institute Goettingen-A Joint Initiative of the University Medical Center Goettingen and the Max-Planck-Society, Germany, Goettingen, 37077, Germany
| | - Shilpa Dang
- Perception and Cognition Lab, European Neuroscience Institute Goettingen-A Joint Initiative of the University Medical Center Goettingen and the Max-Planck-Society, Germany, Goettingen, 37077, Germany
- School of Artificial Intelligence and Data Science, Indian Institute of Technology Jodhpur, Karwar, Jodhpur 342030, India
| | - Ryszard Auksztulewicz
- Center for Cognitive Neuroscience Berlin, Free University Berlin, Berlin, 14195, Germany
| | - Arezoo Pooresmaeili
- Perception and Cognition Lab, European Neuroscience Institute Goettingen-A Joint Initiative of the University Medical Center Goettingen and the Max-Planck-Society, Germany, Goettingen, 37077, Germany
| |
Collapse
|
7
|
Ben-Artzi I, Kessler Y, Nicenboim B, Shahar N. Computational mechanisms underlying latent value updating of unchosen actions. SCIENCE ADVANCES 2023; 9:eadi2704. [PMID: 37862419 PMCID: PMC10588947 DOI: 10.1126/sciadv.adi2704] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/16/2023] [Accepted: 09/20/2023] [Indexed: 10/22/2023]
Abstract
Current studies suggest that individuals estimate the value of their choices based on observed feedback. Here, we ask whether individuals also update the value of their unchosen actions, even when the associated feedback remains unknown. One hundred seventy-eight individuals completed a multi-armed bandit task, making choices to gain rewards. We found robust evidence suggesting latent value updating of unchosen actions based on the chosen action's outcome. Computational modeling results suggested that this effect is mainly explained by a value updating mechanism whereby individuals integrate the outcome history for choosing an option with that of rejecting the alternative. Properties of the deliberation (i.e., duration/difficulty) did not moderate the latent value updating of unchosen actions, suggesting that memory traces generated during deliberation might take a smaller role in this specific phenomenon than previously thought. We discuss the mechanisms facilitating credit assignment to unchosen actions and their implications for human decision-making.
Collapse
Affiliation(s)
- Ido Ben-Artzi
- School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
- Minducate Science of Learning Research and Innovation Center of the Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
| | - Yoav Kessler
- Department of Psychology and School of Brain Sciences and Cognition, Ben Gurion University of the Negev, Be'er Sheva, Israel
| | - Bruno Nicenboim
- Department of Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg, Netherlands
| | - Nitzan Shahar
- School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
| |
Collapse
|
8
|
Noda M, Tanabe HC, Yoshioka A, Kimura M. Neural substrates on the judgment of sequential benefits and losses. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2023:10.3758/s13415-023-01087-3. [PMID: 37347352 DOI: 10.3758/s13415-023-01087-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Accepted: 02/27/2023] [Indexed: 06/23/2023]
Abstract
People need to adapt to situations where they experience sequential benefits (or losses) to ensure survival. This study investigated the neural substrates involved in judgments of sequential benefits and losses. A total of 29 healthy volunteers participated in this study, in which they were asked to participate in a game of purchasing stocks while a magnetic resonance imaging scan was performed. This game had two main types of trials: (1) participants received four sequential financial benefits (or losses), and (2) participants received an equal amount of benefits (or losses) immediately. The results showed greater activation of the orbitofrontal cortex (OFC) when four benefits were received sequentially than when an equal amount of benefits was received immediately. This indicates that the OFC plays a crucial role in the process of mental integration of sequential benefits and interpretation of their valuations. It also showed greater activation of the dorsal striatum when four sequental losses were received than when an equal amount of losses was received immediately. However, it cannot be concluded that activation of the dorsal striatum reflects the differences between sequential and immediate losses, because previous studies have not confirmed this perspective. Therefore, it is necessary to clarify the function of the striatum in processing these losses.
Collapse
Affiliation(s)
- Masayo Noda
- Department of Multidisciplinary Psychology, Kinjo Gakuin University, 2-1723 Omori, Moriyama-ku, Nagoya, Aichi, 463-8521, Japan.
| | - Hiroki C Tanabe
- Division of Psychology, Department of Cognitive and Psychological Sciences, Graduate School of Informatics, Nagoya University, Nagoya, Japan
| | - Ayumi Yoshioka
- Division of Psychology, Department of Cognitive and Psychological Sciences, Graduate School of Informatics, Nagoya University, Nagoya, Japan
- Division of Cerebral Integration, Department of System Neuroscience, National Institute for Physiological Sciences, Okazaki, Japan
| | | |
Collapse
|
9
|
Zhao B, Wang R, Zhu Z, Yang Q, Chen A. The computational rules of cross-modality suppression in the visual posterior sylvian area. iScience 2023; 26:106973. [PMID: 37378331 PMCID: PMC10291470 DOI: 10.1016/j.isci.2023.106973] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2022] [Revised: 03/13/2023] [Accepted: 05/23/2023] [Indexed: 06/29/2023] Open
Abstract
The macaque visual posterior sylvian area (VPS) is an area with neurons responding selectively to heading direction in both visual and vestibular modalities, but how VPS neurons combined these two sensory signals is still unknown. In contrast to the subadditive characteristics in the medial superior temporal area (MSTd), responses in VPS were dominated by vestibular signals, with approximately a winner-take-all competition. The conditional Fisher information analysis shows that VPS neural population encodes information from distinct sensory modalities under large and small offset conditions, which differs from MSTd whose neural population contains more information about visual stimuli in both conditions. However, the combined responses of single neurons in both areas can be well fit by weighted linear sums of unimodal responses. Furthermore, a normalization model captured most vestibular and visual interaction characteristics for both VPS and MSTd, indicating the divisive normalization mechanism widely exists in the cortex.
Collapse
Affiliation(s)
- Bin Zhao
- Key Laboratory of Brain Functional Genomics, East China Normal University, Shanghai 200062, China
| | - Rong Wang
- Key Laboratory of Brain Functional Genomics, East China Normal University, Shanghai 200062, China
| | - Zhihua Zhu
- CAS Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai 200031, China
| | - Qianli Yang
- CAS Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai 200031, China
| | - Aihua Chen
- Key Laboratory of Brain Functional Genomics, East China Normal University, Shanghai 200062, China
| |
Collapse
|
10
|
Speer SPH, Keysers C, Barrios JC, Teurlings CJS, Smidts A, Boksem MAS, Wager TD, Gazzola V. A multivariate brain signature for reward. Neuroimage 2023; 271:119990. [PMID: 36878456 DOI: 10.1016/j.neuroimage.2023.119990] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2022] [Revised: 02/20/2023] [Accepted: 02/25/2023] [Indexed: 03/07/2023] Open
Abstract
The processing of reinforcers and punishers is crucial to adapt to an ever changing environment and its dysregulation is prevalent in mental health and substance use disorders. While many human brain measures related to reward have been based on activity in individual brain regions, recent studies indicate that many affective and motivational processes are encoded in distributed systems that span multiple regions. Consequently, decoding these processes using individual regions yields small effect sizes and limited reliability, whereas predictive models based on distributed patterns yield larger effect sizes and excellent reliability. To create such a predictive model for the processes of rewards and losses, termed the Brain Reward Signature (BRS), we trained a model to predict the signed magnitude of monetary rewards on the Monetary Incentive Delay task (MID; N = 39) and achieved a highly significant decoding performance (92% for decoding rewards versus losses). We subsequently demonstrate the generalizability of our signature on another version of the MID in a different sample (92% decoding accuracy; N = 12) and on a gambling task from a large sample (73% decoding accuracy, N = 1084). We further provided preliminary data to characterize the specificity of the signature by illustrating that the signature map generates estimates that significantly differ between rewarding and negative feedback (92% decoding accuracy) but do not differ for conditions that differ in disgust rather than reward in a novel Disgust-Delay Task (N = 39). Finally, we show that passively viewing positive and negatively valenced facial expressions loads positively on our signature, in line with previous studies on morbid curiosity. We thus created a BRS that can accurately predict brain responses to rewards and losses in active decision making tasks, and that possibly relates to information seeking in passive observational tasks.
Collapse
Affiliation(s)
- Sebastian P H Speer
- Social Brain Lab, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands; Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA
| | - Christian Keysers
- Social Brain Lab, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands; Brain and Cognition, Department of Psychology, University of Amsterdam, The Netherlands
| | | | - Cas J S Teurlings
- Social Brain Lab, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands
| | - Ale Smidts
- Rotterdam School of Management, Erasmus University, 3062 PA Rotterdam, The Netherlands
| | - Maarten A S Boksem
- Rotterdam School of Management, Erasmus University, 3062 PA Rotterdam, The Netherlands
| | - Tor D Wager
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755, USA
| | - Valeria Gazzola
- Social Brain Lab, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands.
| |
Collapse
|
11
|
Wittmann MK, Scheuplein M, Gibbons SG, Noonan MP. Local and global reward learning in the lateral frontal cortex show differential development during human adolescence. PLoS Biol 2023; 21:e3002010. [PMID: 36862726 PMCID: PMC10013901 DOI: 10.1371/journal.pbio.3002010] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Revised: 03/14/2023] [Accepted: 01/20/2023] [Indexed: 03/03/2023] Open
Abstract
Reward-guided choice is fundamental for adaptive behaviour and depends on several component processes supported by prefrontal cortex. Here, across three studies, we show that two such component processes, linking reward to specific choices and estimating the global reward state, develop during human adolescence and are linked to the lateral portions of the prefrontal cortex. These processes reflect the assignment of rewards contingently to local choices, or noncontingently, to choices that make up the global reward history. Using matched experimental tasks and analysis platforms, we show the influence of both mechanisms increase during adolescence (study 1) and that lesions to lateral frontal cortex (that included and/or disconnected both orbitofrontal and insula cortex) in human adult patients (study 2) and macaque monkeys (study 3) impair both local and global reward learning. Developmental effects were distinguishable from the influence of a decision bias on choice behaviour, known to depend on medial prefrontal cortex. Differences in local and global assignments of reward to choices across adolescence, in the context of delayed grey matter maturation of the lateral orbitofrontal and anterior insula cortex, may underlie changes in adaptive behaviour.
Collapse
Affiliation(s)
- Marco K. Wittmann
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory, Oxford, United Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Headington, Oxford, United Kingdom
- Department of Experimental Psychology, University College London, London, United Kingdom
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, United Kingdom
| | - Maximilian Scheuplein
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory, Oxford, United Kingdom
- Institute of Education and Child Studies, Leiden University, Leiden, the Netherlands
| | - Sophie G. Gibbons
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory, Oxford, United Kingdom
- MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, United Kingdom
| | - MaryAnn P. Noonan
- Department of Experimental Psychology, University of Oxford, Radcliffe Observatory, Oxford, United Kingdom
- Department of Psychology, University of York, York, United Kingdom
- * E-mail:
| |
Collapse
|
12
|
Kohl C, Wong MXM, Wong JJ, Rushworth MFS, Chau BKH. Intraparietal stimulation disrupts negative distractor effects in human multi-alternative decision-making. eLife 2023; 12:e75007. [PMID: 36811348 PMCID: PMC9946441 DOI: 10.7554/elife.75007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2021] [Accepted: 11/22/2022] [Indexed: 02/24/2023] Open
Abstract
There has been debate about whether addition of an irrelevant distractor option to an otherwise binary decision influences which of the two choices is taken. We show that disparate views on this question are reconciled if distractors exert two opposing but not mutually exclusive effects. Each effect predominates in a different part of decision space: (1) a positive distractor effect predicts high-value distractors improve decision-making; (2) a negative distractor effect, of the type associated with divisive normalisation models, entails decreased accuracy with increased distractor values. Here, we demonstrate both distractor effects coexist in human decision making but in different parts of a decision space defined by the choice values. We show disruption of the medial intraparietal area (MIP) by transcranial magnetic stimulation (TMS) increases positive distractor effects at the expense of negative distractor effects. Furthermore, individuals with larger MIP volumes are also less susceptible to the disruption induced by TMS. These findings also demonstrate a causal link between MIP and the impact of distractors on decision-making via divisive normalisation.
Collapse
Affiliation(s)
- Carmen Kohl
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic UniversityHong KongChina
- Department Neuroscience, Carney Institute for Brain Sciences, Brown UniversityProvidenceUnited States
| | - Michelle XM Wong
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic UniversityHong KongChina
| | - Jing Jun Wong
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic UniversityHong KongChina
| | | | - Bolton KH Chau
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic UniversityHong KongChina
- University Research Facility in Behavioral and Systems Neuroscience, The Hong Kong Polytechnic UniversityHong KongChina
| |
Collapse
|
13
|
Li Y, Daddaoua N, Horan M, Foley NC, Gottlieb J. Uncertainty modulates visual maps during noninstrumental information demand. Nat Commun 2022; 13:5911. [PMID: 36207316 PMCID: PMC9547007 DOI: 10.1038/s41467-022-33585-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Accepted: 09/22/2022] [Indexed: 11/23/2022] Open
Abstract
Animals are intrinsically motivated to obtain information independently of instrumental incentives. This motivation depends on two factors: a desire to resolve uncertainty by gathering accurate information and a desire to obtain positively-valenced observations, which predict favorable rather than unfavorable outcomes. To understand the neural mechanisms, we recorded parietal cortical activity implicated in prioritizing stimuli for spatial attention and gaze, in a task in which monkeys were free (but not trained) to obtain information about probabilistic non-contingent rewards. We show that valence and uncertainty independently modulated parietal neuronal activity, and uncertainty but not reward-related enhancement consistently correlated with behavioral sensitivity. The findings suggest uncertainty-driven and valence-driven information demand depend on partially distinct pathways, with the former being consistently related to parietal responses and the latter depending on additional mechanisms implemented in downstream structures. Curiosity is motivated by uncertainty and valence, but how uncertainty and valence are encoded in the brain remains poorly understood. Here, the authors show that parietal neurons are enhanced by both factors, but that they specifically predict visual information seeking based on uncertainty.
Collapse
Affiliation(s)
- Yvonne Li
- Department of Neuroscience, Columbia University, New York, NY, USA.,Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Nabil Daddaoua
- Department of Neuroscience, Columbia University, New York, NY, USA.,Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Mattias Horan
- Department of Neuroscience, Columbia University, New York, NY, USA
| | - Nicholas C Foley
- Department of Neuroscience, Columbia University, New York, NY, USA.,Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Jacqueline Gottlieb
- Department of Neuroscience, Columbia University, New York, NY, USA. .,Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA. .,Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| |
Collapse
|
14
|
Divisive normalization is an efficient code for multivariate Pareto-distributed environments. Proc Natl Acad Sci U S A 2022; 119:e2120581119. [PMID: 36161961 PMCID: PMC9546555 DOI: 10.1073/pnas.2120581119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
Divisive normalization is a canonical computation in the brain, observed across neural systems, that is often considered to be an implementation of the efficient coding principle. We provide a theoretical result that makes the conditions under which divisive normalization is an efficient code analytically precise: We show that, in a low-noise regime, encoding an n-dimensional stimulus via divisive normalization is efficient if and only if its prevalence in the environment is described by a multivariate Pareto distribution. We generalize this multivariate analog of histogram equalization to allow for arbitrary metabolic costs of the representation, and show how different assumptions on costs are associated with different shapes of the distributions that divisive normalization efficiently encodes. Our result suggests that divisive normalization may have evolved to efficiently represent stimuli with Pareto distributions. We demonstrate that this efficiently encoded distribution is consistent with stylized features of naturalistic stimulus distributions such as their characteristic conditional variance dependence, and we provide empirical evidence suggesting that it may capture the statistics of filter responses to naturalistic images. Our theoretical finding also yields empirically testable predictions across sensory domains on how the divisive normalization parameters should be tuned to features of the input distribution.
Collapse
|
15
|
Neuronal Response to Reward and Luminance in Macaque LIP During Saccadic Choice. Neurosci Bull 2022; 39:14-28. [PMID: 36114983 PMCID: PMC9849667 DOI: 10.1007/s12264-022-00948-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Accepted: 06/18/2022] [Indexed: 01/22/2023] Open
Abstract
Recent work in decision neuroscience suggests that visual saliency can interact with reward-based choice, and the lateral intraparietal cortex (LIP) is implicated in this process. In this study, we recorded from LIP neurons while monkeys performed a two alternative choice task in which the reward and luminance associated with each offer were varied independently. We discovered that the animal's choice was dictated by the reward amount while the luminance had a marginal effect. In the LIP, neuronal activity corresponded well with the animal's choice pattern, in that a majority of reward-modulated neurons encoded the reward amount in the neuron's preferred hemifield with a positive slope. In contrast, compared to their responses to low luminance, an approximately equal proportion of luminance-sensitive neurons responded to high luminance with increased or decreased activity, leading to a much weaker population-level response. Meanwhile, in the non-preferred hemifield, the strength of encoding for reward amount and luminance was positively correlated, suggesting the integration of these two factors in the LIP. Moreover, neurons encoding reward and luminance were homogeneously distributed along the anterior-posterior axis of the LIP. Overall, our study provides further evidence supporting the neural instantiation of a priority map in the LIP in reward-based decisions.
Collapse
|
16
|
Reply to Pirrone and Tsetsos: Robust evidence for enhanced high-value sensitivity. Proc Natl Acad Sci U S A 2022; 119:e2209521119. [PMID: 35969800 PMCID: PMC9459315 DOI: 10.1073/pnas.2209521119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
|
17
|
On multiple sources of value sensitivity. Proc Natl Acad Sci U S A 2022; 119:e2207053119. [PMID: 35969802 PMCID: PMC9459316 DOI: 10.1073/pnas.2207053119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
|
18
|
Louie K. Asymmetric and adaptive reward coding via normalized reinforcement learning. PLoS Comput Biol 2022; 18:e1010350. [PMID: 35862443 PMCID: PMC9345478 DOI: 10.1371/journal.pcbi.1010350] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2022] [Revised: 08/02/2022] [Accepted: 07/01/2022] [Indexed: 11/18/2022] Open
Abstract
Learning is widely modeled in psychology, neuroscience, and computer science by prediction error-guided reinforcement learning (RL) algorithms. While standard RL assumes linear reward functions, reward-related neural activity is a saturating, nonlinear function of reward; however, the computational and behavioral implications of nonlinear RL are unknown. Here, we show that nonlinear RL incorporating the canonical divisive normalization computation introduces an intrinsic and tunable asymmetry in prediction error coding. At the behavioral level, this asymmetry explains empirical variability in risk preferences typically attributed to asymmetric learning rates. At the neural level, diversity in asymmetries provides a computational mechanism for recently proposed theories of distributional RL, allowing the brain to learn the full probability distribution of future rewards. This behavioral and computational flexibility argues for an incorporation of biologically valid value functions in computational models of learning and decision-making. Reinforcement learning models are widely used to characterize reward-driven learning in biological and computational agents. Standard reinforcement learning models use linear value functions, despite strong empirical evidence that biological value representations are nonlinear functions of external rewards. Here, we examine the properties of a biologically-based nonlinear reinforcement learning algorithm employing the canonical divisive normalization function, a neural computation commonly found in sensory, cognitive, and reward coding. We show that this normalized reinforcement learning algorithm implements a simple but powerful control of how reward learning reflects relative gains and losses. This property explains diverse behavioral and neural phenomena, and suggests the importance of using biologically valid value functions in computational models of learning and decision-making.
Collapse
Affiliation(s)
- Kenway Louie
- Center for Neural Science, New York University, New York, United States of America
- Neuroscience Institute, New York University Grossman School of Medicine, New York, United States of America
- * E-mail:
| |
Collapse
|
19
|
Abstract
People with damage to the orbitofrontal cortex (OFC) have specific problems making decisions, whereas their other cognitive functions are spared. Neurophysiological studies have shown that OFC neurons fire in proportion to the value of anticipated outcomes. Thus, a central role of the OFC is to guide optimal decision-making by signalling values associated with different choices. Until recently, this view of OFC function dominated the field. New data, however, suggest that the OFC may have a much broader role in cognition by representing cognitive maps that can be used to guide behaviour and that value is just one of many variables that are important for behavioural control. In this Review, we critically evaluate these two alternative accounts of OFC function and examine how they might be reconciled.
Collapse
Affiliation(s)
- Eric B Knudsen
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California, USA
| | - Joni D Wallis
- Department of Psychology and Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California, USA.
| |
Collapse
|
20
|
Dennison JB, Sazhin D, Smith DV. Decision neuroscience and neuroeconomics: Recent progress and ongoing challenges. WILEY INTERDISCIPLINARY REVIEWS. COGNITIVE SCIENCE 2022; 13:e1589. [PMID: 35137549 PMCID: PMC9124684 DOI: 10.1002/wcs.1589] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/05/2020] [Revised: 11/28/2021] [Accepted: 12/21/2021] [Indexed: 01/10/2023]
Abstract
In the past decade, decision neuroscience and neuroeconomics have developed many new insights in the study of decision making. This review provides an overarching update on how the field has advanced in this time period. Although our initial review a decade ago outlined several theoretical, conceptual, methodological, empirical, and practical challenges, there has only been limited progress in resolving these challenges. We summarize significant trends in decision neuroscience through the lens of the challenges outlined for the field and review examples where the field has had significant, direct, and applicable impacts across economics and psychology. First, we review progress on topics including reward learning, explore-exploit decisions, risk and ambiguity, intertemporal choice, and valuation. Next, we assess the impacts of emotion, social rewards, and social context on decision making. Then, we follow up with how individual differences impact choices and new exciting developments in the prediction and neuroforecasting of future decisions. Finally, we consider how trends in decision-neuroscience research reflect progress toward resolving past challenges, discuss new and exciting applications of recent research, and identify new challenges for the field. This article is categorized under: Psychology > Reasoning and Decision Making Psychology > Emotion and Motivation.
Collapse
Affiliation(s)
- Jeffrey B Dennison
- Department of Psychology, Temple University, Philadelphia, Pennsylvania, USA
| | - Daniel Sazhin
- Department of Psychology, Temple University, Philadelphia, Pennsylvania, USA
| | - David V Smith
- Department of Psychology, Temple University, Philadelphia, Pennsylvania, USA
| |
Collapse
|
21
|
Efficiently irrational: deciphering the riddle of human choice. Trends Cogn Sci 2022; 26:669-687. [PMID: 35643845 PMCID: PMC9283329 DOI: 10.1016/j.tics.2022.04.007] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Revised: 04/18/2022] [Accepted: 04/20/2022] [Indexed: 11/18/2022]
Abstract
For the past half-century, cognitive and social scientists have struggled with the irrationalities of human choice behavior; people consistently make choices that are logically inconsistent. Is human choice behavior evolutionarily adaptive or is it an inefficient patchwork of competing mechanisms? In this review, I present an interdisciplinary synthesis arguing for a novel interpretation: choice is efficiently irrational. Connecting findings across disciplines suggests that observed choice behavior reflects a precise optimization of the trade-off between the costs of increasing the precision of the choice mechanism and the declining benefits that come as precision increases. Under these constraints, a rationally imprecise strategy emerges that works toward optimal efficiency rather than toward optimal rationality. This approach rationalizes many of the puzzling inconsistencies of human choice behavior, explaining why these inconsistencies arise as an optimizing solution in biological choosers.
Collapse
|
22
|
Donut-like organization of inhibition underlies categorical neural responses in the midbrain. Nat Commun 2022; 13:1680. [PMID: 35354821 PMCID: PMC8967821 DOI: 10.1038/s41467-022-29318-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2020] [Accepted: 03/10/2022] [Indexed: 11/08/2022] Open
Abstract
Categorical neural responses underlie various forms of selection and decision-making. Such binary-like responses promote robust signaling of the winner in the presence of input ambiguity and neural noise. Here, we show that a ‘donut-like’ inhibitory mechanism in which each competing option suppresses all options except itself, is highly effective at generating categorical neural responses. It surpasses motifs of feedback inhibition, recurrent excitation, and divisive normalization invoked frequently in decision-making models. We demonstrate experimentally not only that this mechanism operates in the midbrain spatial selection network in barn owls, but also that it is necessary for categorical signaling by it. The functional pattern of neural inhibition in the midbrain forms an exquisitely structured ‘multi-holed’ donut consistent with this network’s combinatorial inhibitory function for stimulus selection. Additionally, modeling reveals a generalizable neural implementation of the donut-like motif for categorical selection. Self-sparing inhibition may, therefore, be a powerful circuit module central to categorization. Decision making is facilitated by categorical neuronal responses, which robustly signal a winner despite input noise. In this study, the authors demonstrate that a donut-like inhibition motif effectively generates such categorical responses.
Collapse
|
23
|
Frömer R, Shenhav A. Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making. Neurosci Biobehav Rev 2022; 134:104483. [PMID: 34902441 PMCID: PMC8844247 DOI: 10.1016/j.neubiorev.2021.12.006] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2021] [Revised: 12/01/2021] [Accepted: 12/04/2021] [Indexed: 12/26/2022]
Abstract
While often seeming to investigate rather different problems, research into value-based decision making and cognitive control have historically offered parallel insights into how people select thoughts and actions. While the former studies how people weigh costs and benefits to make a decision, the latter studies how they adjust information processing to achieve their goals. Recent work has highlighted ways in which decision-making research can inform our understanding of cognitive control. Here, we provide the complementary perspective: how cognitive control research has informed understanding of decision-making. We highlight three particular areas of research where this critical interchange has occurred: (1) how different types of goals shape the evaluation of choice options, (2) how people use control to adjust the ways they make their decisions, and (3) how people monitor decisions to inform adjustments to control at multiple levels and timescales. We show how adopting this alternate viewpoint offers new insight into the determinants of both decisions and control; provides alternative interpretations for common neuroeconomic findings; and generates fruitful directions for future research.
Collapse
Affiliation(s)
- R Frömer
- Cognitive, Linguistic, and Psychological Sciences, Carney Institute for Brain Science, Brown University, Providence, RI, United States.
| | - A Shenhav
- Cognitive, Linguistic, and Psychological Sciences, Carney Institute for Brain Science, Brown University, Providence, RI, United States.
| |
Collapse
|
24
|
Al-Mohammad A, Schultz W. Reward Value Revealed by Auction in Rhesus Monkeys. J Neurosci 2022; 42:1510-1528. [PMID: 34937703 PMCID: PMC8883853 DOI: 10.1523/jneurosci.1275-21.2021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2021] [Revised: 11/25/2021] [Accepted: 12/01/2021] [Indexed: 11/21/2022] Open
Abstract
Economic choice is thought to involve the elicitation of the subjective values of the choice options. Thus far, value estimation in animals has relied on stochastic choices between multiple options presented in repeated trials and expressed from averages of dozens of trials. However, subjective reward valuations are made moment-to-moment and do not always require alternative options; their consequences are usually felt immediately. Here, we describe a Becker-DeGroot-Marschak (BDM) auction-like mechanism that provides more direct and simple valuations with immediate consequences. The BDM encourages agents to truthfully reveal their true subjective value in individual choices ("incentive compatibility"). Male monkeys reliably placed well-ranked BDM bids for up to five juice volumes while paying from a water budget. The bids closely approximated the average subjective values estimated with conventional binary choices (BCs), thus demonstrating procedural invariance and aligning with the wealth of knowledge acquired with these less direct estimation methods. The feasibility of BDM bidding in monkeys paves the way for an analysis of subjective neuronal value signals in single trials rather than from averages; the feasibility also bridges the gap to the increasingly used BDM method in human neuroeconomics.SIGNIFICANCE STATEMENT The subjective economic value of rewards cannot be measured directly but must be inferred from observable behavior. Until now, the estimation method in animals was rather complex and required comparison between several choice options during repeated choices; thus, such methods did not respect the imminence of the outcome from individual choices. However, human economic research has developed a simple auction-like procedure that can reveal in a direct and immediate manner the true subjective value in individual choices [Becker-DeGroot-Marschak (BDM) mechanism]. The current study implemented this mechanism in rhesus monkeys and demonstrates its usefulness for eliciting meaningful value estimates of liquid rewards. The mechanism allows future neurophysiological assessment of subjective reward value signals in single trials of controlled animal tasks.
Collapse
Affiliation(s)
- Alaa Al-Mohammad
- Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge CB2 3DY, United Kingdom
| | - Wolfram Schultz
- Department of Physiology, Development and Neuroscience, University of Cambridge, Cambridge CB2 3DY, United Kingdom
| |
Collapse
|
25
|
Hirschbichler ST, Rothwell JC, Manohar SG. Dopamine increases risky choice while D2 blockade shortens decision time. Exp Brain Res 2022; 240:3351-3360. [PMID: 36350356 PMCID: PMC9678996 DOI: 10.1007/s00221-022-06501-9] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2022] [Accepted: 10/27/2022] [Indexed: 11/11/2022]
Abstract
Dopamine is crucially involved in decision-making and overstimulation within dopaminergic pathways can lead to impulsive behaviour, including a desire to take risks and reduced deliberation before acting. These behavioural changes are side effects of treatment with dopaminergic drugs in Parkinson disease, but their likelihood of occurrence is difficult to predict and may be influenced by the individual's baseline endogenous dopamine state, and indeed correlate with sensation-seeking personality traits. We here collected data on a standard gambling task in healthy volunteers given either placebo, 2.5 mg of the dopamine antagonist haloperidol or 100/25 mg of the dopamine precursor levodopa in a within-subject design. We found an increase in risky choices on levodopa. Choices were, however, made faster on haloperidol with no effect of levodopa on deliberation time. Shortened deliberation times on haloperidol occurred in low sensation-seekers only, suggesting a correlation between sensation-seeking personality trait and baseline dopamine levels. We hypothesise that levodopa increases risk-taking behaviour via overstimulation at both D1 and D2 receptor level, while a single low dose of haloperidol, as previously reported (Frank and O'Reilly 2006), may block D2 receptors pre- and post-synaptically and may paradoxically lead to higher striatal dopamine acting on remaining striatal D1 receptors, causing speedier decision without influencing risk tolerance. These effects could also fit with a recently proposed computational model of the basal ganglia (Moeller and Bogacz 2019; Moeller et al. 2021). Furthermore, our data suggest that the actual dopaminergic drug effect may be dependent on the individual's baseline dopamine state, which may influence our therapeutic decision as clinicians in the future.
Collapse
Affiliation(s)
- Stephanie T. Hirschbichler
- Department of Clinical and Movement Neurosciences, UCL Queen Square Institute of Neurology, London, WC1N 3BG UK ,Department of Neurology, University Hospital St. Pölten, Dunant-Platz 1, 3100 St. Pölten, Austria ,Karl Landsteiner University of Health Sciences, Dr. Karl-Dorrek-Straße 30, 3500 Krems, Austria
| | - John C. Rothwell
- Department of Clinical and Movement Neurosciences, UCL Queen Square Institute of Neurology, London, WC1N 3BG UK
| | - Sanjay G. Manohar
- Department of Clinical and Movement Neurosciences, UCL Queen Square Institute of Neurology, London, WC1N 3BG UK ,Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, Oxford, OX3 9DU UK
| |
Collapse
|
26
|
Yao Z, Hessburg JP, Francis JT. Normalization by valence and motivational intensity in the sensorimotor cortices (PMd, M1, and S1). Sci Rep 2021; 11:24221. [PMID: 34930930 PMCID: PMC8688489 DOI: 10.1038/s41598-021-03200-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2020] [Accepted: 11/26/2021] [Indexed: 12/27/2022] Open
Abstract
Our brain's ability to represent vast amounts of information, such as continuous ranges of reward spanning orders of magnitude, with limited dynamic range neurons, may be possible due to normalization. Recently our group and others have shown that the sensorimotor cortices are sensitive to reward value. Here we ask if psychological affect causes normalization of the sensorimotor cortices by modulating valence and motivational intensity. We had two non-human primates (NHP) subjects (one male bonnet macaque and one female rhesus macaque) make visually cued grip-force movements while simultaneously cueing the level of possible reward if successful, or timeout punishment, if unsuccessful. We recorded simultaneously from 96 electrodes in each the following: caudal somatosensory, rostral motor, and dorsal premotor cortices (cS1, rM1, PMd). We utilized several normalization models for valence and motivational intensity in all three regions. We found three types of divisive normalized relationships between neural activity and the representation of valence and motivation, linear, sigmodal, and hyperbolic. The hyperbolic relationships resemble receptive fields in psychological affect space, where a unit is susceptible to a small range of the valence/motivational space. We found that these cortical regions have both strong valence and motivational intensity representations.
Collapse
Affiliation(s)
- Zhao Yao
- Department of Physiology and Pharmacology, State University of New York Downstate Medical Center, Brooklyn, NY, 11203, USA
| | - John P Hessburg
- Department of Physiology and Pharmacology, State University of New York Downstate Medical Center, Brooklyn, NY, 11203, USA
| | - Joseph Thachil Francis
- Departments of Biomedical Engineering and Electrical and Computer Engineering, Cullen College of Engineering at The University of Houston, Houston, TX, 77204, USA.
| |
Collapse
|
27
|
Pirrone A, Reina A, Stafford T, Marshall JAR, Gobet F. Magnitude-sensitivity: rethinking decision-making. Trends Cogn Sci 2021; 26:66-80. [PMID: 34750080 DOI: 10.1016/j.tics.2021.10.006] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2021] [Revised: 10/05/2021] [Accepted: 10/05/2021] [Indexed: 11/25/2022]
Abstract
Magnitude-sensitivity refers to the result that performance in decision-making, across domains and organisms, is affected by the total value of the possible alternatives. This simple result offers a window into fundamental issues in decision-making and has led to a reconsideration of ecological decision-making, prominent computational models of decision-making, and optimal decision-making. Moreover, magnitude-sensitivity has inspired the design of new robotic systems that exploit natural solutions and apply optimal decision-making policies. In this article, we review the key theoretical and empirical results about magnitude-sensitivity and highlight the importance that this phenomenon has for the understanding of decision-making. Furthermore, we discuss open questions and ideas for future research.
Collapse
Affiliation(s)
- Angelo Pirrone
- Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science, London, UK.
| | - Andreagiovanni Reina
- Institute for Interdisciplinary Studies on Artificial Intelligence (IRIDIA), Université Libre de Bruxelles, Brussels, Belgium
| | - Tom Stafford
- Department of Psychology, University of Sheffield, Sheffield, UK
| | | | - Fernand Gobet
- Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science, London, UK
| |
Collapse
|
28
|
Klímová M, Bloem IM, Ling S. The specificity of orientation-tuned normalization within human early visual cortex. J Neurophysiol 2021; 126:1536-1546. [PMID: 34550028 PMCID: PMC8794056 DOI: 10.1152/jn.00203.2021] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Revised: 09/20/2021] [Accepted: 09/20/2021] [Indexed: 11/22/2022] Open
Abstract
Normalization within visual cortex is modulated by contextual influences; stimuli sharing similar features suppress each other more than dissimilar stimuli. This feature-tuned component of suppression depends on multiple factors, including the orientation content of stimuli. Indeed, pairs of stimuli arranged in a center-surround configuration attenuate each other's response to a greater degree when oriented collinearly than when oriented orthogonally. Although numerous studies have examined the nature of surround suppression at these two extremes, far less is known about how the strength of tuned normalization varies as a function of continuous changes in orientation similarity, particularly in humans. In this study, we used functional magnetic resonance imaging (fMRI) to examine the bandwidth of orientation-tuned suppression within human visual cortex. Blood-oxygen-level-dependent (BOLD) responses were acquired as participants viewed a full-field circular stimulus composed of wedges of orientation-bandpass filtered noise. This stimulus configuration allowed us to parametrically vary orientation differences between neighboring wedges in gradual steps between collinear and orthogonal. We found the greatest suppression for collinearly arranged stimuli with a gradual increase in BOLD response as the orientation content became more dissimilar. We quantified the tuning width of orientation-tuned suppression, finding that the voxel-wise bandwidth of orientation tuned normalization was between 20° and 30°, and did not differ substantially between early visual areas. Voxel-wise analyses revealed that suppression width covaried with retinotopic preference, with the tightest bandwidths at outer eccentricities. Having an estimate of orientation-tuned suppression bandwidth can serve to constrain models of tuned normalization, establishing the precise degree to which suppression strength depends on similarity between visual stimulus components.NEW & NOTEWORTHY Neurons in the early visual cortex are subject to divisive normalization, but the feature-tuning aspect of this computation remains understudied, particularly in humans. We investigated orientation tuning of normalization in human early visual cortex using fMRI and estimated the bandwidth of the tuned normalization function across observers. Our findings provide a characterization of tuned normalization in early visual cortex that could help constrain models of divisive normalization in vision.
Collapse
Affiliation(s)
- Michaela Klímová
- Department of Psychological and Brain Sciences, Boston University, Boston, Massachusetts
- Center for Systems Neuroscience, Boston University, Boston, Massachusetts
| | - Ilona M Bloem
- Department of Psychological and Brain Sciences, Boston University, Boston, Massachusetts
- Center for Systems Neuroscience, Boston University, Boston, Massachusetts
- Department of Psychology, New York University, New York City, New York
| | - Sam Ling
- Department of Psychological and Brain Sciences, Boston University, Boston, Massachusetts
- Center for Systems Neuroscience, Boston University, Boston, Massachusetts
| |
Collapse
|
29
|
Setting the space for deliberation in decision-making. Cogn Neurodyn 2021; 15:743-755. [PMID: 34603540 DOI: 10.1007/s11571-021-09681-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2020] [Revised: 03/12/2021] [Accepted: 04/16/2021] [Indexed: 10/21/2022] Open
Abstract
Decision-making models in the behavioral, cognitive, and neural sciences typically consist of forced-choice paradigms with two alternatives. While theoretically it is feasible to translate any decision situation to a sequence of binary choices, real-life decision-making is typically more complex and nonlinear, involving choices among multiple items, graded judgments, and deferments of decision-making. Here, we discuss how the complexity of real-life decision-making can be addressed using conventional decision-making models by focusing on the interactive dynamics between criteria settings and the collection of evidence. Decision-makers can engage in multi-stage, parallel decision-making by exploiting the space for deliberation, with non-binary readings of evidence available at any point in time. The interactive dynamics principally adhere to the speed-accuracy tradeoff, such that increasing the space for deliberation enables extended data collection. The setting of space for deliberation reflects a form of meta-decision-making that can, and should be, studied empirically as a value-based exercise that weighs the prior propensities, the economics of information seeking, and the potential outcomes. Importantly, the control of the space for deliberation raises a question of agency. Decision-makers may actively and explicitly set their own decision parameters, but these parameters may also be set by environmental pressures. Thus, decision-makers may be influenced-or nudged in a particular direction-by how decision problems are framed, with a sense of urgency or a binary definition of choice options. We argue that a proper understanding of these mechanisms has important practical implications toward the optimal usage of space for deliberation.
Collapse
|
30
|
Abstract
A variety of behavioral and neural phenomena suggest that organisms evaluate outcomes not on an absolute utility scale, but relative to some dynamic and context-sensitive reference or scale. Sometimes, as in foraging tasks, this results in sensible choices; in other situations, like choosing between options learned in different contexts, irrational choices can result. We argue that what unites and demystifies these various phenomena is that the brain's goal is not assessing utility as an end in itself, but rather comparing different options to choose the better one. In the presence of uncertainty, noise, or costly computation, adjusting options to the context can produce more accurate choices.
Collapse
|
31
|
|
32
|
Schultz W, Stauffer WR, Lak A, Pastor-Bernier A. Smarter than humans: rationality reflected in primate neuronal reward signals. Curr Opin Behav Sci 2021. [DOI: 10.1016/j.cobeha.2021.03.021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
33
|
Abstract
The decisions we make are shaped by a lifetime of learning. Past experience guides the way that we encode information in neural systems for perception and valuation, and determines the information we retrieve when making decisions. Distinct literatures have discussed how lifelong learning and local context shape decisions made about sensory signals, propositional information, or economic prospects. Here, we build bridges between these literatures, arguing for common principles of adaptive rationality in perception, cognition, and economic choice. We discuss how a single common framework, based on normative principles of efficient coding and Bayesian inference, can help us understand a myriad of human decision biases, including sensory illusions, adaptive aftereffects, choice history biases, central tendency effects, anchoring effects, contrast effects, framing effects, congruency effects, reference-dependent valuation, nonlinear utility functions, and discretization heuristics. We describe a simple computational framework for explaining these phenomena. Expected final online publication date for the Annual Review of Psychology, Volume 73 is January 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
Collapse
Affiliation(s)
- Christopher Summerfield
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom;
| | - Paula Parpart
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom;
| |
Collapse
|
34
|
Kryven M, Ullman TD, Cowan W, Tenenbaum JB. Plans or Outcomes: How Do We Attribute Intelligence to Others? Cogn Sci 2021; 45:e13041. [PMID: 34490914 DOI: 10.1111/cogs.13041] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2019] [Revised: 08/06/2021] [Accepted: 08/12/2021] [Indexed: 11/30/2022]
Abstract
Humans routinely make inferences about both the contents and the workings of other minds based on observed actions. People consider what others want or know, but also how intelligent, rational, or attentive they might be. Here, we introduce a new methodology for quantitatively studying the mechanisms people use to attribute intelligence to others based on their behavior. We focus on two key judgments previously proposed in the literature: judgments based on observed outcomes (you're smart if you won the game) and judgments based on evaluating the quality of an agent's planning that led to their outcomes (you're smart if you made the right choice, even if you didn't succeed). We present a novel task, the maze search task (MST), in which participants rate the intelligence of agents searching a maze for a hidden goal. We model outcome-based attributions based on the observed utility of the agent upon achieving a goal, with higher utilities indicating higher intelligence, and model planning-based attributions by measuring the proximity of the observed actions to an ideal planner, such that agents who produce closer approximations of optimal plans are seen as more intelligent. We examine human attributions of intelligence in three experiments that use MST and find that participants used both outcome and planning as indicators of intelligence. However, observing the outcome was not necessary, and participants still made planning-based attributions of intelligence when the outcome was not observed. We also found that the weights individuals placed on plans and on outcome correlated with an individual's ability to engage in cognitive reflection. Our results suggest that people attribute intelligence based on plans given sufficient context and cognitive resources and rely on the outcome when computational resources or context are limited.
Collapse
Affiliation(s)
- Marta Kryven
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology
| | | | - William Cowan
- Department of Computer Science, University of Waterloo
| | - Joshua B Tenenbaum
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology
| |
Collapse
|
35
|
Abstract
Choking under pressure is a frustrating phenomenon experienced sometimes by skilled performers as well as during everyday life. The phenomenon has been extensively studied in humans, but it has not been previously shown whether animals also choke under pressure. Here we report that rhesus monkeys also choke under pressure. This indicates that there may be shared neural mechanisms that underlie the behavior in both humans and monkeys. Introducing an animal model for choking under pressure allows for opportunities to study the neural causes of this paradoxical behavior. In high-stakes situations, people sometimes exhibit a frustrating phenomenon known as “choking under pressure.” Usually, we perform better when the potential payoff is larger. However, once potential rewards get too high, performance paradoxically decreases—we “choke.” Why do we choke under pressure? An animal model of choking would facilitate the investigation of its neural basis. However, it could be that choking is a uniquely human occurrence. To determine whether animals also choke, we trained three rhesus monkeys to perform a difficult reaching task in which they knew in advance the amount of reward to be given upon successful completion. Like humans, monkeys performed worse when potential rewards were exceptionally valuable. Failures that occurred at the highest level of reward were due to overly cautious reaching, in line with the psychological theory that explicit monitoring of behavior leads to choking. Our results demonstrate that choking under pressure is not unique to humans, and thus, its neural basis might be conserved across species.
Collapse
|
36
|
Piray P, Daw ND. Linear reinforcement learning in planning, grid fields, and cognitive control. Nat Commun 2021; 12:4942. [PMID: 34400622 PMCID: PMC8368103 DOI: 10.1038/s41467-021-25123-3] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2020] [Accepted: 07/19/2021] [Indexed: 12/02/2022] Open
Abstract
It is thought that the brain’s judicious reuse of previous computation underlies our ability to plan flexibly, but also that inappropriate reuse gives rise to inflexibilities like habits and compulsion. Yet we lack a complete, realistic account of either. Building on control engineering, here we introduce a model for decision making in the brain that reuses a temporally abstracted map of future events to enable biologically-realistic, flexible choice at the expense of specific, quantifiable biases. It replaces the classic nonlinear, model-based optimization with a linear approximation that softly maximizes around (and is weakly biased toward) a default policy. This solution demonstrates connections between seemingly disparate phenomena across behavioral neuroscience, notably flexible replanning with biases and cognitive control. It also provides insight into how the brain can represent maps of long-distance contingencies stably and componentially, as in entorhinal response fields, and exploit them to guide choice even under changing goals. Models of decision making have so far been unable to account for how humans’ choices can be flexible yet efficient. Here the authors present a linear reinforcement learning model which explains both flexibility, and rare limitations such as habits, as arising from efficient approximate computation
Collapse
Affiliation(s)
- Payam Piray
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
| | - Nathaniel D Daw
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| |
Collapse
|
37
|
Knudsen EB, Wallis JD. Hippocampal neurons construct a map of an abstract value space. Cell 2021; 184:4640-4650.e10. [PMID: 34348112 DOI: 10.1016/j.cell.2021.07.010] [Citation(s) in RCA: 37] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2020] [Revised: 05/11/2021] [Accepted: 07/08/2021] [Indexed: 02/06/2023]
Abstract
The hippocampus is thought to encode a "cognitive map," a structural organization of knowledge about relationships in the world. Place cells, spatially selective hippocampal neurons that have been extensively studied in rodents, are one component of this map, describing the relative position of environmental features. However, whether this map extends to abstract, cognitive information remains unknown. Using the relative reward value of cues to define continuous "paths" through an abstract value space, we show that single neurons in primate hippocampus encode this space through value place fields, much like a rodent's place neurons encode paths through physical space. Value place fields remapped when cues changed but also became increasingly correlated across contexts, allowing maps to become generalized. Our findings help explain the critical contribution of the hippocampus to value-based decision-making, providing a mechanism by which knowledge of relationships in the world can be incorporated into reward predictions for guiding decisions.
Collapse
Affiliation(s)
- Eric B Knudsen
- Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, CA, USA.
| | - Joni D Wallis
- Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, CA, USA; Department of Psychology, University of California at Berkeley, Berkeley, CA, USA
| |
Collapse
|
38
|
Wang L, Luo X, Yuan TF, Zhou X. Reward facilitates response conflict resolution via global motor inhibition: Electromyography evidence. Psychophysiology 2021; 58:e13896. [PMID: 34231226 DOI: 10.1111/psyp.13896] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2020] [Revised: 04/29/2021] [Accepted: 06/18/2021] [Indexed: 11/26/2022]
Abstract
It is crucial for humans to coordinate between behavioural tendencies that can lead to reward but are in conflict with each other. This response conflict can be measured in a reward-modulated Simon task, in which a discriminative response to the identity of a lateral target is required and the target is associated with either high- or low-reward. Critically, the lateral target is presented either congruent or incongruent with the location of the responding hand. It has been shown that relative to the low-reward target, the high-reward target induced a larger response conflict when the target was incongruent with the position of the task-required, reward-obtaining hand. Here we investigated how this response conflict is resolved by acquiring 24 healthy participants' electromyography (EMG) signals from both the task-required responding hand (i.e., goal-directed effector) and the alternative hand (i.e., inappropriate effector). During the coping with the response conflict, motor inhibition (indexed by reduction in EMG signals between conditions) was observed not only at the inappropriate effector but also at the goal-directed effector. Individuals who showed stronger inhibition on the inappropriate effector suffered less from the inhibition on the goal-directed effector, and had more efficient implementation of the reward-obtaining response. Our findings suggest a global motor inhibition that may function to increase the signal-noise ratio in the motor system so as to implement reward-guided behavior.
Collapse
Affiliation(s)
- Lihui Wang
- Institute of Psychology and Behavioral Science, Shanghai Jiao Tong University, Shanghai, China.,Shanghai Key Laboratory of Psychotic Disorders, Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China.,Department of Experimental Psychology, Otto-von-Guericke University, Magdeburg, Germany
| | - Xiaoxiao Luo
- School of Psychology and Cognitive Science, East China Normal University, Shanghai, China.,School of Psychological and Cognitive Sciences, Peking University, Beijing, China
| | - Ti-Fei Yuan
- Shanghai Key Laboratory of Psychotic Disorders, Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Xiaolin Zhou
- School of Psychology and Cognitive Science, East China Normal University, Shanghai, China.,School of Psychological and Cognitive Sciences, Peking University, Beijing, China.,Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China.,PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China
| |
Collapse
|
39
|
Booras A, Stevenson T, McCormack CN, Rhoads ME, Hanks TD. Change point detection with multiple alternatives reveals parallel evaluation of the same stream of evidence along distinct timescales. Sci Rep 2021; 11:13098. [PMID: 34162943 PMCID: PMC8222317 DOI: 10.1038/s41598-021-92470-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2020] [Accepted: 06/08/2021] [Indexed: 11/09/2022] Open
Abstract
In order to behave appropriately in a rapidly changing world, individuals must be able to detect when changes occur in that environment. However, at any given moment, there are a multitude of potential changes of behavioral significance that could occur. Here we investigate how knowledge about the space of possible changes affects human change point detection. We used a stochastic auditory change point detection task that allowed model-free and model-based characterization of the decision process people employ. We found that subjects can simultaneously apply distinct timescales of evidence evaluation to the same stream of evidence when there are multiple types of changes possible. Informative cues that specified the nature of the change led to improved accuracy for change point detection through mechanisms involving both the timescales of evidence evaluation and adjustments of decision bounds. These results establish three important capacities of information processing for decision making that any proposed neural mechanism of evidence evaluation must be able to support: the ability to simultaneously employ multiple timescales of evidence evaluation, the ability to rapidly adjust those timescales, and the ability to modify the amount of information required to make a decision in the context of flexible timescales.
Collapse
Affiliation(s)
- Alexa Booras
- grid.27860.3b0000 0004 1936 9684Center for Neuroscience, University of California Davis, Davis, CA USA
| | - Tanner Stevenson
- grid.27860.3b0000 0004 1936 9684Center for Neuroscience, University of California Davis, Davis, CA USA
| | - Connor N. McCormack
- grid.27860.3b0000 0004 1936 9684Center for Neuroscience, University of California Davis, Davis, CA USA
| | - Marie E. Rhoads
- grid.27860.3b0000 0004 1936 9684Center for Neuroscience, University of California Davis, Davis, CA USA ,grid.19006.3e0000 0000 9632 6718Department of Neuroscience, University of California Los Angeles, Los Angeles, CA USA
| | - Timothy D. Hanks
- grid.27860.3b0000 0004 1936 9684Center for Neuroscience, University of California Davis, Davis, CA USA ,grid.27860.3b0000 0004 1936 9684Department of Neurology, University of California Davis, Sacramento, CA USA
| |
Collapse
|
40
|
Zhang B, Kan JYY, Yang M, Wang X, Tu J, Dorris MC. Transforming absolute value to categorical choice in primate superior colliculus during value-based decision making. Nat Commun 2021; 12:3410. [PMID: 34099726 PMCID: PMC8184840 DOI: 10.1038/s41467-021-23747-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Accepted: 04/23/2021] [Indexed: 11/09/2022] Open
Abstract
Value-based decision making involves choosing from multiple options with different values. Despite extensive studies on value representation in various brain regions, the neural mechanism for how multiple value options are converted to motor actions remains unclear. To study this, we developed a multi-value foraging task with varying menu of items in non-human primates using eye movements that dissociates value and choice, and conducted electrophysiological recording in the midbrain superior colliculus (SC). SC neurons encoded "absolute" value, independent of available options, during late fixation. In addition, SC neurons also represent value threshold, modulated by available options, different from conventional motor threshold. Electrical stimulation of SC neurons biased choices in a manner predicted by the difference between the value representation and the value threshold. These results reveal a neural mechanism directly transforming absolute values to categorical choices within SC, supporting highly efficient value-based decision making critical for real-world economic behaviors.
Collapse
Affiliation(s)
- Beizhen Zhang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China. .,University of Chinese Academy of Sciences, Beijing, China.
| | - Janis Ying Ying Kan
- Department of Biomedical and Molecular Sciences, Centre for Neuroscience Studies, Queen's University, Kingston, ON, Canada
| | - Mingpo Yang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China
| | - Xiaochun Wang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China
| | - Jiahao Tu
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China
| | - Michael Christopher Dorris
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China.
| |
Collapse
|
41
|
Azab H, Hayden BY. Partial integration of the components of value in anterior cingulate cortex. Behav Neurosci 2021; 134:296-308. [PMID: 32658523 DOI: 10.1037/bne0000382] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
Evaluation often involves integrating multiple determinants of value, such as the different possible outcomes in risky choice. A brain region can be placed either before or after a presumed evaluation stage by measuring how responses of its neurons depend on multiple determinants of value. A brain region could also, in principle, show partial integration, which would indicate that it occupies a middle position between (preevaluative) nonintegration and (postevaluative) full integration. Existing mathematical techniques cannot distinguish full from partial integration and therefore risk misidentifying regional function. Here we use a new Bayesian regression-based approach to analyze responses of neurons in dorsal anterior cingulate cortex (dACC) to risky offers. We find that dACC neurons only partially integrate across outcome dimensions, indicating that dACC cannot be assigned to either a pre- or postevaluative position. Neurons in dACC also show putative signatures of value comparison, thereby demonstrating that comparison does not require complete evaluation before proceeding. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Collapse
Affiliation(s)
- Habiba Azab
- Department of Neuroscience, Center for Magnetic Resonance Research (CMRR), University of Minnesota, Twin Cities
| | - Benjamin Y Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research (CMRR), University of Minnesota, Twin Cities
| |
Collapse
|
42
|
Moore B, Khang S, Francis JT. Noise-Correlation Is Modulated by Reward Expectation in the Primary Motor Cortex Bilaterally During Manual and Observational Tasks in Primates. Front Behav Neurosci 2020; 14:541920. [PMID: 33343308 PMCID: PMC7739882 DOI: 10.3389/fnbeh.2020.541920] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2020] [Accepted: 09/30/2020] [Indexed: 11/17/2022] Open
Abstract
Reward modulation is represented in the motor cortex (M1) and could be used to implement more accurate decoding models to improve brain-computer interfaces (BCIs; Zhao et al., 2018). Analyzing trial-to-trial noise-correlations between neural units in the presence of rewarding (R) and non-rewarding (NR) stimuli adds to our understanding of cortical network dynamics. We utilized Pearson's correlation coefficient to measure shared variability between simultaneously recorded units (32-112) and found significantly higher noise-correlation and positive correlation between the populations' signal- and noise-correlation during NR trials as compared to R trials. This pattern is evident in data from two non-human primates (NHPs) during single-target center out reaching tasks, both manual and action observation versions. We conducted a mean matched noise-correlation analysis to decouple known interactions between event-triggered firing rate changes and neural correlations. Isolated reward discriminatory units demonstrated stronger correlational changes than units unresponsive to reward firing rate modulation, however, the qualitative response was similar, indicating correlational changes within the network as a whole can serve as another information channel to be exploited by BCIs that track the underlying cortical state, such as reward expectation, or attentional modulation. Reward expectation and attention in return can be utilized with reinforcement learning (RL) towards autonomous BCI updating.
Collapse
Affiliation(s)
- Brittany Moore
- Department of Biomedical Engineering, Cullen College of Engineering, The University of Houston, Houston, TX, United States
| | - Sheng Khang
- Department of Biomedical Engineering, Cullen College of Engineering, The University of Houston, Houston, TX, United States
| | - Joseph Thachil Francis
- Department of Biomedical Engineering, Cullen College of Engineering, The University of Houston, Houston, TX, United States
- Department of Electrical and Computer Engineering, Cullen College of Engineering, The University of Houston, Houston, TX, United States
| |
Collapse
|
43
|
Divisive normalization does influence decisions with multiple alternatives. Nat Hum Behav 2020; 4:1118-1120. [PMID: 32929203 DOI: 10.1038/s41562-020-00941-5] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2020] [Accepted: 08/04/2020] [Indexed: 11/08/2022]
|
44
|
Klein SD, Olman CA, Sponheim SR. Perceptual Mechanisms of Visual Hallucinations and Illusions in Psychosis. JOURNAL OF PSYCHIATRY AND BRAIN SCIENCE 2020; 5:e200020. [PMID: 32944656 PMCID: PMC7494209 DOI: 10.20900/jpbs.20200020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
Abstract
Psychosis has been associated with neural anomalies across a number of brain regions and cortical networks. Nevertheless, the exact pathophysiology of the disorder remains unclear. Aberrant visual perceptions such as hallucinations are evident in psychosis, while the occurrence of visual distortions is elevated in individuals with genetic liability for psychosis. The overall goals of this project are to: (1) use psychophysical tasks and neuroimaging to characterize deficits in visual perception; (2) acquire a mechanistic understanding of these deficits through development and validation of a computational model; and (3) determine if said mechanisms mark genetic liability for psychosis. Visual tasks tapping both low- and high-level visual processing are being completed as individuals with psychotic disorders (IPD), first-degree biological siblings of IPDs (SibIPDs) and healthy controls (HCs) undergo 248-channel magneto-encephalography (MEG) recordings followed by 7 Tesla functional magnetic resonance imaging (MRI). By deriving cortical source signals from MEG and MRI data, we will characterize the timing, location and coordination of neural processes. We hypothesize that IPDs prone to visual hallucinations will exhibit deviant functions within early visual cortex, and that aberrant contextual influences on visual perception will involve higher-level visual cortical regions and be associated with visual hallucinations. SibIPDs who experience visual distortions-but not hallucinations-are hypothesized to exhibit deficits in higher-order visual processing reflected in abnormal inter-regional neural synchronization. We hope the results lead to the development of targeted interventions for psychotic disorders, as well as identify useful biomarkers for aberrant neural functions that give rise to psychosis.
Collapse
Affiliation(s)
- Samuel D. Klein
- Clinical Science and Psychopathology Research Program, University of Minnesota-Twin Cities, 75 East River Road, Minneapolis, MN 55455, USA
| | - Cheryl A. Olman
- Department of Psychology, University of Minnesota-Twin Cities, 75 East River Road, Minneapolis, MN 55455, USA
- Center for Magnetic Resonance Research, University of Minnesota-Twin Cities, 2021 6th St SE, Minneapolis, MN 55455, USA
| | - Scott R. Sponheim
- Minneapolis Veterans Affairs Health Care System, 1 Veterans Dr, Minneapolis, MN 55417, USA
- Department of Psychiatry and Behavioral Science, University of Minnesota, 606 24th Ave S, Minneapolis, MN 55454, USA
| |
Collapse
|
45
|
Chau BKH, Law CK, Lopez-Persem A, Klein-Flügge MC, Rushworth MFS. Consistent patterns of distractor effects during decision making. eLife 2020; 9:e53850. [PMID: 32628109 PMCID: PMC7371422 DOI: 10.7554/elife.53850] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2019] [Accepted: 07/06/2020] [Indexed: 01/24/2023] Open
Abstract
The value of a third potential option or distractor can alter the way in which decisions are made between two other options. Two hypotheses have received empirical support: that a high value distractor improves the accuracy with which decisions between two other options are made and that it impairs accuracy. Recently, however, it has been argued that neither observation is replicable. Inspired by neuroimaging data showing that high value distractors have different impacts on prefrontal and parietal regions, we designed a dual route decision-making model that mimics the neural signals of these regions. Here we show in the dual route model and empirical data that both enhancement and impairment effects are robust phenomena but predominate in different parts of the decision space defined by the options' and the distractor's values. However, beyond these constraints, both effects co-exist under similar conditions. Moreover, both effects are robust and observable in six experiments.
Collapse
Affiliation(s)
- Bolton KH Chau
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic UniversityHong KongHong Kong
- University Research Facility in Behavioral and Systems Neuroscience, The Hong Kong Polytechnic UniversityHong KongHong Kong
| | - Chun-Kit Law
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic UniversityHong KongHong Kong
| | - Alizée Lopez-Persem
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
- FrontLab, Paris Brain Institute (ICM), Inserm U 1127, CNRS UMR 7225, Sorbonne UniversitéParisFrance
| | - Miriam C Klein-Flügge
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
| | - Matthew FS Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
| |
Collapse
|
46
|
Gluth S, Kern N, Kortmann M, Vitali CL. Value-based attention but not divisive normalization influences decisions with multiple alternatives. Nat Hum Behav 2020; 4:634-645. [PMID: 32015490 PMCID: PMC7306407 DOI: 10.1038/s41562-020-0822-0] [Citation(s) in RCA: 58] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2019] [Accepted: 01/07/2020] [Indexed: 01/13/2023]
Abstract
Violations of economic rationality principles in choices between three or more options are critical for understanding the neural and cognitive mechanisms of decision-making. A recent study reported that the relative choice accuracy between two options decreases as the value of a third (distractor) option increases and attributed this effect to divisive normalization of neural value representations. In two preregistered experiments, a direct replication and an eye-tracking experiment, we assessed the replicability of this effect and tested an alternative account that assumes value-based attention to mediate the distractor effect. Surprisingly, we could not replicate the distractor effect in our experiments. However, we found a dynamic influence of distractor value on fixations to distractors as predicted by the value-based attention theory. Computationally, we show that extending an established sequential sampling decision-making model by a value-based attention mechanism offers a comprehensive account of the interplay between value, attention, response times and decisions.
Collapse
Affiliation(s)
- Sebastian Gluth
- Department of Psychology, University of Basel, Basel, Switzerland.
| | - Nadja Kern
- Department of Psychology, University of Basel, Basel, Switzerland
| | - Maria Kortmann
- Department of Psychology, University of Basel, Basel, Switzerland
| | - Cécile L Vitali
- Department of Psychology, University of Basel, Basel, Switzerland
| |
Collapse
|
47
|
Mysore SP, Kothari NB. Mechanisms of competitive selection: A canonical neural circuit framework. eLife 2020; 9:e51473. [PMID: 32431293 PMCID: PMC7239658 DOI: 10.7554/elife.51473] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2019] [Accepted: 04/02/2020] [Indexed: 01/25/2023] Open
Abstract
Competitive selection, the transformation of multiple competing sensory inputs and internal states into a unitary choice, is a fundamental component of animal behavior. Selection behaviors have been studied under several intersecting umbrellas including decision-making, action selection, perceptual categorization, and attentional selection. Neural correlates of these behaviors and computational models have been investigated extensively. However, specific, identifiable neural circuit mechanisms underlying the implementation of selection remain elusive. Here, we employ a first principles approach to map competitive selection explicitly onto neural circuit elements. We decompose selection into six computational primitives, identify demands that their execution places on neural circuit design, and propose a canonical neural circuit framework. The resulting framework has several links to neural literature, indicating its biological feasibility, and has several common elements with prominent computational models, suggesting its generality. We propose that this framework can help catalyze experimental discovery of the neural circuit underpinnings of competitive selection.
Collapse
Affiliation(s)
- Shreesh P Mysore
- Department of Psychological and Brain Sciences, Johns Hopkins UniversityBaltimoreUnited States
- The Solomon H. Snyder Department of Neuroscience, Johns Hopkins UniversityBaltimoreUnited States
| | - Ninad B Kothari
- Department of Psychological and Brain Sciences, Johns Hopkins UniversityBaltimoreUnited States
| |
Collapse
|
48
|
Keung W, Hagen TA, Wilson RC. A divisive model of evidence accumulation explains uneven weighting of evidence over time. Nat Commun 2020; 11:2160. [PMID: 32358501 PMCID: PMC7195479 DOI: 10.1038/s41467-020-15630-0] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2019] [Accepted: 03/12/2020] [Indexed: 12/21/2022] Open
Abstract
Divisive normalization has long been used to account for computations in various neural processes and behaviours. The model proposes that inputs into a neural system are divisively normalized by the system’s total activity. More recently, dynamical versions of divisive normalization have been shown to account for how neural activity evolves over time in value-based decision making. Despite its ubiquity, divisive normalization has not been studied in decisions that require evidence to be integrated over time. Such decisions are important when the information is not all available at once. A key feature of such decisions is how evidence is weighted over time, known as the integration kernel. Here, we provide a formal expression for the integration kernel in divisive normalization, and show that divisive normalization quantitatively accounts for 133 human participants’ perceptual decision making behaviour, performing as well as the state-of-the-art Drift Diffusion Model, the predominant model for perceptual evidence accumulation. Divisive normalization is thought to be a ubiquitous computation in the brain, but has not been studied in decisions that require integrating evidence over time. Here, the authors show in humans that dynamic divisive normalization accounts for the uneven weighting of perceptual evidence over time.
Collapse
Affiliation(s)
- Waitsang Keung
- Department of Psychology, University of Arizona, Tucson, AZ, 85719, USA.
| | - Todd A Hagen
- Department of Psychology, University of Arizona, Tucson, AZ, 85719, USA
| | - Robert C Wilson
- Department of Psychology, University of Arizona, Tucson, AZ, 85719, USA.,Cognitive Science Program, University of Arizona, Tucson, AZ, 85719, USA
| |
Collapse
|
49
|
Confidence reports in decision-making with multiple alternatives violate the Bayesian confidence hypothesis. Nat Commun 2020; 11:2004. [PMID: 32332712 PMCID: PMC7181620 DOI: 10.1038/s41467-020-15581-6] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2019] [Accepted: 03/12/2020] [Indexed: 11/16/2022] Open
Abstract
Decision confidence reflects our ability to evaluate the quality of decisions and guides subsequent behavior. Experiments on confidence reports have almost exclusively focused on two-alternative decision-making. In this realm, the leading theory is that confidence reflects the probability that a decision is correct (the posterior probability of the chosen option). There is, however, another possibility, namely that people are less confident if the best two options are closer to each other in posterior probability, regardless of how probable they are in absolute terms. This possibility has not previously been considered because in two-alternative decisions, it reduces to the leading theory. Here, we test this alternative theory in a three-alternative visual categorization task. We found that confidence reports are best explained by the difference between the posterior probabilities of the best and the next-best options, rather than by the posterior probability of the chosen (best) option alone, or by the overall uncertainty (entropy) of the posterior distribution. Our results upend the leading notion of decision confidence and instead suggest that confidence reflects the observer’s subjective probability that they made the best possible decision. Conventional theory suggests that people’s confidence about a decision reflects their subjective probability that the decision was correct. By studying decisions with multiple alternatives, the authors show that confidence reports instead reflect the difference in probabilities between the chosen and the next-best alternative.
Collapse
|
50
|
Lin WH, Gardner JL, Wu SW. Context effects on probability estimation. PLoS Biol 2020; 18:e3000634. [PMID: 32134917 PMCID: PMC7077880 DOI: 10.1371/journal.pbio.3000634] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2019] [Revised: 03/17/2020] [Accepted: 02/14/2020] [Indexed: 11/18/2022] Open
Abstract
Many decisions rely on how we evaluate potential outcomes and estimate their corresponding probabilities of occurrence. Outcome evaluation is subjective because it requires consulting internal preferences and is sensitive to context. In contrast, probability estimation requires extracting statistics from the environment and therefore imposes unique challenges to the decision maker. Here, we show that probability estimation, like outcome evaluation, is subject to context effects that bias probability estimates away from other events present in the same context. However, unlike valuation, these context effects appeared to be scaled by estimated uncertainty, which is largest at intermediate probabilities. Blood-oxygen-level-dependent (BOLD) imaging showed that patterns of multivoxel activity in the dorsal anterior cingulate cortex (dACC), ventromedial prefrontal cortex (VMPFC), and intraparietal sulcus (IPS) predicted individual differences in context effects on probability estimates. These results establish VMPFC as the neurocomputational substrate shared between valuation and probability estimation and highlight the additional involvement of dACC and IPS that can be uniquely attributed to probability estimation. Because probability estimation is a required component of computational accounts from sensory inference to higher cognition, the context effects found here may affect a wide array of cognitive computations. This study shows how probability estimation can be affected by the context of our recent experience, namely, how the presence of multiple events experienced closed in time can influence their respective probability estimates.
Collapse
Affiliation(s)
- Wei-Hsiang Lin
- Brain Mind Institute, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Justin L. Gardner
- Department of Psychology, Stanford University, Stanford, California, United States of America
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, California, United States of America
| | - Shih-Wei Wu
- Institute of Neuroscience, National Yang-Ming University, Taipei, Taiwan
- Brain Research Center, National Yang-Ming University, Taipei, Taiwan
- * E-mail:
| |
Collapse
|