51
|
Kuperberg GR. Tea With Milk? A Hierarchical Generative Framework of Sequential Event Comprehension. Top Cogn Sci 2021; 13:256-298. [PMID: 33025701 PMCID: PMC7897219 DOI: 10.1111/tops.12518] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2019] [Revised: 07/11/2020] [Accepted: 07/11/2020] [Indexed: 10/23/2022]
Abstract
To make sense of the world around us, we must be able to segment a continual stream of sensory inputs into discrete events. In this review, I propose that in order to comprehend events, we engage hierarchical generative models that "reverse engineer" the intentions of other agents as they produce sequential action in real time. By generating probabilistic predictions for upcoming events, generative models ensure that we are able to keep up with the rapid pace at which perceptual inputs unfold. By tracking our certainty about other agents' goals and the magnitude of prediction errors at multiple temporal scales, generative models enable us to detect event boundaries by inferring when a goal has changed. Moreover, by adapting flexibly to the broader dynamics of the environment and our own comprehension goals, generative models allow us to optimally allocate limited resources. Finally, I argue that we use generative models not only to comprehend events but also to produce events (carry out goal-relevant sequential action) and to continually learn about new events from our surroundings. Taken together, this hierarchical generative framework provides new insights into how the human brain processes events so effortlessly while highlighting the fundamental links between event comprehension, production, and learning.
Collapse
Affiliation(s)
- Gina R. Kuperberg
- Department of Psychology and Center for Cognitive Science, Tufts University
- Department of Psychiatry and the Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School
| |
Collapse
|
52
|
Signed Reward Prediction Errors in the Ventral Striatum Drive Episodic Memory. J Neurosci 2020; 41:1716-1726. [PMID: 33334870 DOI: 10.1523/jneurosci.1785-20.2020] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2020] [Revised: 12/04/2020] [Accepted: 12/07/2020] [Indexed: 11/21/2022] Open
Abstract
Recent behavioral evidence implicates reward prediction errors (RPEs) as a key factor in the acquisition of episodic memory. Yet, important neural predictions related to the role of RPEs in episodic memory acquisition remain to be tested. Humans (both sexes) performed a novel variable-choice task where we experimentally manipulated RPEs and found support for key neural predictions with fMRI. Our results show that in line with previous behavioral observations, episodic memory accuracy increases with the magnitude of signed (i.e., better/worse-than-expected) RPEs (SRPEs). Neurally, we observe that SRPEs are encoded in the ventral striatum (VS). Crucially, we demonstrate through mediation analysis that activation in the VS mediates the experimental manipulation of SRPEs on episodic memory accuracy. In particular, SRPE-based responses in the VS (during learning) predict the strength of subsequent episodic memory (during recollection). Furthermore, functional connectivity between task-relevant processing areas (i.e., face-selective areas) and hippocampus and ventral striatum increased as a function of RPE value (during learning), suggesting a central role of these areas in episodic memory formation. Our results consolidate reinforcement learning theory and striatal RPEs as key factors subtending the formation of episodic memory.SIGNIFICANCE STATEMENT Recent behavioral research has shown that reward prediction errors (RPEs), a key concept of reinforcement learning theory, are crucial to the formation of episodic memories. In this study, we reveal the neural underpinnings of this process. Using fMRI, we show that signed RPEs (SRPEs) are encoded in the ventral striatum (VS), and crucially, that SRPE VS activity is responsible for the subsequent recollection accuracy of one-shot learned episodic memory associations.
Collapse
|
53
|
Zhang S, Yoshida W, Mano H, Yanagisawa T, Mancini F, Shibata K, Kawato M, Seymour B. Pain Control by Co-adaptive Learning in a Brain-Machine Interface. Curr Biol 2020; 30:3935-3944.e7. [PMID: 32795441 PMCID: PMC7575198 DOI: 10.1016/j.cub.2020.07.066] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2020] [Revised: 06/22/2020] [Accepted: 07/21/2020] [Indexed: 11/21/2022]
Abstract
Innovation in the field of brain-machine interfacing offers a new approach to managing human pain. In principle, it should be possible to use brain activity to directly control a therapeutic intervention in an interactive, closed-loop manner. But this raises the question as to whether the brain activity changes as a function of this interaction. Here, we used real-time decoded functional MRI responses from the insula cortex as input into a closed-loop control system aimed at reducing pain and looked for co-adaptive neural and behavioral changes. As subjects engaged in active cognitive strategies orientated toward the control system, such as trying to enhance their brain activity, pain encoding in the insula was paradoxically degraded. From a mechanistic perspective, we found that cognitive engagement was accompanied by activation of the endogenous pain modulation system, manifested by the attentional modulation of pain ratings and enhanced pain responses in pregenual anterior cingulate cortex and periaqueductal gray. Further behavioral evidence of endogenous modulation was confirmed in a second experiment using an EEG-based closed-loop system. Overall, the results show that implementing brain-machine control systems for pain induces a parallel set of co-adaptive changes in the brain, and this can interfere with the brain signals and behavior under control. More generally, this illustrates a fundamental challenge of brain decoding applications-that the brain inherently adapts to being decoded, especially as a result of cognitive processes related to learning and cooperation. Understanding the nature of these co-adaptive processes informs strategies to mitigate or exploit them.
Collapse
Affiliation(s)
- Suyi Zhang
- Computational and Biological Learning Laboratory, Department of Engineering, University of Cambridge, Cambridge, CB2 1PZ, UK; Brain Information Communication Research Laboratory Group, Advanced Telecommunications Research Institute International, Kyoto 619-0237, Japan; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford OX3 9DU, UK.
| | - Wako Yoshida
- Brain Information Communication Research Laboratory Group, Advanced Telecommunications Research Institute International, Kyoto 619-0237, Japan
| | - Hiroaki Mano
- Center for Information and Neural Networks, National Institute for Information and Communications Technology, Osaka 565-0871, Japan
| | - Takufumi Yanagisawa
- Endowed Research Department of Clinical Neuroengineering, Global Center for Medical Engineering and Informatics, Osaka University, Osaka 565-0043, Japan
| | - Flavia Mancini
- Computational and Biological Learning Laboratory, Department of Engineering, University of Cambridge, Cambridge, CB2 1PZ, UK
| | - Kazuhisa Shibata
- Lab for Human Cognition and Learning, Center for Brain Science, RIKEN, Wako, Saitama 351-0198, Japan
| | - Mitsuo Kawato
- Brain Information Communication Research Laboratory Group, Advanced Telecommunications Research Institute International, Kyoto 619-0237, Japan.
| | - Ben Seymour
- Computational and Biological Learning Laboratory, Department of Engineering, University of Cambridge, Cambridge, CB2 1PZ, UK; Brain Information Communication Research Laboratory Group, Advanced Telecommunications Research Institute International, Kyoto 619-0237, Japan; Center for Information and Neural Networks, National Institute for Information and Communications Technology, Osaka 565-0871, Japan; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford OX3 9DU, UK.
| |
Collapse
|
54
|
Gottlieb J, Cohanpour M, Li Y, Singletary N, Zabeh E. Curiosity, information demand and attentional priority. Curr Opin Behav Sci 2020. [DOI: 10.1016/j.cobeha.2020.07.016] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
|
55
|
Abstract
Individual differences in learning can influence how animals respond to and communicate about their environment, which may nonlinearly shape how a social group accomplishes a collective task. There are few empirical examples of how differences in collective dynamics emerge from variation among individuals in cognition. Here, we use a naturally variable and heritable learning behavior called latent inhibition (LI) to show that interactions among individuals that differ in this cognitive ability drive collective foraging behavior in honey bee colonies. We artificially selected two distinct phenotypes: high-LI bees that ignore previously familiar stimuli in favor of novel ones and low-LI bees that learn familiar and novel stimuli equally well. We then provided colonies differentially composed of different ratios of these phenotypes with a choice between familiar and novel feeders. Colonies of predominantly high-LI individuals preferred to visit familiar food locations, while low-LI colonies visited novel and familiar food locations equally. Interestingly, in colonies of mixed learning phenotypes, the low-LI individuals showed a preference to visiting familiar feeders, which contrasts with their behavior when in a uniform low-LI group. We show that the shift in feeder preference of low-LI bees is driven by foragers of the high-LI phenotype dancing more intensely and attracting more followers. Our results reveal that cognitive abilities of individuals and their social interactions, which we argue relate to differences in attention, drive emergent collective outcomes.
Collapse
|
56
|
A simple model for learning in volatile environments. PLoS Comput Biol 2020; 16:e1007963. [PMID: 32609755 PMCID: PMC7329063 DOI: 10.1371/journal.pcbi.1007963] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2019] [Accepted: 05/18/2020] [Indexed: 11/19/2022] Open
Abstract
Sound principles of statistical inference dictate that uncertainty shapes learning. In this work, we revisit the question of learning in volatile environments, in which both the first and second-order statistics of observations dynamically evolve over time. We propose a new model, the volatile Kalman filter (VKF), which is based on a tractable state-space model of uncertainty and extends the Kalman filter algorithm to volatile environments. The proposed model is algorithmically simple and encompasses the Kalman filter as a special case. Specifically, in addition to the error-correcting rule of Kalman filter for learning observations, the VKF learns volatility according to a second error-correcting rule. These dual updates echo and contextualize classical psychological models of learning, in particular hybrid accounts of Pearce-Hall and Rescorla-Wagner. At the computational level, compared with existing models, the VKF gives up some flexibility in the generative model to enable a more faithful approximation to exact inference. When fit to empirical data, the VKF is better behaved than alternatives and better captures human choice data in two independent datasets of probabilistic learning tasks. The proposed model provides a coherent account of learning in stable or volatile environments and has implications for decision neuroscience research.
Collapse
|
57
|
Brain dynamics for confidence-weighted learning. PLoS Comput Biol 2020; 16:e1007935. [PMID: 32484806 PMCID: PMC7292419 DOI: 10.1371/journal.pcbi.1007935] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2019] [Revised: 06/12/2020] [Accepted: 05/07/2020] [Indexed: 12/11/2022] Open
Abstract
Learning in a changing, uncertain environment is a difficult problem. A popular solution is to predict future observations and then use surprising outcomes to update those predictions. However, humans also have a sense of confidence that characterizes the precision of their predictions. Bayesian models use a confidence-weighting principle to regulate learning: for a given surprise, the update is smaller when the confidence about the prediction was higher. Prior behavioral evidence indicates that human learning adheres to this confidence-weighting principle. Here, we explored the human brain dynamics sub-tending the confidence-weighting of learning using magneto-encephalography (MEG). During our volatile probability learning task, subjects’ confidence reports conformed with Bayesian inference. MEG revealed several stimulus-evoked brain responses whose amplitude reflected surprise, and some of them were further shaped by confidence: surprise amplified the stimulus-evoked response whereas confidence dampened it. Confidence about predictions also modulated several aspects of the brain state: pupil-linked arousal and beta-range (15–30 Hz) oscillations. The brain state in turn modulated specific stimulus-evoked surprise responses following the confidence-weighting principle. Our results thus indicate that there exist, in the human brain, signals reflecting surprise that are dampened by confidence in a way that is appropriate for learning according to Bayesian inference. They also suggest a mechanism for confidence-weighted learning: confidence about predictions would modulate intrinsic properties of the brain state to amplify or dampen surprise responses evoked by discrepant observations. Learning in a changing and uncertain world is difficult. In this context, facing a discrepancy between my current belief and new observations may reflect random fluctuations (e.g. my commute train is unexpectedly late, but it happens sometimes), if so, I should ignore this discrepancy and not change erratically my belief. However, this discrepancy could also denote a profound change (e.g. the train company changed and is less reliable), in this case, I should promptly revise my current belief. Human learning is adaptive: we change how much we learn from new observations, in particular, we promote flexibility when facing profound changes. A mathematical analysis of the problem shows that we should increase flexibility when the confidence about our current belief is low, which occurs when a change is suspected. Here, I show that human learners entertain rational confidence levels during the learning of changing probabilities. This confidence modulates intrinsic properties of the brain state (oscillatory activity and neuromodulation) which in turn amplifies or reduces, depending on whether confidence is low or high, the neural responses to discrepant observations. This confidence-weighting mechanism could underpin adaptive learning.
Collapse
|
58
|
Adams RA, Moutoussis M, Nour MM, Dahoun T, Lewis D, Illingworth B, Veronese M, Mathys C, de Boer L, Guitart-Masip M, Friston KJ, Howes OD, Roiser JP. Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models. Cereb Cortex 2020; 30:3573-3589. [PMID: 32083297 PMCID: PMC7233027 DOI: 10.1093/cercor/bhz327] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 11/18/2019] [Accepted: 12/05/2019] [Indexed: 12/17/2022] Open
Abstract
Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans.
Collapse
Affiliation(s)
- Rick A Adams
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
- Division of Psychiatry, University College London, London W1T 7NF, UK
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
| | - Michael Moutoussis
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
| | - Matthew M Nour
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King’s College London, London SE5 8AF, UK
| | - Tarik Dahoun
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford OX3 7JX, UK
| | - Declan Lewis
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| | - Benjamin Illingworth
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| | - Mattia Veronese
- Centre for Neuroimaging Sciences, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King's College London, London SE5 8AF, UK
| | - Christoph Mathys
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
- Scuola Internazionale Superiore di Studi Avanzati (SISSA), 34136 Trieste, Italy
- Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich and ETH Zurich, 8032 Zurich, Switzerland
| | - Lieke de Boer
- Aging Research Center, Karolinska Institute, 171 65 Stockholm, Sweden
| | - Marc Guitart-Masip
- Max Planck-UCL Centre for Computational Psychiatry and Ageing Research, London WC1B 5EH, UK
- Aging Research Center, Karolinska Institute, 171 65 Stockholm, Sweden
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3BG, UK
| | - Oliver D Howes
- Psychiatric Imaging Group, Robert Steiner MRI Unit, MRC London Institute of Medical Sciences, Hammersmith Hospital, London W12 0NN, UK
- Institute of Clinical Sciences, Faculty of Medicine, Imperial College London, Hammersmith Hospital, London W12 0NN, UK
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King’s College London, London SE5 8AF, UK
| | - Jonathan P Roiser
- Institute of Cognitive Neuroscience, University College London, London WC1N 3AZ, UK
| |
Collapse
|
59
|
Guirado R, Perez-Rando M, Ferragud A, Gutierrez-Castellanos N, Umemori J, Carceller H, Nacher J, Castillo-Gómez E. A Critical Period for Prefrontal Network Configurations Underlying Psychiatric Disorders and Addiction. Front Behav Neurosci 2020; 14:51. [PMID: 32317945 PMCID: PMC7155216 DOI: 10.3389/fnbeh.2020.00051] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2019] [Accepted: 03/19/2020] [Indexed: 12/24/2022] Open
Abstract
The medial prefrontal cortex (mPFC) has been classically defined as the brain region responsible for higher cognitive functions, including the decision-making process. Ample information has been gathered during the last 40 years in an attempt to understand how it works. We now know extensively about the connectivity of this region and its relationship with neuromodulatory ascending projection areas, such as the dorsal raphe nucleus (DRN) or the ventral tegmental area (VTA). Both areas are well-known regulators of the reward-based decision-making process and hence likely to be involved in processes like evidence integration, impulsivity or addiction biology, but also in helping us to predict the valence of our future actions: i.e., what is “good” and what is “bad.” Here we propose a hypothesis of a critical period, during which the inputs of the mPFC compete for target innervation, establishing specific prefrontal network configurations in the adult brain. We discuss how these different prefrontal configurations are linked to brain diseases such as addiction or neuropsychiatric disorders, and especially how drug abuse and other events during early life stages might lead to the formation of more vulnerable prefrontal network configurations. Finally, we show different promising pharmacological approaches that, when combined with the appropriate stimuli, will be able to re-establish these functional prefrontocortical configurations during adulthood.
Collapse
Affiliation(s)
- Ramon Guirado
- Neurobiology Unit, Department of Cell Biology, Interdisciplinary Research Structure for Biotechnology and Biomedicine (BIOTECMED), Universitat de Valencia, Valencia, Spain.,Neuroscience Center, University of Helsinki, Helsinki, Finland.,Spanish National Network for Research in Mental Health, Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Madrid, Spain.,Dirección General de Universidades, Gobierno de Aragón, Zaragoza, Spain
| | - Marta Perez-Rando
- MassGeneral Institute for Neurodegenerative Disease, Massachusetts General Hospital and Harvard Medical School, Boston, MA, United States
| | - Antonio Ferragud
- Department of Psychology, Cambridge University, Cambridge, United Kingdom
| | | | - Juzoh Umemori
- Neuroscience Center, University of Helsinki, Helsinki, Finland
| | - Hector Carceller
- Neurobiology Unit, Department of Cell Biology, Interdisciplinary Research Structure for Biotechnology and Biomedicine (BIOTECMED), Universitat de Valencia, Valencia, Spain
| | - Juan Nacher
- Neurobiology Unit, Department of Cell Biology, Interdisciplinary Research Structure for Biotechnology and Biomedicine (BIOTECMED), Universitat de Valencia, Valencia, Spain.,Spanish National Network for Research in Mental Health, Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Madrid, Spain.,Fundación Investigación Hospital Clínico de Valencia, INCLIVA, Valencia, Spain
| | - Esther Castillo-Gómez
- Spanish National Network for Research in Mental Health, Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Madrid, Spain.,Department of Medicine, School of Medical Sciences, Universitat Jaume I, Valencia, Spain
| |
Collapse
|
60
|
Corlett P. Predicting to Perceive and Learning When to Learn. Trends Cogn Sci 2020; 24:259-260. [PMID: 32160559 PMCID: PMC7509799 DOI: 10.1016/j.tics.2019.12.005] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2019] [Accepted: 12/05/2019] [Indexed: 10/25/2022]
Affiliation(s)
- Philip Corlett
- Clinical Neuroscience Research Unit (CNRU), Yale University, 34 Park Street, New Haven, CT 06519, USA.
| |
Collapse
|
61
|
Prior cocaine self-administration impairs attention signals in anterior cingulate cortex. Neuropsychopharmacology 2020; 45:833-841. [PMID: 31775158 PMCID: PMC7075947 DOI: 10.1038/s41386-019-0578-2] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/05/2019] [Revised: 11/04/2019] [Accepted: 11/15/2019] [Indexed: 01/11/2023]
Abstract
Although maladaptive decision-making is a defining feature of drug abuse and addiction, we have yet to ascertain how cocaine self-administration disrupts neural signals in anterior cingulate cortex (ACC), a brain region thought to contribute to attentional control. To address this issue, rats were trained on a reward-guided decision-making task; reward value was manipulated by independently varying the size of or the delay to reward over several trial blocks. Subsequently, rats self-administered either a cocaine (experimental group) or sucrose (control) during 12 consecutive days, after which they underwent a 1-month withdrawal period. Upon completion of this period, rats performed the previously learned reward-guided decision-making task while we recorded from single neurons in ACC. We demonstrate that prior cocaine self-administration attenuates attention and attention-related ACC signals in an intake-dependent manner, and that changes in attention are decoupled from ACC firing. These effects likely contribute to the impaired decision-making-typified by chronic substance abuse and relapse-observed after drug use.
Collapse
|
62
|
Ergo K, De Loof E, Verguts T. Reward Prediction Error and Declarative Memory. Trends Cogn Sci 2020; 24:388-397. [PMID: 32298624 DOI: 10.1016/j.tics.2020.02.009] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2019] [Revised: 02/03/2020] [Accepted: 02/22/2020] [Indexed: 01/04/2023]
Abstract
Learning based on reward prediction error (RPE) was originally proposed in the context of nondeclarative memory. We postulate that RPE may support declarative memory as well. Indeed, recent years have witnessed a number of independent empirical studies reporting effects of RPE on declarative memory. We provide a brief overview of these studies, identify emerging patterns, and discuss open issues such as the role of signed versus unsigned RPEs in declarative learning.
Collapse
Affiliation(s)
- Kate Ergo
- Department of Experimental Psychology, Ghent University, Henri Dunantlaan 2, B-9000 Ghent, Belgium
| | - Esther De Loof
- Department of Experimental Psychology, Ghent University, Henri Dunantlaan 2, B-9000 Ghent, Belgium
| | - Tom Verguts
- Department of Experimental Psychology, Ghent University, Henri Dunantlaan 2, B-9000 Ghent, Belgium.
| |
Collapse
|
63
|
Song MR, Lee SW. Dynamic resource allocation during reinforcement learning accounts for ramping and phasic dopamine activity. Neural Netw 2020; 126:95-107. [PMID: 32203877 DOI: 10.1016/j.neunet.2020.03.005] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2019] [Revised: 01/22/2020] [Accepted: 03/02/2020] [Indexed: 11/29/2022]
Abstract
For an animal to learn about its environment with limited motor and cognitive resources, it should focus its resources on potentially important stimuli. However, too narrow focus is disadvantageous for adaptation to environmental changes. Midbrain dopamine neurons are excited by potentially important stimuli, such as reward-predicting or novel stimuli, and allocate resources to these stimuli by modulating how an animal approaches, exploits, explores, and attends. The current study examined the theoretical possibility that dopamine activity reflects the dynamic allocation of resources for learning. Dopamine activity may transition between two patterns: (1) phasic responses to cues and rewards, and (2) ramping activity arising as the agent approaches the reward. Phasic excitation has been explained by prediction errors generated by experimentally inserted cues. However, when and why dopamine activity transitions between the two patterns remain unknown. By parsimoniously modifying a standard temporal difference (TD) learning model to accommodate a mixed presentation of both experimental and environmental stimuli, we simulated dopamine transitions and compared them with experimental data from four different studies. The results suggested that dopamine transitions from ramping to phasic patterns as the agent focuses its resources on a small number of reward-predicting stimuli, thus leading to task dimensionality reduction. The opposite occurs when the agent re-distributes its resources to adapt to environmental changes, resulting in task dimensionality expansion. This research elucidates the role of dopamine in a broader context, providing a potential explanation for the diverse repertoire of dopamine activity that cannot be explained solely by prediction error.
Collapse
Affiliation(s)
- Minryung R Song
- Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, South Korea
| | - Sang Wan Lee
- Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, South Korea; Program of Brain and Cognitive Engineering, Daejeon, 34141, South Korea; KAIST Institute for Health, Science, and Technology, Daejeon, 34141, South Korea; KAIST Institute for Artificial Intelligence, Daejeon, 34141, South Korea; KAIST Center for Neuroscience-inspired AI, Daejeon, 34141, South Korea.
| |
Collapse
|
64
|
Soltani A, Izquierdo A. Adaptive learning under expected and unexpected uncertainty. Nat Rev Neurosci 2020; 20:635-644. [PMID: 31147631 DOI: 10.1038/s41583-019-0180-y] [Citation(s) in RCA: 114] [Impact Index Per Article: 28.5] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
The outcome of a decision is often uncertain, and outcomes can vary over repeated decisions. Whether decision outcomes should substantially affect behaviour and learning depends on whether they are representative of a typically experienced range of outcomes or signal a change in the reward environment. Successful learning and decision-making therefore require the ability to estimate expected uncertainty (related to the variability of outcomes) and unexpected uncertainty (related to the variability of the environment). Understanding the bases and effects of these two types of uncertainty and the interactions between them - at the computational and the neural level - is crucial for understanding adaptive learning. Here, we examine computational models and experimental findings to distil computational principles and neural mechanisms for adaptive learning under uncertainty.
Collapse
Affiliation(s)
- Alireza Soltani
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA.
| | - Alicia Izquierdo
- Department of Psychology, The Brain Research Institute, University of California, Los Angeles, Los Angeles, CA, USA.
| |
Collapse
|
65
|
Stojić H, Orquin JL, Dayan P, Dolan RJ, Speekenbrink M. Uncertainty in learning, choice, and visual fixation. Proc Natl Acad Sci U S A 2020; 117:3291-3300. [PMID: 31980535 PMCID: PMC7022187 DOI: 10.1073/pnas.1911348117] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
Uncertainty plays a critical role in reinforcement learning and decision making. However, exactly how it influences behavior remains unclear. Multiarmed-bandit tasks offer an ideal test bed, since computational tools such as approximate Kalman filters can closely characterize the interplay between trial-by-trial values, uncertainty, learning, and choice. To gain additional insight into learning and choice processes, we obtained data from subjects' overt allocation of gaze. The estimated value and estimation uncertainty of options influenced what subjects looked at before choosing; these same quantities also influenced choice, as additionally did fixation itself. A momentary measure of uncertainty in the form of absolute prediction errors determined how long participants looked at the obtained outcomes. These findings affirm the importance of uncertainty in multiple facets of behavior and help delineate its effects on decision making.
Collapse
Affiliation(s)
- Hrvoje Stojić
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, United Kingdom;
| | - Jacob L Orquin
- Department of Management/MAPP, Aarhus University, Aarhus 8210, Denmark
- Centre for Research in Marketing and Consumer Psychology, Reykjavik University, 101 Reykjavik, Iceland
| | - Peter Dayan
- Department of Computational Neuroscience, Max Planck Institute for Biological Cybernetics, Tübingen 72076, Germany
| | - Raymond J Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, United Kingdom
| | - Maarten Speekenbrink
- Department of Experimental Psychology, University College London, London WC1H 0AP, United Kingdom
| |
Collapse
|
66
|
Corsi MC, Chavez M, Schwartz D, George N, Hugueville L, Kahn AE, Dupont S, Bassett DS, De Vico Fallani F. Functional disconnection of associative cortical areas predicts performance during BCI training. Neuroimage 2020; 209:116500. [PMID: 31927130 PMCID: PMC7056534 DOI: 10.1016/j.neuroimage.2019.116500] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2019] [Revised: 12/13/2019] [Accepted: 12/25/2019] [Indexed: 11/21/2022] Open
Abstract
Brain-computer interfaces (BCIs) have been largely developed to allow communication, control, and neurofeedback in human beings. Despite their great potential, BCIs perform inconsistently across individuals and the neural processes that enable humans to achieve good control remain poorly understood. To address this question, we performed simultaneous high-density electroencephalographic (EEG) and magnetoencephalographic (MEG) recordings in a motor imagery-based BCI training involving a group of healthy subjects. After reconstructing the signals at the cortical level, we showed that the reinforcement of motor-related activity during the BCI skill acquisition is paralleled by a progressive disconnection of associative areas which were not directly targeted during the experiments. Notably, these network connectivity changes reflected growing automaticity associated with BCI performance and predicted future learning rate. Altogether, our findings provide new insights into the large-scale cortical organizational mechanisms underlying BCI learning, which have implications for the improvement of this technology in a broad range of real-life applications.
Collapse
Affiliation(s)
- Marie-Constance Corsi
- Inria Paris, Aramis Project-team, F-75013, Paris, France; Institut du Cerveau et de la Moelle Epinière, ICM, Inserm, U 1127, CNRS, UMR 7225, Sorbonne Université, F-75013, Paris, France.
| | | | - Denis Schwartz
- Institut du Cerveau et de la Moelle Epinière, ICM, Inserm U 1127, CNRS UMR 7225, Sorbonne Université, Ecole Normale Supérieure, ENS, Centre MEG-EEG, F-75013, Paris, France
| | - Nathalie George
- Institut du Cerveau et de la Moelle Epinière, ICM, Inserm, U 1127, CNRS, UMR 7225, Sorbonne Université, F-75013, Paris, France; Institut du Cerveau et de la Moelle Epinière, ICM, Inserm U 1127, CNRS UMR 7225, Sorbonne Université, Ecole Normale Supérieure, ENS, Centre MEG-EEG, F-75013, Paris, France
| | - Laurent Hugueville
- Institut du Cerveau et de la Moelle Epinière, ICM, Inserm U 1127, CNRS UMR 7225, Sorbonne Université, Ecole Normale Supérieure, ENS, Centre MEG-EEG, F-75013, Paris, France
| | - Ari E Kahn
- Department of Bioengineering, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA, 19104, USA
| | - Sophie Dupont
- Institut du Cerveau et de la Moelle Epinière, ICM, Inserm, U 1127, CNRS, UMR 7225, Sorbonne Université, F-75013, Paris, France
| | - Danielle S Bassett
- Department of Bioengineering, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA, 19104, USA; Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, 19104, USA; Department of Physics and Astronomy, College of Arts and Sciences, University of Pennsylvania, Philadelphia, PA, 19104, USA; Department of Electrical and Systems Engineering, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA, 19104, USA; Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, 19104, USA; Santa Fe Institute, Santa Fe, NM, 87501, USA
| | - Fabrizio De Vico Fallani
- Inria Paris, Aramis Project-team, F-75013, Paris, France; Institut du Cerveau et de la Moelle Epinière, ICM, Inserm, U 1127, CNRS, UMR 7225, Sorbonne Université, F-75013, Paris, France.
| |
Collapse
|
67
|
Kim H, Schlichting ML, Preston AR, Lewis-Peacock JA. Predictability Changes What We Remember in Familiar Temporal Contexts. J Cogn Neurosci 2020; 32:124-140. [PMID: 31560266 PMCID: PMC6996874 DOI: 10.1162/jocn_a_01473] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
The human brain constantly anticipates the future based on memories of the past. Encountering a familiar situation reactivates memory of previous encounters, which can trigger a prediction of what comes next to facilitate responsiveness. However, a prediction error can lead to pruning of the offending memory, a process that weakens its representation in the brain and leads to forgetting. Our goal in this study was to evaluate whether memories are spared from such pruning in situations that allow for accurate predictions at the categorical level, despite prediction errors at the item level. Participants viewed a sequence of objects, some of which reappeared multiple times ("cues"), followed always by novel items. Half of the cues were followed by new items from different (unpredictable) categories, while others were followed by new items from a single (predictable) category. Pattern classification of fMRI data was used to identify category-specific predictions after each cue. Pruning was observed only in unpredictable contexts, while encoding of new items was less robust in predictable contexts. These findings demonstrate that how associative memories are updated is influenced by the reliability of abstract-level predictions in familiar contexts.
Collapse
Affiliation(s)
- Hyojeong Kim
- Department of Psychology, University of Texas at Austin, Austin, TX
| | | | - Alison R. Preston
- Department of Psychology, University of Texas at Austin, Austin, TX
- Department of Neuroscience, University of Texas at Austin, Austin, TX
| | | |
Collapse
|
68
|
Drugowitsch J, Mendonça AG, Mainen ZF, Pouget A. Learning optimal decisions with confidence. Proc Natl Acad Sci U S A 2019; 116:24872-24880. [PMID: 31732671 PMCID: PMC6900530 DOI: 10.1073/pnas.1906787116] [Citation(s) in RCA: 33] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
Diffusion decision models (DDMs) are immensely successful models for decision making under uncertainty and time pressure. In the context of perceptual decision making, these models typically start with two input units, organized in a neuron-antineuron pair. In contrast, in the brain, sensory inputs are encoded through the activity of large neuronal populations. Moreover, while DDMs are wired by hand, the nervous system must learn the weights of the network through trial and error. There is currently no normative theory of learning in DDMs and therefore no theory of how decision makers could learn to make optimal decisions in this context. Here, we derive such a rule for learning a near-optimal linear combination of DDM inputs based on trial-by-trial feedback. The rule is Bayesian in the sense that it learns not only the mean of the weights but also the uncertainty around this mean in the form of a covariance matrix. In this rule, the rate of learning is proportional (respectively, inversely proportional) to confidence for incorrect (respectively, correct) decisions. Furthermore, we show that, in volatile environments, the rule predicts a bias toward repeating the same choice after correct decisions, with a bias strength that is modulated by the previous choice's difficulty. Finally, we extend our learning rule to cases for which one of the choices is more likely a priori, which provides insights into how such biases modulate the mechanisms leading to optimal decisions in diffusion models.
Collapse
Affiliation(s)
- Jan Drugowitsch
- Department of Neurobiology, Harvard Medical School, Boston, MA 02115;
| | - André G Mendonça
- Champalimaud Research, Champalimaud Centre for the Unknown, 1400-038 Lisbon, Portugal
| | - Zachary F Mainen
- Champalimaud Research, Champalimaud Centre for the Unknown, 1400-038 Lisbon, Portugal
| | - Alexandre Pouget
- Department of Basic Neuroscience, University of Geneva, CH-1211 Geneva, Switzerland
| |
Collapse
|
69
|
Dissociating neural learning signals in human sign- and goal-trackers. Nat Hum Behav 2019; 4:201-214. [DOI: 10.1038/s41562-019-0765-5] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2017] [Accepted: 09/25/2019] [Indexed: 01/20/2023]
|
70
|
Wise T, Michely J, Dayan P, Dolan RJ. A computational account of threat-related attentional bias. PLoS Comput Biol 2019; 15:e1007341. [PMID: 31600187 PMCID: PMC6786521 DOI: 10.1371/journal.pcbi.1007341] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2019] [Accepted: 08/18/2019] [Indexed: 12/15/2022] Open
Abstract
Visual selective attention acts as a filter on perceptual information, facilitating learning and inference about important events in an agent's environment. A role for visual attention in reward-based decisions has previously been demonstrated, but it remains unclear how visual attention is recruited during aversive learning, particularly when learning about multiple stimuli concurrently. This question is of particular importance in psychopathology, where enhanced attention to threat is a putative feature of pathological anxiety. Using an aversive reversal learning task that required subjects to learn, and exploit, predictions about multiple stimuli, we show that the allocation of visual attention is influenced significantly by aversive value but not by uncertainty. Moreover, this relationship is bidirectional in that attention biases value updates for attended stimuli, resulting in heightened value estimates. Our findings have implications for understanding biased attention in psychopathology and support a role for learning in the expression of threat-related attentional biases in anxiety.
Collapse
Affiliation(s)
- Toby Wise
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, United Kingdom
- * E-mail:
| | - Jochen Michely
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, United Kingdom
| | - Peter Dayan
- Max Planck Institute for Biological Cybernetics, Tübingen, Germany
| | - Raymond J. Dolan
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, United Kingdom
| |
Collapse
|
71
|
Minnekhanov AA, Emelyanov AV, Lapkin DA, Nikiruy KE, Shvetsov BS, Nesmelov AA, Rylkov VV, Demin VA, Erokhin VV. Parylene Based Memristive Devices with Multilevel Resistive Switching for Neuromorphic Applications. Sci Rep 2019; 9:10800. [PMID: 31346245 PMCID: PMC6658497 DOI: 10.1038/s41598-019-47263-9] [Citation(s) in RCA: 66] [Impact Index Per Article: 13.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2019] [Accepted: 07/12/2019] [Indexed: 11/09/2022] Open
Abstract
In this paper, the resistive switching and neuromorphic behaviour of memristive devices based on parylene, a polymer both low-cost and safe for the human body, is comprehensively studied. The Metal/Parylene/ITO sandwich structures were prepared by means of the standard gas phase surface polymerization method with different top active metal electrodes (Ag, Al, Cu or Ti of ~500 nm thickness). These organic memristive devices exhibit excellent performance: low switching voltage (down to 1 V), large OFF/ON resistance ratio (up to 104), retention (≥104 s) and high multilevel resistance switching (at least 16 stable resistive states in the case of Cu electrodes). We have experimentally shown that parylene-based memristive elements can be trained by a biologically inspired spike-timing-dependent plasticity (STDP) mechanism. The obtained results have been used to implement a simple neuromorphic network model of classical conditioning. The described advantages allow considering parylene-based organic memristors as prospective devices for hardware realization of spiking artificial neuron networks capable of supervised and unsupervised learning and suitable for biomedical applications.
Collapse
Affiliation(s)
| | - Andrey V Emelyanov
- National Research Centre "Kurchatov Institute", 123182, Moscow, Russia.,Moscow Institute of Physics and Technology, 141700, Dolgoprudny, Moscow Region, Russia
| | - Dmitry A Lapkin
- National Research Centre "Kurchatov Institute", 123182, Moscow, Russia.,Deutsches Elektronen-Synchrotron DESY, 22607, Hamburg, Germany
| | - Kristina E Nikiruy
- National Research Centre "Kurchatov Institute", 123182, Moscow, Russia.,Moscow Institute of Physics and Technology, 141700, Dolgoprudny, Moscow Region, Russia
| | | | | | - Vladimir V Rylkov
- National Research Centre "Kurchatov Institute", 123182, Moscow, Russia.,Kotel'nikov Institute of Radio Engineering and Electronics RAS, 141190, Fryazino, Moscow Region, Russia
| | - Vyacheslav A Demin
- National Research Centre "Kurchatov Institute", 123182, Moscow, Russia.,Moscow Institute of Physics and Technology, 141700, Dolgoprudny, Moscow Region, Russia
| | - Victor V Erokhin
- National Research Centre "Kurchatov Institute", 123182, Moscow, Russia. .,CNR-IMEM (National Research Council, Institute of Materials for Electronics and Magnetism), 43124, Parma, Italy.
| |
Collapse
|
72
|
Horan M, Daddaoua N, Gottlieb J. Parietal neurons encode information sampling based on decision uncertainty. Nat Neurosci 2019; 22:1327-1335. [PMID: 31285613 PMCID: PMC6660422 DOI: 10.1038/s41593-019-0440-1] [Citation(s) in RCA: 40] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2018] [Accepted: 05/28/2019] [Indexed: 01/19/2023]
Abstract
In natural behavior animals actively gather information that is relevant for learning or actions, but the mechanisms of active sampling are rarely investigated. We tested parietal neurons involved in oculomotor control in a task in which monkeys made saccades to gather visual information before reporting a decision based on the information. We show that the neurons encode, before the saccade, the information gains (reduction in decision uncertainty) that the saccade was expected to bring, correlating with the monkeys’ efficiency in processing the information in the post-saccadic fixation. Informational sensitivity is independent of the neurons’ reward sensitivity, which is unreliable across task contexts, inconsistent with the view that the cells encode economic utility. Instead, we suggest that parietal cells are involved in implementing active sampling policies, showing uncertainty-dependent boosts of neural gain that facilitate the selection of relevant cues and the efficient use of the information delivered by these cues.
Collapse
Affiliation(s)
- Mattias Horan
- Department of Neuroscience, Columbia University, New York, NY, USA
| | - Nabil Daddaoua
- Department of Neuroscience, Columbia University, New York, NY, USA.,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Jacqueline Gottlieb
- Department of Neuroscience, Columbia University, New York, NY, USA. .,Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA. .,The Kavli Institute for Brain Science, Columbia University, New York, NY, USA.
| |
Collapse
|
73
|
Fisher AV. Selective sustained attention: a developmental foundation for cognition. Curr Opin Psychol 2019; 29:248-253. [PMID: 31284233 DOI: 10.1016/j.copsyc.2019.06.002] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2018] [Revised: 06/04/2019] [Accepted: 06/10/2019] [Indexed: 10/26/2022]
Abstract
Higher-order cognition, particularly in real-life settings, often requires that parts of the sensory input be processed at the exclusion of others over a period of time. Consequently, this review focuses on the development of attention that is both selective (which entails processing parts of the sensory input at the exclusion of others) and sustained (which entails maintaining sensitivity to incoming stimuli for a period of time). Recent findings from four distinct areas of research reviewed here suggest that: (1) the underlying neural circuitry of selective sustained attention involves multiple cortical and subcortical brain regions; (2) selective sustained attention in infancy provides a developmental foundation for the emergence of executive function later in life; (3) suppression-based mechanisms of attentional selection that begin to emerge during the first year of life are important for memory and learning; and (4) selective sustained attention appears to be malleable through pre-natal and post-natal nutritional supplementation and interactions with mature social partners.
Collapse
Affiliation(s)
- Anna V Fisher
- Department of Psychology, Carnegie Mellon University, 33-I Baker Hall, 5000 Forbes Ave., Pittsburgh, PA 15213, United States.
| |
Collapse
|
74
|
Chen J, Wang C, Shi Q, Feng Y, Chen C. Social recommendation based on users’ attention and preference. Neurocomputing 2019. [DOI: 10.1016/j.neucom.2019.02.045] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
75
|
Watson P, Pearson D, Wiers RW, Le Pelley ME. Prioritizing pleasure and pain: attentional capture by reward-related and punishment-related stimuli. Curr Opin Behav Sci 2019. [DOI: 10.1016/j.cobeha.2018.12.002] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
76
|
Radulescu A, Niv Y, Ballard I. Holistic Reinforcement Learning: The Role of Structure and Attention. Trends Cogn Sci 2019; 23:278-292. [PMID: 30824227 PMCID: PMC6472955 DOI: 10.1016/j.tics.2019.01.010] [Citation(s) in RCA: 43] [Impact Index Per Article: 8.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2018] [Revised: 01/20/2019] [Accepted: 01/24/2019] [Indexed: 10/27/2022]
Abstract
Compact representations of the environment allow humans to behave efficiently in a complex world. Reinforcement learning models capture many behavioral and neural effects but do not explain recent findings showing that structure in the environment influences learning. In parallel, Bayesian cognitive models predict how humans learn structured knowledge but do not have a clear neurobiological implementation. We propose an integration of these two model classes in which structured knowledge learned via approximate Bayesian inference acts as a source of selective attention. In turn, selective attention biases reinforcement learning towards relevant dimensions of the environment. An understanding of structure learning will help to resolve the fundamental challenge in decision science: explaining why people make the decisions they do.
Collapse
Affiliation(s)
- Angela Radulescu
- Psychology Department, Princeton University, Princeton, NJ, USA; Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Yael Niv
- Psychology Department, Princeton University, Princeton, NJ, USA; Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Ian Ballard
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA.
| |
Collapse
|
77
|
Abstract
Habits form a crucial component of behavior. In recent years, key computational models have conceptualized habits as arising from model-free reinforcement learning mechanisms, which typically select between available actions based on the future value expected to result from each. Traditionally, however, habits have been understood as behaviors that can be triggered directly by a stimulus, without requiring the animal to evaluate expected outcomes. Here, we develop a computational model instantiating this traditional view, in which habits develop through the direct strengthening of recently taken actions rather than through the encoding of outcomes. We demonstrate that this model accounts for key behavioral manifestations of habits, including insensitivity to outcome devaluation and contingency degradation, as well as the effects of reinforcement schedule on the rate of habit formation. The model also explains the prevalent observation of perseveration in repeated-choice tasks as an additional behavioral manifestation of the habit system. We suggest that mapping habitual behaviors onto value-free mechanisms provides a parsimonious account of existing behavioral and neural data. This mapping may provide a new foundation for building robust and comprehensive models of the interaction of habits with other, more goal-directed types of behaviors and help to better guide research into the neural mechanisms underlying control of instrumental behavior more generally. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
Collapse
Affiliation(s)
| | - Amitai Shenhav
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown Institute for Brain Science, Brown University
| | | |
Collapse
|
78
|
Oemisch M, Westendorff S, Azimi M, Hassani SA, Ardid S, Tiesinga P, Womelsdorf T. Feature-specific prediction errors and surprise across macaque fronto-striatal circuits. Nat Commun 2019; 10:176. [PMID: 30635579 PMCID: PMC6329800 DOI: 10.1038/s41467-018-08184-9] [Citation(s) in RCA: 33] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2018] [Accepted: 12/20/2018] [Indexed: 01/23/2023] Open
Abstract
To adjust expectations efficiently, prediction errors need to be associated with the precise features that gave rise to the unexpected outcome, but this credit assignment may be problematic if stimuli differ on multiple dimensions and it is ambiguous which feature dimension caused the outcome. Here, we report a potential solution: neurons in four recorded areas of the anterior fronto-striatal networks encode prediction errors that are specific to feature values of different dimensions of attended multidimensional stimuli. The most ubiquitous prediction error occurred for the reward-relevant dimension. Feature-specific prediction error signals a) emerge on average shortly after non-specific prediction error signals, b) arise earliest in the anterior cingulate cortex and later in dorsolateral prefrontal cortex, caudate and ventral striatum, and c) contribute to feature-based stimulus selection after learning. Thus, a widely-distributed feature-specific eligibility trace may be used to update synaptic weights for improved feature-based attention. In order to adjust expectations efficiently, prediction errors need to be associated with the features that gave rise to the unexpected outcome. Here, the authors show that neurons in anterior fronto-striatal networks encode prediction errors that are specific to feature values of different stimulus dimensions.
Collapse
Affiliation(s)
- Mariann Oemisch
- Department of Biology, Centre for Vision Research, York University, 4700 Keele Street, Toronto, ON, M6J 1P3, Canada. .,Department of Neuroscience, Yale University School of Medicine, New Haven, CT, 06510, USA.
| | - Stephanie Westendorff
- Department of Biology, Centre for Vision Research, York University, 4700 Keele Street, Toronto, ON, M6J 1P3, Canada.,Institute of Neurobiology, University of Tübingen, Tübingen, 72076, Germany
| | - Marzyeh Azimi
- Department of Biology, Centre for Vision Research, York University, 4700 Keele Street, Toronto, ON, M6J 1P3, Canada
| | - Seyed Alireza Hassani
- Department of Biology, Centre for Vision Research, York University, 4700 Keele Street, Toronto, ON, M6J 1P3, Canada.,Department of Psychology, Vanderbilt University, Nashville, TN, 37240, USA
| | - Salva Ardid
- Department of Mathematics and Statistics, Boston University, Boston, MA, 02215, USA
| | - Paul Tiesinga
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, 6525 EN, Netherlands
| | - Thilo Womelsdorf
- Department of Biology, Centre for Vision Research, York University, 4700 Keele Street, Toronto, ON, M6J 1P3, Canada. .,Department of Psychology, Vanderbilt University, Nashville, TN, 37240, USA.
| |
Collapse
|
79
|
Mao H, Yuan Y, Si J. Cortical neural modulation by previous trial outcome differentiates fast- from slow-learning rats on a visuomotor directional choice task. J Neurophysiol 2019; 121:50-60. [PMID: 30379632 DOI: 10.1152/jn.00950.2016] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023] Open
Abstract
To better understand the neural cortical underpinnings that explain behavioral differences in learning rate, we recorded single-unit activity in primary motor (M1) and secondary motor (M2) areas while rats learned to perform a directional (left or right) operant visuomotor association task. Analysis of neural activity during the early portion of the cue period showed that neural modulation in the motor cortex was most strongly associated with two task factors: the previous trial outcome (success or error) and the current trial's directional choice (left or right). Furthermore, the fast learners, defined as those who had steeper learning curves and required fewer learning sessions to reach criterion performance, encoded the previous trial outcome factor more strongly than the directional choice factor. Conversely, the slow learners encoded directional choice more strongly than previous trial outcome. These differences in task factor encoding were observed in both the percentage of neurons and the neural modulation depth. These results suggest that fast learning is accompanied by a stronger component of previous trial outcome in the modulation representation present in motor cortex, which therefore may be a contributing factor to behavioral differences in learning rate. NEW & NOTEWORTHY We chronically recorded neural activity as rats learned a visuomotor directional choice task from a naive state. Learning rates varied. Single-unit neural modulation of two motor areas revealed that the fast learners encoded previous trial outcome more strongly than directional choice, whereas the reverse was true for slow learners. This finding provides novel evidence that rat learning rate is strongly correlated with the strength of neural modulation by previous trial outcome in motor cortex.
Collapse
Affiliation(s)
- Hongwei Mao
- Systems Neuroscience Institute, University of Pittsburgh , Pittsburgh, Pennsylvania.,Center for the Neural Basis of Cognition, University of Pittsburgh , Pittsburgh, Pennsylvania
| | - Yuan Yuan
- School of Electrical, Computer, and Energy Engineering, Arizona State University , Tempe, Arizona
| | - Jennie Si
- School of Electrical, Computer, and Energy Engineering, Arizona State University , Tempe, Arizona.,School of Biological and Health Systems Engineering, Arizona State University , Tempe, Arizona.,Interdisciplinary Graduate Program in Neuroscience, Arizona State University , Tempe, Arizona
| |
Collapse
|
80
|
van Kampen HS. The principle of consistency and the cause and function of behaviour. Behav Processes 2018; 159:42-54. [PMID: 30562561 DOI: 10.1016/j.beproc.2018.12.013] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2018] [Revised: 11/23/2018] [Accepted: 12/14/2018] [Indexed: 01/01/2023]
Abstract
At all levels of information processing in the brain, neural and cognitive structures tend towards a state of consistency. When two or more simultaneously active cognitive structures are logically inconsistent, arousal is increased, which activates processes with the expected consequence of increasing consistency and decreasing arousal. Increased arousal is experienced as aversive, while the expected or actual decrease in arousal as a result of increased consistency is experienced as rewarding. Modes of resolution of inconsistency can be divided into purely cognitive solutions, such as changing an attitude or an associated motor plan, and behavioural solutions, such as exploration, aggression, fear, and feeding. Models and theories consistent with the principle of consistency are numerous, have a long and continuing history, and come from many different scientific fields, such as social psychology, perception, neurocognition, learning, motor control, system control, ethology, and stress. The present paper presents a brief overview of relevant information from these fields of research, while focusing mainly on the implications of the principle of consistency for the understanding of the cause and function of behaviour. Based on this overview, it is proposed that all behaviour involving cognitive processing is caused by the activation of inconsistent cognitions and functions to increase perceived consistency.
Collapse
Affiliation(s)
- Hendrik S van Kampen
- Cognitive Neurobiology and Helmholtz Institute, Department of Psychology, Utrecht University, Padualaan 8, 3584 CH Utrecht, the Netherlands.
| |
Collapse
|
81
|
Hallquist MN, Dombrovski AY. Selective maintenance of value information helps resolve the exploration/exploitation dilemma. Cognition 2018; 183:226-243. [PMID: 30502584 DOI: 10.1016/j.cognition.2018.11.004] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2017] [Revised: 11/06/2018] [Accepted: 11/08/2018] [Indexed: 10/27/2022]
Abstract
In natural environments with many options of uncertain value, one faces a difficult tradeoff between exploiting familiar, valuable options or searching for better alternatives. Reinforcement learning models of this exploration/exploitation dilemma typically modulate the rate of exploratory choices or preferentially sample uncertain options. The extent to which such models capture human behavior remains unclear, in part because they do not consider the constraints on remembering what is learned. Using reinforcement-based timing as a motivating example, we show that selectively maintaining high-value actions compresses the amount of information to be tracked in learning, as quantified by Shannon's entropy. In turn, the information content of the value representation controls the balance between exploration (high entropy) and exploitation (low entropy). Selectively maintaining preferred action values while allowing others to decay renders the choices increasingly exploitative across learning episodes. To adjudicate among alternative maintenance and sampling strategies, we developed a new reinforcement learning model, StrategiC ExPloration/ExPloitation of Temporal Instrumental Contingencies (SCEPTIC). In computational studies, a resource-rational selective maintenance approach was as successful as more resource-intensive strategies. Furthermore, human behavior was consistent with selective maintenance; information compression was most pronounced in subjects with superior performance and non-verbal intelligence, and in learnable vs. unlearnable contingencies. Cognitively demanding uncertainty-directed exploration recovered a more accurate representation in simulations with no foraging advantage and was strongly unsupported in our human study.
Collapse
Affiliation(s)
- Michael N Hallquist
- Penn State University, Department of Psychology, 309 Moore Building, Penn State University, University Park, PA 16801, USA; University of Pittsburgh, Department of Psychiatry, 3811 O'Hara St., BT 742, Pittsburgh, PA 15213, USA.
| | - Alexandre Y Dombrovski
- University of Pittsburgh, Department of Psychiatry, 3811 O'Hara St., BT 742, Pittsburgh, PA 15213, USA.
| |
Collapse
|
82
|
Tripathi S, Taneja P, Jha SK. Training on an Appetitive (Delay)-Conditioning Task Enhances Oscillatory Waves During Sleep in the Cortical and Amygdalar Network. Front Behav Neurosci 2018; 12:260. [PMID: 30464744 PMCID: PMC6234907 DOI: 10.3389/fnbeh.2018.00260] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2017] [Accepted: 10/15/2018] [Indexed: 11/13/2022] Open
Abstract
Oscillating waves during sleep play an essential role in memory consolidation. The cortical slow wave activity (SWA) and sigma waves during NREM sleep and theta waves during REM sleep increase after a variety of memory tasks including declarative, procedural and associative learning tasks. These oscillatory waves during sleep help to promote neural dialog between circuitries, which possibly plays a causal role in memory consolidation. However, the role of sleep-associated oscillating waves in a complex appetitive-conditioning paradigm is not clear. The parietal cortex and amygdala are involved in the cognitive evaluation of the environmental stimuli, and appetitive conditioning. Here, we have studied the changes in sleep architecture and oscillatory waves during NREM and REM sleep in the parietal cortices and amygdalar-local field potential (A-LFP) after appetitive-conditioning in the rat. We observed that REM sleep increased significantly after appetitive conditioning, which significantly positively correlated with performance on the appetitive-conditioning task. Further, the cortical SWA (0.1-4.5 Hz), and sigma (12-14.25 Hz) waves during NREM sleep, theta (6-9 Hz) waves during REM sleep, the amygdalar SWA (0.1-3.75 Hz) during NREM sleep and theta (6-8.25 Hz) waves during REM sleep significantly increased after appetitive conditioning. Interestingly, the augmented oscillatory waves significantly positively correlated with the performances on the appetitive-conditioning task. Our results suggest that the augmented REM sleep after conditioning may be required for the consolidation of appetitive-conditioned memory. Further, a significant correlation between augmented power in oscillatory waves during sleep and performance suggesting that these waves may be playing a crucial role in the consolidation of appetitive-conditioned memory.
Collapse
Affiliation(s)
- Shweta Tripathi
- Department of Biotechnology, School of Engineering and Technology, Sharda University, Greater Noida, India.,School of Life Sciences, Jawaharlal Nehru University, New Delhi, India
| | - Pankaj Taneja
- Department of Biotechnology, School of Engineering and Technology, Sharda University, Greater Noida, India
| | - Sushil K Jha
- School of Life Sciences, Jawaharlal Nehru University, New Delhi, India
| |
Collapse
|
83
|
Lo Gerfo E, De Angelis J, Vergallito A, Bossi F, Romero Lauro LJ, Ricciardelli P. Can Monetary Reward Modulate Social Attention? Front Psychol 2018; 9:2213. [PMID: 30487771 PMCID: PMC6246685 DOI: 10.3389/fpsyg.2018.02213] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2018] [Accepted: 10/25/2018] [Indexed: 11/24/2022] Open
Abstract
Selective visual attention is a primary cognitive function, which allows the selection of the most relevant stimuli in the environment by prioritizing their processing. Several studies showed that this process can be influenced by both social signals, such as gaze direction (i.e., the Gaze Cueing Effect, GCE) and by the motivational valence of gratifying stimuli, such as monetary rewards. The aim of this study was to explore whether GCE could be modulated by a monetary reward. To this end, we created an experiment in which participants performed a gaze cuing task before and after an implicit learning task aiming to induce an association between gaze direction and monetary reward (experimental condition), or after a perceptual task (control condition). Statistical analyses were conducted following both a frequentist and a Bayesian approach. Results supported previous findings showing the presence of the GCE, i.e., faster responses in congruent trials when the target appeared in the gazed-at location. Interestingly, our results did not reveal significant differences among the conditions. Therefore, contrary to what was reported by previous attentional orienting studies with non-social stimuli, monetary reward does not seem to be able to modulate (or interfere with) the orienting of attention mediated by gaze direction as measured by the GCE. Taken together our results suggest that social signals such as gaze direction have a greater impact than monetary reward in orienting selective attention.
Collapse
Affiliation(s)
- Emanuele Lo Gerfo
- Department of Economics, Management and Statistics, University of Milano-Bicocca, Milan, Italy
- CISEPS, University of Milano-Bicocca, Milan, Italy
- NeuroMI – Milan Center for Neuroscience, Milan, Italy
| | - Jacopo De Angelis
- Department of Psychology, University of Milano-Bicocca, Milan, Italy
| | - Alessandra Vergallito
- NeuroMI – Milan Center for Neuroscience, Milan, Italy
- Department of Psychology, University of Milano-Bicocca, Milan, Italy
| | - Francesco Bossi
- NeuroMI – Milan Center for Neuroscience, Milan, Italy
- Social Cognition in Human-Robot Interaction, Istituto Italiano di Tecnologia, Genoa, Italy
| | - Leonor Josefina Romero Lauro
- NeuroMI – Milan Center for Neuroscience, Milan, Italy
- Department of Psychology, University of Milano-Bicocca, Milan, Italy
| | - Paola Ricciardelli
- NeuroMI – Milan Center for Neuroscience, Milan, Italy
- Department of Psychology, University of Milano-Bicocca, Milan, Italy
| |
Collapse
|
84
|
Abnormal reward prediction-error signalling in antipsychotic naive individuals with first-episode psychosis or clinical risk for psychosis. Neuropsychopharmacology 2018; 43:1691-1699. [PMID: 29748629 PMCID: PMC6006166 DOI: 10.1038/s41386-018-0056-2] [Citation(s) in RCA: 47] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/07/2017] [Revised: 03/12/2018] [Accepted: 03/13/2018] [Indexed: 12/20/2022]
Abstract
Ongoing research suggests preliminary, though not entirely consistent, evidence of neural abnormalities in signalling prediction errors in schizophrenia. Supporting theories suggest mechanistic links between the disruption of these processes and the generation of psychotic symptoms. However, it is unknown at what stage in the pathogenesis of psychosis these impairments in prediction-error signalling develop. One major confound in prior studies is the use of medicated patients with strongly varying disease durations. Our study aims to investigate the involvement of the meso-cortico-striatal circuitry during reward prediction-error signalling in earliest stages of psychosis. We studied patients with first-episode psychosis (FEP) and help-seeking individuals at-risk for psychosis due to sub-threshold prodromal psychotic symptoms. Patients with either FEP (n = 14), or at-risk for developing psychosis (n = 30), and healthy volunteers (n = 39) performed a reinforcement learning task during fMRI scanning. ANOVA revealed significant (p < 0.05 family-wise error corrected) prediction-error signalling differences between groups in the dopaminergic midbrain and right middle frontal gyrus (dorsolateral prefrontal cortex, DLPFC). FEP patients showed disrupted reward prediction-error signalling compared to controls in both regions. At-risk patients showed intermediate activation in the midbrain that significantly differed from controls and from FEP patients, but DLPFC activation that did not differ from controls. Our study confirms that FEP patients have abnormal meso-cortical signalling of reward-prediction errors, whereas reward-prediction-error dysfunction in the at-risk patients appears to show a more nuanced pattern of activation with a degree of midbrain impairment but preserved cortical function.
Collapse
|
85
|
Iigaya K, Fonseca MS, Murakami M, Mainen ZF, Dayan P. An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals. Nat Commun 2018; 9:2477. [PMID: 29946069 PMCID: PMC6018802 DOI: 10.1038/s41467-018-04840-2] [Citation(s) in RCA: 49] [Impact Index Per Article: 8.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2017] [Accepted: 05/22/2018] [Indexed: 12/02/2022] Open
Abstract
Serotonin has widespread, but computationally obscure, modulatory effects on learning and cognition. Here, we studied the impact of optogenetic stimulation of dorsal raphe serotonin neurons in mice performing a non-stationary, reward-driven decision-making task. Animals showed two distinct choice strategies. Choices after short inter-trial-intervals (ITIs) depended only on the last trial outcome and followed a win-stay-lose-switch pattern. In contrast, choices after long ITIs reflected outcome history over multiple trials, as described by reinforcement learning models. We found that optogenetic stimulation during a trial significantly boosted the rate of learning that occurred due to the outcome of that trial, but these effects were only exhibited on choices after long ITIs. This suggests that serotonin neurons modulate reinforcement learning rates, and that this influence is masked by alternate, unaffected, decision mechanisms. These results provide insight into the role of serotonin in treating psychiatric disorders, particularly its modulation of neural plasticity and learning. Serotonin (5-HT) plays many important roles in reward, punishment, patience and beyond, and optogenetic stimulation of 5-HT neurons has not crisply parsed them. The authors report a novel analysis of a reward-based decision-making experiment, and show that 5-HT stimulation increases the learning rate, but only on a select subset of choices.
Collapse
Affiliation(s)
- Kiyohito Iigaya
- Gatsby Computational Neuroscience Unit, University College London, 25 Howland Street, London, W1T 4JG, UK. .,Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Russell Square House, 10-12 Russell Square, London, WC1B 5EH, UK. .,Division of Humanities and Social Sciences, California Institute of Technology, 1200 E California Blvd, Pasadena, CA, 91125, USA.
| | - Madalena S Fonseca
- Champalimaud Research, Champalimaud Centre for the Unknown, Avenida Brasília, 1400-038, Lisbon, Portugal
| | - Masayoshi Murakami
- Champalimaud Research, Champalimaud Centre for the Unknown, Avenida Brasília, 1400-038, Lisbon, Portugal
| | - Zachary F Mainen
- Champalimaud Research, Champalimaud Centre for the Unknown, Avenida Brasília, 1400-038, Lisbon, Portugal
| | - Peter Dayan
- Gatsby Computational Neuroscience Unit, University College London, 25 Howland Street, London, W1T 4JG, UK.,Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Russell Square House, 10-12 Russell Square, London, WC1B 5EH, UK
| |
Collapse
|
86
|
Hassall CD, Connor PC, Trappenberg TP, McDonald JJ, Krigolson OE. Learning what matters: A neural explanation for the sparsity bias. Int J Psychophysiol 2018; 127:62-72. [PMID: 29551656 DOI: 10.1016/j.ijpsycho.2018.03.006] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2017] [Revised: 03/06/2018] [Accepted: 03/09/2018] [Indexed: 10/17/2022]
Abstract
The visual environment is filled with complex, multi-dimensional objects that vary in their value to an observer's current goals. When faced with multi-dimensional stimuli, humans may rely on biases to learn to select those objects that are most valuable to the task at hand. Here, we show that decision making in a complex task is guided by the sparsity bias: the focusing of attention on a subset of available features. Participants completed a gambling task in which they selected complex stimuli that varied randomly along three dimensions: shape, color, and texture. Each dimension comprised three features (e.g., color: red, green, yellow). Only one dimension was relevant in each block (e.g., color), and a randomly-chosen value ranking determined outcome probabilities (e.g., green > yellow > red). Participants were faster to respond to infrequent probe stimuli that appeared unexpectedly within stimuli that possessed a more valuable feature than to probes appearing within stimuli possessing a less valuable feature. Event-related brain potentials recorded during the task provided a neurophysiological explanation for sparsity as a learning-dependent increase in optimal attentional performance (as measured by the N2pc component of the human event-related potential) and a concomitant learning-dependent decrease in prediction errors (as measured by the feedback-elicited reward positivity). Together, our results suggest that the sparsity bias guides human reinforcement learning in complex environments.
Collapse
Affiliation(s)
- Cameron D Hassall
- Centre for Biomedical Research, University of Victoria, Victoria, British Columbia V8W 2Y2, Canada.
| | - Patrick C Connor
- Faculty of Computer Science, Dalhousie University, Halifax, Nova Scotia B3H 4R2, Canada
| | - Thomas P Trappenberg
- Faculty of Computer Science, Dalhousie University, Halifax, Nova Scotia B3H 4R2, Canada
| | - John J McDonald
- Department of Psychology, Simon Fraser University, Vancouver, British Columbia V5A 1S6, Canada
| | - Olave E Krigolson
- Centre for Biomedical Research, University of Victoria, Victoria, British Columbia V8W 2Y2, Canada
| |
Collapse
|
87
|
|
88
|
Stimuli that signal the availability of reward break into attentional focus. Vision Res 2018; 144:20-28. [DOI: 10.1016/j.visres.2017.10.008] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2017] [Revised: 09/28/2017] [Accepted: 10/11/2017] [Indexed: 11/19/2022]
|
89
|
Zhang S, Mano H, Lee M, Yoshida W, Kawato M, Robbins TW, Seymour B. The control of tonic pain by active relief learning. eLife 2018; 7:31949. [PMID: 29482716 PMCID: PMC5843408 DOI: 10.7554/elife.31949] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2017] [Accepted: 02/08/2018] [Indexed: 01/04/2023] Open
Abstract
Tonic pain after injury characterises a behavioural state that prioritises recovery. Although generally suppressing cognition and attention, tonic pain needs to allow effective relief learning to reduce the cause of the pain. Here, we describe a central learning circuit that supports learning of relief and concurrently suppresses the level of ongoing pain. We used computational modelling of behavioural, physiological and neuroimaging data in two experiments in which subjects learned to terminate tonic pain in static and dynamic escape-learning paradigms. In both studies, we show that active relief-seeking involves a reinforcement learning process manifest by error signals observed in the dorsal putamen. Critically, this system uses an uncertainty ('associability') signal detected in pregenual anterior cingulate cortex that both controls the relief learning rate, and endogenously and parametrically modulates the level of tonic pain. The results define a self-organising learning circuit that reduces ongoing pain when learning about potential relief.
Collapse
Affiliation(s)
- Suyi Zhang
- Computational and Biological Learning Laboratory, Department of Engineering, University of Cambridge, Cambridge, United Kingdom.,Brain Information Communication Research Laboratory Group, Advanced Telecommunications Research Institute International, Kyoto, Japan
| | - Hiroaki Mano
- Computational and Biological Learning Laboratory, Department of Engineering, University of Cambridge, Cambridge, United Kingdom.,Brain Information Communication Research Laboratory Group, Advanced Telecommunications Research Institute International, Kyoto, Japan.,Center for Information and Neural Networks, National Institute for Information and Communications Technology, Osaka, Japan
| | - Michael Lee
- Division of Anaesthesia, University of Cambridge, Cambridge, United Kingdom
| | - Wako Yoshida
- Brain Information Communication Research Laboratory Group, Advanced Telecommunications Research Institute International, Kyoto, Japan
| | - Mitsuo Kawato
- Brain Information Communication Research Laboratory Group, Advanced Telecommunications Research Institute International, Kyoto, Japan
| | - Trevor W Robbins
- Behavioural and Clinical Neuroscience Institute, Department of Psychology, University of Cambridge, Cambridge, United Kingdom
| | - Ben Seymour
- Computational and Biological Learning Laboratory, Department of Engineering, University of Cambridge, Cambridge, United Kingdom.,Brain Information Communication Research Laboratory Group, Advanced Telecommunications Research Institute International, Kyoto, Japan.,Center for Information and Neural Networks, National Institute for Information and Communications Technology, Osaka, Japan
| |
Collapse
|
90
|
Zommara NM, Takahashi M, Lauwereyns J. Influence of multiple action-outcome associations on the transition dynamics toward an optimal choice in rats. Cogn Neurodyn 2018; 12:43-53. [PMID: 29435086 PMCID: PMC5801281 DOI: 10.1007/s11571-017-9458-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2017] [Revised: 10/05/2017] [Accepted: 10/11/2017] [Indexed: 10/18/2022] Open
Abstract
When faced with familiar versus novel options, animals may exploit the acquired action-outcome associations or attempt to form new associations. Little is known about which factors determine the strategy of choice behavior in partially comprehended environments. Here we examine the influence of multiple action-outcome associations on choice behavior in the context of rewarding outcomes (food) and aversive outcomes (electric foot-shock). We used a nose-poke paradigm with rats, incorporating a dilemma between a familiar option and a novel, higher-value option. In Experiment 1, two groups of rats were trained with different outcome schedules: either a single action-outcome association ("Reward-Only") or dual action-outcome associations ("Reward-Shock"; with the added opportunity to avoid an electric foot-shock). In Experiment 2, we employed the same paradigm with two groups of rats performing the task under dual action-outcome associations, with different levels of threat (a low- or high-amplitude electric foot-shock). The choice behavior was clearly influenced by the action-outcome associations, with more efficient transition dynamics to the optimal choice with dual rather than single action-outcome associations. The level of threat did not affect the transition dynamics. Taken together, the data suggested that the strategy of choice behavior was modulated by the information complexity of the environment.
Collapse
Affiliation(s)
- Noha Mohsen Zommara
- Graduate School of Systems Life Sciences, Kyushu University, 3-1-1, Maidashi, Higashi-ku, Fukuoka, 812-8582 Japan
| | | | - Johan Lauwereyns
- Graduate School of Systems Life Sciences, Kyushu University, 3-1-1, Maidashi, Higashi-ku, Fukuoka, 812-8582 Japan
- Brain Science Institute, Tamagawa University, Machida, Japan
| |
Collapse
|
91
|
Brown VM, Zhu L, Wang JM, Frueh BC, King-Casas B, Chiu PH. Associability-modulated loss learning is increased in posttraumatic stress disorder. eLife 2018; 7:e30150. [PMID: 29313489 PMCID: PMC5760201 DOI: 10.7554/elife.30150] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2017] [Accepted: 12/02/2017] [Indexed: 11/30/2022] Open
Abstract
Disproportionate reactions to unexpected stimuli in the environment are a cardinal symptom of posttraumatic stress disorder (PTSD). Here, we test whether these heightened responses are associated with disruptions in distinct components of reinforcement learning. Specifically, using functional neuroimaging, a loss-learning task, and a computational model-based approach, we assessed the mechanistic hypothesis that overreactions to stimuli in PTSD arise from anomalous gating of attention during learning (i.e., associability). Behavioral choices of combat-deployed veterans with and without PTSD were fit to a reinforcement learning model, generating trial-by-trial prediction errors (signaling unexpected outcomes) and associability values (signaling attention allocation to the unexpected outcomes). Neural substrates of associability value and behavioral parameter estimates of associability updating, but not prediction error, increased with PTSD during loss learning. Moreover, the interaction of PTSD severity with neural markers of associability value predicted behavioral choices. These results indicate that increased attention-based learning may underlie aspects of PTSD and suggest potential neuromechanistic treatment targets.
Collapse
Affiliation(s)
- Vanessa M Brown
- Virginia Tech Carilion Research InstituteRoanokeUnited States
- Department of PsychologyVirginia TechBlacksburgUnited States
| | - Lusha Zhu
- Virginia Tech Carilion Research InstituteRoanokeUnited States
- School of Psychological and Cognitive SciencesIDG/McGovern Institute for Brain Research, Beijing Key Laboratory of Behavior and Mental Health, Peking–Tsinghua Center for Life Sciences, Peking UniversityBeijingChina
| | - John M Wang
- Virginia Tech Carilion Research InstituteRoanokeUnited States
- Department of PsychologyVirginia TechBlacksburgUnited States
| | | | - Brooks King-Casas
- Virginia Tech Carilion Research InstituteRoanokeUnited States
- Department of PsychologyVirginia TechBlacksburgUnited States
- Salem Veterans Affairs Medical CenterSalemUnited States
- School of Biomedical Engineering and SciencesVirginia Tech-Wake Forest UniversityBlacksburgUnited States
| | - Pearl H Chiu
- Virginia Tech Carilion Research InstituteRoanokeUnited States
- Department of PsychologyVirginia TechBlacksburgUnited States
- Salem Veterans Affairs Medical CenterSalemUnited States
| |
Collapse
|
92
|
Computational Phenotypes Revealed by Interactive Economic Games. COMPUTATIONAL PSYCHIATRY 2018. [DOI: 10.1016/b978-0-12-809825-7.00011-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/08/2023]
|
93
|
Jones EJH, Dawson G, Webb SJ. Sensory hypersensitivity predicts enhanced attention capture by faces in the early development of ASD. Dev Cogn Neurosci 2018; 29:11-20. [PMID: 28457745 PMCID: PMC5638681 DOI: 10.1016/j.dcn.2017.04.001] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2016] [Revised: 01/04/2017] [Accepted: 04/03/2017] [Indexed: 11/29/2022] Open
Abstract
Sensory sensitivity is prevalent among young children with ASD, but its relation to social communication impairment is unclear. Recently, increased sensory hypersensitivity has been linked to greater activity of the neural salience network (Green et al., 2016). Increased neural sensitivity to stimuli, especially social stimuli, could provide greater opportunity for social learning and improved outcomes. Consistent with this framework, in Experiment 1 we found that parent report of greater sensory hypersensitivity at 2 years in toddlers with ASD (N=27) was predictive of increased neural responsiveness to social stimuli (larger amplitude event-related potential/ERP responses to faces at P1, P400 and Nc) at 4 years, and this in turn was related to parent report of increased social approach at 4 years. In Experiment 2, parent report of increased perceptual sensitivity at 6 months in infants at low and high familial risk for ASD (N=35) predicted larger ERP P1 amplitude to faces at 18 months. Increased sensory hypersensitivity in early development thus predicted greater attention capture by faces in later development, and this related to more optimal social behavioral development. Sensory hypersensitivity may index a child's ability to benefit from supportive environments during development. Early sensory symptoms may not always be developmentally problematic for individuals with ASD.
Collapse
Affiliation(s)
- E J H Jones
- Centre for Brain and Cognitive Development, Department of Psychological Sciences, Birkbeck College, University of London, UK.
| | - G Dawson
- Department of Psychiatry and Behavioral Sciences, Duke University, Durham, NC, United States; Duke Center for Autism and Brain Development, Duke University, Durham, NC, United States; Duke Institute for Brain Sciences, Durham, NC, United States
| | - S J Webb
- Center on Human Development and Disability, University of Washington, Seattle, WA, United States; Department of Psychiatry & Behavioral Science, University of Washington, Seattle, WA, United States; Center on Child Behavior and Development, Seattle Children's Research Institute, Seattle, WA, United States
| |
Collapse
|
94
|
|
95
|
Abstract
The hypothesis that the phasic dopamine response reports a reward prediction error has become deeply entrenched. However, dopamine neurons exhibit several notable deviations from this hypothesis. A coherent explanation for these deviations can be obtained by analyzing the dopamine response in terms of Bayesian reinforcement learning. The key idea is that prediction errors are modulated by probabilistic beliefs about the relationship between cues and outcomes, updated through Bayesian inference. This account can explain dopamine responses to inferred value in sensory preconditioning, the effects of cue preexposure (latent inhibition), and adaptive coding of prediction errors when rewards vary across orders of magnitude. We further postulate that orbitofrontal cortex transforms the stimulus representation through recurrent dynamics, such that a simple error-driven learning rule operating on the transformed representation can implement the Bayesian reinforcement learning update.
Collapse
Affiliation(s)
- Samuel J. Gershman
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, MA 02138, U.S.A
| |
Collapse
|
96
|
Colas JT, Lu J. Learning Where to Look for High Value Improves Decision Making Asymmetrically. Front Psychol 2017; 8:2000. [PMID: 29187831 PMCID: PMC5695242 DOI: 10.3389/fpsyg.2017.02000] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2017] [Accepted: 10/31/2017] [Indexed: 01/22/2023] Open
Abstract
Decision making in any brain is imperfect and costly in terms of time and energy. Operating under such constraints, an organism could be in a position to improve performance if an opportunity arose to exploit informative patterns in the environment being searched. Such an improvement of performance could entail both faster and more accurate (i.e., reward-maximizing) decisions. The present study investigated the extent to which human participants could learn to take advantage of immediate patterns in the spatial arrangement of serially presented foods such that a region of space would consistently be associated with greater subjective value. Eye movements leading up to choices demonstrated rapidly induced biases in the selective allocation of visual fixation and attention that were accompanied by both faster and more accurate choices of desired goods as implicit learning occurred. However, for the control condition with its spatially balanced reward environment, these subjects exhibited preexisting lateralized biases for eye and hand movements (i.e., leftward and rightward, respectively) that could act in opposition not only to each other but also to the orienting biases elicited by the experimental manipulation, producing an asymmetry between the left and right hemifields with respect to performance. Potentially owing at least in part to learned cultural conventions (e.g., reading from left to right), the findings herein particularly revealed an intrinsic leftward bias underlying initial saccades in the midst of more immediate feedback-directed processes for which spatial biases can be learned flexibly to optimize oculomotor and manual control in value-based decision making. The present study thus replicates general findings of learned attentional biases in a novel context with inherently rewarding stimuli and goes on to further elucidate the interactions between endogenous and exogenous biases.
Collapse
Affiliation(s)
- Jaron T. Colas
- Computation and Neural Systems Program, California Institute of Technology, Pasadena, CA, United States
| | - Joy Lu
- Computation and Neural Systems Program, California Institute of Technology, Pasadena, CA, United States
- The Wharton School, University of Pennsylvania, Philadelphia, PA, United States
| |
Collapse
|
97
|
Oemisch M, Watson MR, Womelsdorf T, Schubö A. Changes of Attention during Value-Based Reversal Learning Are Tracked by N2pc and Feedback-Related Negativity. Front Hum Neurosci 2017; 11:540. [PMID: 29163113 PMCID: PMC5681986 DOI: 10.3389/fnhum.2017.00540] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2017] [Accepted: 10/24/2017] [Indexed: 01/28/2023] Open
Abstract
Previously learned reward values can have a pronounced impact, behaviorally and neurophysiologically, on the allocation of selective attention. All else constant, stimuli previously associated with a high value gain stronger attentional prioritization than stimuli previously associated with a low value. The N2pc, an ERP component indicative of attentional target selection, has been shown to reflect aspects of this prioritization, by changes of mean amplitudes closely corresponding to selective enhancement of high value target processing and suppression of high value distractor processing. What has remained unclear so far is whether the N2pc also reflects the flexible and repeated behavioral adjustments needed in a volatile task environment, in which the values of stimuli are reversed often and unannounced. Using a value-based reversal learning task, we found evidence that the N2pc amplitude flexibly and reversibly tracks value-based choices during the learning of reward associated stimulus colors. Specifically, successful learning of current value-contingencies was associated with reduced N2pc amplitudes, and this effect was more apparent for distractor processing, compared with target processing. In addition, following a value reversal the feedback related negativity(FRN), an ERP component that reflects feedback processing, was amplified and co-occurred with increased N2pc amplitudes in trials following low-value feedback. Importantly, participants that showed the greatest adjustment in N2pc amplitudes based on feedback were also the most efficient learners. These results allow further insight into how changes in attentional prioritization in an uncertain and volatile environment support flexible adjustments of behavior.
Collapse
Affiliation(s)
- Mariann Oemisch
- Department of Biology, Centre for Vision Research, York University, Toronto, ON, Canada
| | - Marcus R Watson
- Department of Biology, Centre for Vision Research, York University, Toronto, ON, Canada
| | - Thilo Womelsdorf
- Department of Biology, Centre for Vision Research, York University, Toronto, ON, Canada.,Department of Psychology, Vanderbilt University, Nashville, TN, United States
| | - Anna Schubö
- Department of Psychology, Philipps-University Marburg, Marburg, Germany
| |
Collapse
|
98
|
Stimulus preexposure speeds or slows subsequent acquisition of associative learning depending on learning test procedures and response measure. Learn Behav 2017; 46:134-156. [PMID: 29052113 DOI: 10.3758/s13420-017-0297-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Prior exposure to a conditioned stimulus (CS) typically results in latent inhibition-slower acquisition of associative learning about that stimulus in subsequent training. Here, we found that CS preexposure had different effects on the appetitive conditioning of rats with a sucrose unconditioned stimulus (US) depending on training test procedures, the similarity of preexposure and training procedures, and the choice of response measure. Preexposure to a visual or an auditory stimulus produced facilitation of acquisition of food-cup-directed responding when both of those cues were (separately) paired with sucrose delivery in the training test (Experiments 1 and 3). By contrast, the same preexposure procedure resulted in latent inhibition of food-cup learning if the second stimulus in the test phase was of the same modality as the preexposed stimulus (Experiment 2). In Experiment 3, latent inhibition was enhanced if both phases included a single CS or both phases included both auditory and visual CSs, compared to treatments in which only one CS was presented in one phase but two CSs were presented in the other phase. In Experiment 4, preexposure of an auditory cue slowed subsequent learning about it if the context was salient but enhanced learning if the context was of weaker salience. Finally, a measure of general activity revealed latent inhibition after preexposure in all conditions in all 4 experiments. We discuss the results within several classes of latent inhibition theories, none of which provides a comprehensive account.
Collapse
|
99
|
Yousefi A, Dougherty DD, Eskandar EN, Widge AS, Eden UT. Estimating Dynamic Signals From Trial Data With Censored Values. COMPUTATIONAL PSYCHIATRY (CAMBRIDGE, MASS.) 2017; 1:58-81. [PMID: 29601047 PMCID: PMC5774187 DOI: 10.1162/cpsy_a_00003] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/03/2016] [Accepted: 04/05/2017] [Indexed: 11/12/2022]
Abstract
Censored data occur commonly in trial-structured behavioral experiments and many other forms of longitudinal data. They can lead to severe bias and reduction of statistical power in subsequent analyses. Principled approaches for dealing with censored data, such as data imputation and methods based on the complete data's likelihood, work well for estimating fixed features of statistical models but have not been extended to dynamic measures, such as serial estimates of an underlying latent variable over time. Here we propose an approach to the censored-data problem for dynamic behavioral signals. We developed a state-space modeling framework with a censored observation process at the trial timescale. We then developed a filter algorithm to compute the posterior distribution of the state process using the available data. We showed that special cases of this framework can incorporate the three most common approaches to censored observations: ignoring trials with censored data, imputing the censored data values, or using the full information available in the data likelihood. Finally, we derived a computationally efficient approximate Gaussian filter that is similar in structure to a Kalman filter, but that efficiently accounts for censored data. We compared the performances of these methods in a simulation study and provide recommendations of approaches to use, based on the expected amount of censored data in an experiment. These new techniques can broadly be applied in many research domains in which censored data interfere with estimation, including survival analysis and other clinical trial applications.
Collapse
Affiliation(s)
- Ali Yousefi
- Department of Neurological Surgery, Massachusetts General Hospital and Harvard Medical School, Boston, MA
- Department of Mathematics and Statistics, Boston University, Boston, MA
| | - Darin D. Dougherty
- Department of Psychiatry, Massachusetts General Hospital and Harvard Medical School, Boston, MA
| | - Emad N. Eskandar
- Department of Neurological Surgery, Massachusetts General Hospital and Harvard Medical School, Boston, MA
| | - Alik S. Widge
- Department of Psychiatry, Massachusetts General Hospital and Harvard Medical School, Boston, MA
- Picower Institute for Learning & Memory, Massachusetts Institute of Technology, Cambridge, MA
| | - Uri T. Eden
- Department of Mathematics and Statistics, Boston University, Boston, MA
| |
Collapse
|
100
|
van Schalkwyk GI, Volkmar FR, Corlett PR. A Predictive Coding Account of Psychotic Symptoms in Autism Spectrum Disorder. J Autism Dev Disord 2017; 47:1323-1340. [PMID: 28185044 DOI: 10.1007/s10803-017-3065-9] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
The co-occurrence of psychotic and autism spectrum disorder (ASD) symptoms represents an important clinical challenge. Here we consider this problem in the context of a computational psychiatry approach that has been applied to both conditions-predictive coding. Some symptoms of schizophrenia have been explained in terms of a failure of top-down predictions or an enhanced weighting of bottom-up prediction errors. Likewise, autism has been explained in terms of similar perturbations. We suggest that this theoretical overlap may explain overlapping symptomatology. Experimental evidence highlights meaningful distinctions and consistencies between these disorders. We hypothesize individuals with ASD may experience some degree of delusions without the presence of any additional impairment, but that hallucinations are likely indicative of a distinct process.
Collapse
Affiliation(s)
| | - Fred R Volkmar
- Child Study Center, Yale University School of Medicine, New Haven, CT, USA
| | - Philip R Corlett
- Department of Psychiatry, Clinical Neuroscience Research Unit, Connecticut Mental Health Center, Yale University School of Medicine, 230 South Frontage Road, New Haven, CT, 06510, USA
| |
Collapse
|