1
|
Dercksen TT, Widmann A, Noesselt T, Wetzel N. Somatosensory omissions reveal action-related predictive processing. Hum Brain Mapp 2024; 45:e26550. [PMID: 38050773 PMCID: PMC10915725 DOI: 10.1002/hbm.26550] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Revised: 11/07/2023] [Accepted: 11/16/2023] [Indexed: 12/06/2023] Open
Abstract
The intricate relation between action and somatosensory perception has been studied extensively in the past decades. Generally, a forward model is thought to predict the somatosensory consequences of an action. These models propose that when an action is reliably coupled to a tactile stimulus, unexpected absence of the stimulus should elicit prediction error. Although such omission responses have been demonstrated in the auditory modality, it remains unknown whether this mechanism generalizes across modalities. This study therefore aimed to record action-induced somatosensory omission responses using EEG in humans. Self-paced button presses were coupled to somatosensory stimuli in 88% of trials, allowing a prediction, or in 50% of trials, not allowing a prediction. In the 88% condition, stimulus omission resulted in a neural response consisting of multiple components, as revealed by temporal principal component analysis. The oN1 response suggests similar sensory sources as stimulus-evoked activity, but an origin outside primary cortex. Subsequent oN2 and oP3 responses, as previously observed in the auditory domain, likely reflect modality-unspecific higher order processes. Together, findings straightforwardly demonstrate somatosensory predictions during action and provide evidence for a partially amodal mechanism of prediction error generation.
Collapse
Affiliation(s)
- Tjerk T. Dercksen
- Research Group Neurocognitive DevelopmentLeibniz Institute for NeurobiologyMagdeburgGermany
- Center for Behavioral Brain SciencesMagdeburgGermany
| | - Andreas Widmann
- Research Group Neurocognitive DevelopmentLeibniz Institute for NeurobiologyMagdeburgGermany
- Wilhelm Wundt Institute for PsychologyLeipzig UniversityLeipzigGermany
| | - Tömme Noesselt
- Center for Behavioral Brain SciencesMagdeburgGermany
- Department of Biological PsychologyOtto‐von‐Guericke‐University MagdeburgMagdeburgGermany
| | - Nicole Wetzel
- Research Group Neurocognitive DevelopmentLeibniz Institute for NeurobiologyMagdeburgGermany
- Center for Behavioral Brain SciencesMagdeburgGermany
- University of Applied Sciences Magdeburg‐StendalStendalGermany
| |
Collapse
|
2
|
Barry MLLR, Gerstner W. Fast adaptation to rule switching using neuronal surprise. PLoS Comput Biol 2024; 20:e1011839. [PMID: 38377112 PMCID: PMC10906910 DOI: 10.1371/journal.pcbi.1011839] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Revised: 03/01/2024] [Accepted: 01/18/2024] [Indexed: 02/22/2024] Open
Abstract
In humans and animals, surprise is a physiological reaction to an unexpected event, but how surprise can be linked to plausible models of neuronal activity is an open problem. We propose a self-supervised spiking neural network model where a surprise signal is extracted from an increase in neural activity after an imbalance of excitation and inhibition. The surprise signal modulates synaptic plasticity via a three-factor learning rule which increases plasticity at moments of surprise. The surprise signal remains small when transitions between sensory events follow a previously learned rule but increases immediately after rule switching. In a spiking network with several modules, previously learned rules are protected against overwriting, as long as the number of modules is larger than the total number of rules-making a step towards solving the stability-plasticity dilemma in neuroscience. Our model relates the subjective notion of surprise to specific predictions on the circuit level.
Collapse
Affiliation(s)
- Martin L. L. R. Barry
- School of Computer and Communication Sciences and School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Wulfram Gerstner
- School of Computer and Communication Sciences and School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| |
Collapse
|
3
|
Grundei M, Schmidt TT, Blankenburg F. A multimodal cortical network of sensory expectation violation revealed by fMRI. Hum Brain Mapp 2023; 44:5871-5891. [PMID: 37721377 PMCID: PMC10619418 DOI: 10.1002/hbm.26482] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Revised: 07/04/2023] [Accepted: 08/29/2023] [Indexed: 09/19/2023] Open
Abstract
The brain is subjected to multi-modal sensory information in an environment governed by statistical dependencies. Mismatch responses (MMRs), classically recorded with EEG, have provided valuable insights into the brain's processing of regularities and the generation of corresponding sensory predictions. Only few studies allow for comparisons of MMRs across multiple modalities in a simultaneous sensory stream and their corresponding cross-modal context sensitivity remains unknown. Here, we used a tri-modal version of the roving stimulus paradigm in fMRI to elicit MMRs in the auditory, somatosensory and visual modality. Participants (N = 29) were simultaneously presented with sequences of low and high intensity stimuli in each of the three senses while actively observing the tri-modal input stream and occasionally reporting the intensity of the previous stimulus in a prompted modality. The sequences were based on a probabilistic model, defining transition probabilities such that, for each modality, stimuli were more likely to repeat (p = .825) than change (p = .175) and stimulus intensities were equiprobable (p = .5). Moreover, each transition was conditional on the configuration of the other two modalities comprising global (cross-modal) predictive properties of the sequences. We identified a shared mismatch network of modality general inferior frontal and temporo-parietal areas as well as sensory areas, where the connectivity (psychophysiological interaction) between these regions was modulated during mismatch processing. Further, we found deviant responses within the network to be modulated by local stimulus repetition, which suggests highly comparable processing of expectation violation across modalities. Moreover, hierarchically higher regions of the mismatch network in the temporo-parietal area around the intraparietal sulcus were identified to signal cross-modal expectation violation. With the consistency of MMRs across audition, somatosensation and vision, our study provides insights into a shared cortical network of uni- and multi-modal expectation violation in response to sequence regularities.
Collapse
Affiliation(s)
- Miro Grundei
- Neurocomputation and Neuroimaging UnitFreie Universität BerlinBerlinGermany
- Berlin School of Mind and BrainHumboldt Universität zu BerlinBerlinGermany
| | | | - Felix Blankenburg
- Neurocomputation and Neuroimaging UnitFreie Universität BerlinBerlinGermany
- Berlin School of Mind and BrainHumboldt Universität zu BerlinBerlinGermany
| |
Collapse
|
4
|
Poublan-Couzardot A, Lecaignard F, Fucci E, Davidson RJ, Mattout J, Lutz A, Abdoun O. Time-resolved dynamic computational modeling of human EEG recordings reveals gradients of generative mechanisms for the MMN response. PLoS Comput Biol 2023; 19:e1010557. [PMID: 38091350 PMCID: PMC10752554 DOI: 10.1371/journal.pcbi.1010557] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Revised: 12/27/2023] [Accepted: 11/20/2023] [Indexed: 12/28/2023] Open
Abstract
Despite attempts to unify the different theoretical accounts of the mismatch negativity (MMN), there is still an ongoing debate on the neurophysiological mechanisms underlying this complex brain response. On one hand, neuronal adaptation to recurrent stimuli is able to explain many of the observed properties of the MMN, such as its sensitivity to controlled experimental parameters. On the other hand, several modeling studies reported evidence in favor of Bayesian learning models for explaining the trial-to-trial dynamics of the human MMN. However, direct comparisons of these two main hypotheses are scarce, and previous modeling studies suffered from methodological limitations. Based on reports indicating spatial and temporal dissociation of physiological mechanisms within the timecourse of mismatch responses in animals, we hypothesized that different computational models would best fit different temporal phases of the human MMN. Using electroencephalographic data from two independent studies of a simple auditory oddball task (n = 82), we compared adaptation and Bayesian learning models' ability to explain the sequential dynamics of auditory deviance detection in a time-resolved fashion. We first ran simulations to evaluate the capacity of our design to dissociate the tested models and found that they were sufficiently distinguishable above a certain level of signal-to-noise ratio (SNR). In subjects with a sufficient SNR, our time-resolved approach revealed a temporal dissociation between the two model families, with high evidence for adaptation during the early MMN window (from 90 to 150-190 ms post-stimulus depending on the dataset) and for Bayesian learning later in time (170-180 ms or 200-220ms). In addition, Bayesian model averaging of fixed-parameter models within the adaptation family revealed a gradient of adaptation rates, resembling the anatomical gradient in the auditory cortical hierarchy reported in animal studies.
Collapse
Affiliation(s)
- Arnaud Poublan-Couzardot
- Cente de Recherche en Neurosciences de Lyon (CRNL), CNRS UMRS5292, INSERM U1028, Université Claude Bernard Lyon 1, Bron, France
| | - Françoise Lecaignard
- Cente de Recherche en Neurosciences de Lyon (CRNL), CNRS UMRS5292, INSERM U1028, Université Claude Bernard Lyon 1, Bron, France
| | - Enrico Fucci
- 2 Institute for Globally Distributed Open Research and Education (IGDORE), Sweden
| | - Richard J. Davidson
- Center for Healthy Minds, University of Wisconsin, Madison, Wisconsin, United States of America
- Department of Psychology, University of Wisconsin, Madison, Wisconsin, United States of America
- Waisman Laboratory for Brain Imaging and Behavior, University of Wisconsin, Madison, Wisconsin, United States of America
- Department of Psychiatry, University of Wisconsin, Madison, Wisconsin, United States of America
| | - Jérémie Mattout
- Cente de Recherche en Neurosciences de Lyon (CRNL), CNRS UMRS5292, INSERM U1028, Université Claude Bernard Lyon 1, Bron, France
| | - Antoine Lutz
- Cente de Recherche en Neurosciences de Lyon (CRNL), CNRS UMRS5292, INSERM U1028, Université Claude Bernard Lyon 1, Bron, France
| | - Oussama Abdoun
- Cente de Recherche en Neurosciences de Lyon (CRNL), CNRS UMRS5292, INSERM U1028, Université Claude Bernard Lyon 1, Bron, France
| |
Collapse
|
5
|
Kumar M, Goldstein A, Michelmann S, Zacks JM, Hasson U, Norman KA. Bayesian Surprise Predicts Human Event Segmentation in Story Listening. Cogn Sci 2023; 47:e13343. [PMID: 37867379 DOI: 10.1111/cogs.13343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Revised: 08/28/2023] [Accepted: 09/01/2023] [Indexed: 10/24/2023]
Abstract
Event segmentation theory posits that people segment continuous experience into discrete events and that event boundaries occur when there are large transient increases in prediction error. Here, we set out to test this theory in the context of story listening, by using a deep learning language model (GPT-2) to compute the predicted probability distribution of the next word, at each point in the story. For three stories, we used the probability distributions generated by GPT-2 to compute the time series of prediction error. We also asked participants to listen to these stories while marking event boundaries. We used regression models to relate the GPT-2 measures to the human segmentation data. We found that event boundaries are associated with transient increases in Bayesian surprise but not with a simpler measure of prediction error (surprisal) that tracks, for each word in the story, how strongly that word was predicted at the previous time point. These results support the hypothesis that prediction error serves as a control mechanism governing event segmentation and point to important differences between operational definitions of prediction error.
Collapse
Affiliation(s)
- Manoj Kumar
- Princeton Neuroscience Institute, Princeton University
| | - Ariel Goldstein
- Department of Cognitive and Brain Sciences and Business School, Hebrew University
- Google Research, Tel-Aviv
| | | | - Jeffrey M Zacks
- Department of Psychological & Brain Sciences, Washington University in St. Louis
| | - Uri Hasson
- Princeton Neuroscience Institute, Princeton University
- Department of Psychology, Princeton University
| | - Kenneth A Norman
- Princeton Neuroscience Institute, Princeton University
- Department of Psychology, Princeton University
| |
Collapse
|
6
|
Modirshanechi A, Becker S, Brea J, Gerstner W. Surprise and novelty in the brain. Curr Opin Neurobiol 2023; 82:102758. [PMID: 37619425 DOI: 10.1016/j.conb.2023.102758] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Revised: 06/30/2023] [Accepted: 07/20/2023] [Indexed: 08/26/2023]
Abstract
Notions of surprise and novelty have been used in various experimental and theoretical studies across multiple brain areas and species. However, 'surprise' and 'novelty' refer to different quantities in different studies, which raises concerns about whether these studies indeed relate to the same functionalities and mechanisms in the brain. Here, we address these concerns through a systematic investigation of how different aspects of surprise and novelty relate to different brain functions and physiological signals. We review recent classifications of definitions proposed for surprise and novelty along with links to experimental observations. We show that computational modeling and quantifiable definitions enable novel interpretations of previous findings and form a foundation for future theoretical and experimental studies.
Collapse
Affiliation(s)
- Alireza Modirshanechi
- Brain-Mind Institute, School of Life Sciences, EPFL, Lausanne, Switzerland; School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland.
| | - Sophia Becker
- Brain-Mind Institute, School of Life Sciences, EPFL, Lausanne, Switzerland; School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland. https://twitter.com/sophiabecker95
| | - Johanni Brea
- Brain-Mind Institute, School of Life Sciences, EPFL, Lausanne, Switzerland; School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland
| | - Wulfram Gerstner
- Brain-Mind Institute, School of Life Sciences, EPFL, Lausanne, Switzerland; School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland.
| |
Collapse
|
7
|
Brændholt M, Kluger DS, Varga S, Heck DH, Gross J, Allen MG. Breathing in waves: Understanding respiratory-brain coupling as a gradient of predictive oscillations. Neurosci Biobehav Rev 2023; 152:105262. [PMID: 37271298 DOI: 10.1016/j.neubiorev.2023.105262] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2022] [Revised: 05/03/2023] [Accepted: 05/24/2023] [Indexed: 06/06/2023]
Abstract
Breathing plays a crucial role in shaping perceptual and cognitive processes by regulating the strength and synchronisation of neural oscillations. Numerous studies have demonstrated that respiratory rhythms govern a wide range of behavioural effects across cognitive, affective, and perceptual domains. Additionally, respiratory-modulated brain oscillations have been observed in various mammalian models and across diverse frequency spectra. However, a comprehensive framework to elucidate these disparate phenomena remains elusive. In this review, we synthesise existing findings to propose a neural gradient of respiratory-modulated brain oscillations and examine recent computational models of neural oscillations to map this gradient onto a hierarchical cascade of precision-weighted prediction errors. By deciphering the computational mechanisms underlying respiratory control of these processes, we can potentially uncover new pathways for understanding the link between respiratory-brain coupling and psychiatric disorders.
Collapse
Affiliation(s)
- Malthe Brændholt
- Center of Functionally Integrative Neuroscience, Aarhus University, Denmark
| | - Daniel S Kluger
- Institute for Biomagnetism and Biosignal Analysis, University of Münster, Germany.
| | - Somogy Varga
- School of Culture and Society, Aarhus University, Denmark; The Centre for Philosophy of Epidemiology, Medicine and Public Health, University of Johannesburg, South Africa
| | - Detlef H Heck
- Department of Biomedical Sciences, University of Minnesota Medical School, Duluth, MN
| | - Joachim Gross
- Institute for Biomagnetism and Biosignal Analysis, University of Münster, Germany; Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Germany
| | - Micah G Allen
- Center of Functionally Integrative Neuroscience, Aarhus University, Denmark; Cambridge Psychiatry, University of Cambridge, UK
| |
Collapse
|
8
|
Visalli A, Capizzi M, Ambrosini E, Kopp B, Vallesi A. P3-like signatures of temporal predictions: a computational EEG study. Exp Brain Res 2023:10.1007/s00221-023-06656-z. [PMID: 37354350 DOI: 10.1007/s00221-023-06656-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Accepted: 06/18/2023] [Indexed: 06/26/2023]
Abstract
Many cognitive processes, ranging from perception to action, depend on the ability to predict the timing of forthcoming events. Yet, how the brain uses predictive models in the temporal domain is still an unsolved question. In previous work, we began to explore the neural correlates of temporal predictions by using a computational approach in which an ideal Bayesian observer learned the temporal probabilities of target onsets in a simple reaction time task. Because the task was specifically designed to disambiguate updating of predictive models and surprise, changes in temporal probabilities were explicitly cued. However, in the real world, we are usually incidentally exposed to changes in the statistics of the environment. Here, we thus aimed to further investigate the electroencephalographic (EEG) correlates of Bayesian belief updating and surprise associated with incidental learning of temporal probabilities. In line with our previous EEG study, results showed distinct P3-like modulations for updating and surprise. While surprise was indexed by an early fronto-central P3-like modulation, updating was associated with a later and more posterior P3 modulation. Moreover, updating was associated with a P2-like potential at centro-parietal electrodes, likely capturing integration processes between prior beliefs and likelihood of the observed event. These findings support previous evidence of trial-by-trial variability of P3 amplitudes as an index of dissociable inferential processes. Coupled with our previous findings, the present study strongly bolsters the view of the P3 as a key brain signature of temporal Bayesian inference. Data and scripts are shared on OSF: osf.io/sdy8j/.
Collapse
Affiliation(s)
- Antonino Visalli
- Department of Neuroscience, University of Padova, 35121, Padua, Italy.
- Padova Neuroscience Center, University of Padova, Padua, Italy.
- IRCCS San Camillo Hospital, 30126, Venice, Italy.
| | - M Capizzi
- Brain and Behavior Research Center (CIMCYC), Department of Experimental Psychology, University of Granada, Granada, Spain
| | - E Ambrosini
- Department of Neuroscience, University of Padova, 35121, Padua, Italy
- Padova Neuroscience Center, University of Padova, Padua, Italy
- Department of General Psychology, University of Padova, Padua, Italy
| | - B Kopp
- Department of Neurology, Hannover Medical School, 30625, Hannover, Germany
| | - Antonino Vallesi
- Department of Neuroscience, University of Padova, 35121, Padua, Italy.
- Padova Neuroscience Center, University of Padova, Padua, Italy.
| |
Collapse
|
9
|
Semantic surprise predicts the N400 brain potential. NEUROIMAGE: REPORTS 2023. [DOI: 10.1016/j.ynirp.2023.100161] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/07/2023]
|
10
|
English G, Ghasemi Nejad N, Sommerfelt M, Yanik MF, von der Behrens W. Bayesian surprise shapes neural responses in somatosensory cortical circuits. Cell Rep 2023; 42:112009. [PMID: 36701237 DOI: 10.1016/j.celrep.2023.112009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Revised: 09/16/2022] [Accepted: 12/31/2022] [Indexed: 01/26/2023] Open
Abstract
Numerous psychophysical studies show that Bayesian inference governs sensory decision-making; however, the specific neural circuitry underlying this probabilistic mechanism remains unclear. We record extracellular neural activity along the somatosensory pathway of mice while delivering sensory stimulation paradigms designed to isolate the response to the surprise generated by Bayesian inference. Our results demonstrate that laminar cortical circuits in early sensory areas encode Bayesian surprise. Systematic sensitivity to surprise is not identified in the somatosensory thalamus, rather emerging in the primary (S1) and secondary (S2) somatosensory cortices. Multiunit spiking activity and evoked potentials in layer 6 of these regions exhibit the highest sensitivity to surprise. Gamma power in S1 layer 2/3 exhibits an NMDAR-dependent scaling with surprise, as does alpha power in layers 2/3 and 6 of S2. These results show a precise spatiotemporal neural representation of Bayesian surprise and suggest that Bayesian inference is a fundamental component of cortical processing.
Collapse
Affiliation(s)
- Gwendolyn English
- Institute of Neuroinformatics, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland; ZNZ Neuroscience Center Zurich, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland.
| | - Newsha Ghasemi Nejad
- Institute of Neuroinformatics, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland; ZNZ Neuroscience Center Zurich, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland
| | - Marcel Sommerfelt
- Institute of Neuroinformatics, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland
| | - Mehmet Fatih Yanik
- Institute of Neuroinformatics, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland; ZNZ Neuroscience Center Zurich, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland
| | - Wolfger von der Behrens
- Institute of Neuroinformatics, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland; ZNZ Neuroscience Center Zurich, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland.
| |
Collapse
|
11
|
Gijsen S, Grundei M, Blankenburg F. Active inference and the two-step task. Sci Rep 2022; 12:17682. [PMID: 36271279 PMCID: PMC9586964 DOI: 10.1038/s41598-022-21766-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Accepted: 09/30/2022] [Indexed: 01/18/2023] Open
Abstract
Sequential decision problems distill important challenges frequently faced by humans. Through repeated interactions with an uncertain world, unknown statistics need to be learned while balancing exploration and exploitation. Reinforcement learning is a prominent method for modeling such behaviour, with a prevalent application being the two-step task. However, recent studies indicate that the standard reinforcement learning model sometimes describes features of human task behaviour inaccurately and incompletely. We investigated whether active inference, a framework proposing a trade-off to the exploration-exploitation dilemma, could better describe human behaviour. Therefore, we re-analysed four publicly available datasets of the two-step task, performed Bayesian model selection, and compared behavioural model predictions. Two datasets, which revealed more model-based inference and behaviour indicative of directed exploration, were better described by active inference, while the models scored similarly for the remaining datasets. Learning using probability distributions appears to contribute to the improved model fits. Further, approximately half of all participants showed sensitivity to information gain as formulated under active inference, although behavioural exploration effects were not fully captured. These results contribute to the empirical validation of active inference as a model of human behaviour and the study of alternative models for the influential two-step task.
Collapse
Affiliation(s)
- Sam Gijsen
- grid.14095.390000 0000 9116 4836Neurocomputation and Neuroimaging Unit, Freie Universität Berlin, 14195 Berlin, Germany ,grid.7468.d0000 0001 2248 7639Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, 10117 Berlin, Germany
| | - Miro Grundei
- grid.14095.390000 0000 9116 4836Neurocomputation and Neuroimaging Unit, Freie Universität Berlin, 14195 Berlin, Germany ,grid.7468.d0000 0001 2248 7639Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, 10117 Berlin, Germany
| | - Felix Blankenburg
- grid.14095.390000 0000 9116 4836Neurocomputation and Neuroimaging Unit, Freie Universität Berlin, 14195 Berlin, Germany ,grid.7468.d0000 0001 2248 7639Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, 10117 Berlin, Germany
| |
Collapse
|
12
|
Majumdar G, Yazin F, Banerjee A, Roy D. Emotion dynamics as hierarchical Bayesian inference in time. Cereb Cortex 2022; 33:3750-3772. [PMID: 36030379 DOI: 10.1093/cercor/bhac305] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2022] [Revised: 07/06/2022] [Accepted: 07/07/2022] [Indexed: 11/13/2022] Open
Abstract
What fundamental property of our environment would be most valuable and optimal in characterizing the emotional dynamics we experience in daily life? Empirical work has shown that an accurate estimation of uncertainty is necessary for our optimal perception, learning, and decision-making. However, the role of this uncertainty in governing our affective dynamics remains unexplored. Using Bayesian encoding, decoding and computational modeling, on a large-scale neuroimaging and behavioral data on a passive movie-watching task, we showed that emotions naturally arise due to ongoing uncertainty estimations about future outcomes in a hierarchical neural architecture. Several prefrontal subregions hierarchically encoded a lower-dimensional signal that highly correlated with the evolving uncertainty. Crucially, the lateral orbitofrontal cortex (lOFC) tracked the temporal fluctuations of this uncertainty and was predictive of the participants' predisposition to anxiety. Furthermore, we observed a distinct functional double-dissociation within OFC with increased connectivity between medial OFC and DMN, while with that of lOFC and FPN in response to the evolving affect. Finally, we uncovered a temporally predictive code updating an individual's beliefs spontaneously with fluctuating outcome uncertainty in the lOFC. A biologically relevant and computationally crucial parameter in the theories of brain function, we propose uncertainty to be central to the definition of complex emotions.
Collapse
Affiliation(s)
- Gargi Majumdar
- Cognitive Brain Dynamics Lab, National Brain Research Centre, NH 8, Manesar, Gurgaon, Haryana 122052, India
| | - Fahd Yazin
- Cognitive Brain Dynamics Lab, National Brain Research Centre, NH 8, Manesar, Gurgaon, Haryana 122052, India
| | - Arpan Banerjee
- Cognitive Brain Dynamics Lab, National Brain Research Centre, NH 8, Manesar, Gurgaon, Haryana 122052, India
| | - Dipanjan Roy
- Cognitive Brain Dynamics Lab, National Brain Research Centre, NH 8, Manesar, Gurgaon, Haryana 122052, India.,Centre for Brain Science and Applications, School of AIDE, IIT Jodhpur, NH 62, Surpura Bypass Rd, Karwar, Rajasthan 342030, India
| |
Collapse
|
13
|
Mousavi Z, Kiani MM, Aghajan H. Spatiotemporal Signatures of Surprise Captured by Magnetoencephalography. Front Syst Neurosci 2022; 16:865453. [PMID: 35770244 PMCID: PMC9235820 DOI: 10.3389/fnsys.2022.865453] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2022] [Accepted: 05/24/2022] [Indexed: 11/21/2022] Open
Abstract
Surprise and social influence are linked through several neuropsychological mechanisms. By garnering attention, causing arousal, and motivating engagement, surprise provides a context for effective or durable social influence. Attention to a surprising event motivates the formation of an explanation or updating of models, while high arousal experiences due to surprise promote memory formation. They both encourage engagement with the surprising event through efforts aimed at understanding the situation. By affecting the behavior of the individual or a social group via setting an attractive engagement context, surprise plays an important role in shaping personal and social change. Surprise is an outcome of the brain’s function in constantly anticipating the future of sensory inputs based on past experiences. When new sensory data is different from the brain’s predictions shaped by recent trends, distinct neural signals are generated to report this surprise. As a quantitative approach to modeling the generation of brain surprise, input stimuli containing surprising elements are employed in experiments such as oddball tasks during which brain activity is recorded. Although surprise has been well characterized in many studies, an information-theoretical model to describe and predict the surprise level of an external stimulus in the recorded MEG data has not been reported to date, and setting forth such a model is the main objective of this paper. Through mining trial-by-trial MEG data in an oddball task according to theoretical definitions of surprise, the proposed surprise decoding model employs the entire epoch of the brain response to a stimulus to measure surprise and assesses which collection of temporal/spatial components in the recorded data can provide optimal power for describing the brain’s surprise. We considered three different theoretical formulations for surprise assuming the brain acts as an ideal observer that calculates transition probabilities to estimate the generative distribution of the input. We found that middle temporal components and the right and left fronto-central regions offer the strongest power for decoding surprise. Our findings provide a practical and rigorous method for measuring the brain’s surprise, which can be employed in conjunction with behavioral data to evaluate the interactive and social effects of surprising events.
Collapse
|
14
|
Foucault C, Meyniel F. Gated recurrence enables simple and accurate sequence prediction in stochastic, changing, and structured environments. eLife 2021; 10:71801. [PMID: 34854377 PMCID: PMC8735865 DOI: 10.7554/elife.71801] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2021] [Accepted: 12/01/2021] [Indexed: 11/13/2022] Open
Abstract
From decision making to perception to language, predicting what is coming next is crucial. It is also challenging in stochastic, changing, and structured environments; yet the brain makes accurate predictions in many situations. What computational architecture could enable this feat? Bayesian inference makes optimal predictions but is prohibitively difficult to compute. Here, we show that a specific recurrent neural network architecture enables simple and accurate solutions in several environments. This architecture relies on three mechanisms: gating, lateral connections, and recurrent weight training. Like the optimal solution and the human brain, such networks develop internal representations of their changing environment (including estimates of the environment’s latent variables and the precision of these estimates), leverage multiple levels of latent structure, and adapt their effective learning rate to changes without changing their connection weights. Being ubiquitous in the brain, gated recurrence could therefore serve as a generic building block to predict in real-life environments.
Collapse
Affiliation(s)
- Cédric Foucault
- INSERM, CEA, Université Paris-Saclay, Gif sur Yvette, France
| | | |
Collapse
|
15
|
Xu HA, Modirshanechi A, Lehmann MP, Gerstner W, Herzog MH. Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making. PLoS Comput Biol 2021; 17:e1009070. [PMID: 34081705 PMCID: PMC8205159 DOI: 10.1371/journal.pcbi.1009070] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2021] [Revised: 06/15/2021] [Accepted: 05/12/2021] [Indexed: 11/19/2022] Open
Abstract
Classic reinforcement learning (RL) theories cannot explain human behavior in the absence of external reward or when the environment changes. Here, we employ a deep sequential decision-making paradigm with sparse reward and abrupt environmental changes. To explain the behavior of human participants in these environments, we show that RL theories need to include surprise and novelty, each with a distinct role. While novelty drives exploration before the first encounter of a reward, surprise increases the rate of learning of a world-model as well as of model-free action-values. Even though the world-model is available for model-based RL, we find that human decisions are dominated by model-free action choices. The world-model is only marginally used for planning, but it is important to detect surprising events. Our theory predicts human action choices with high probability and allows us to dissociate surprise, novelty, and reward in EEG signals.
Collapse
Affiliation(s)
- He A. Xu
- Laboratory of Psychophysics, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Alireza Modirshanechi
- Brain-Mind Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
- School of Computer and Communication Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Marco P. Lehmann
- Brain-Mind Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
- School of Computer and Communication Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Wulfram Gerstner
- Brain-Mind Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
- School of Computer and Communication Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| | - Michael H. Herzog
- Laboratory of Psychophysics, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
- Brain-Mind Institute, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
| |
Collapse
|
16
|
Human Belief State-Based Exploration and Exploitation in an Information-Selective Symmetric Reversal Bandit Task. COMPUTATIONAL BRAIN & BEHAVIOR 2021; 4:442-462. [PMID: 34368622 PMCID: PMC8327602 DOI: 10.1007/s42113-021-00112-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Accepted: 05/24/2021] [Indexed: 02/07/2023]
Abstract
Humans often face sequential decision-making problems, in which information about the environmental reward structure is detached from rewards for a subset of actions. In the current exploratory study, we introduce an information-selective symmetric reversal bandit task to model such situations and obtained choice data on this task from 24 participants. To arbitrate between different decision-making strategies that participants may use on this task, we developed a set of probabilistic agent-based behavioral models, including exploitative and explorative Bayesian agents, as well as heuristic control agents. Upon validating the model and parameter recovery properties of our model set and summarizing the participants' choice data in a descriptive way, we used a maximum likelihood approach to evaluate the participants' choice data from the perspective of our model set. In brief, we provide quantitative evidence that participants employ a belief state-based hybrid explorative-exploitative strategy on the information-selective symmetric reversal bandit task, lending further support to the finding that humans are guided by their subjective uncertainty when solving exploration-exploitation dilemmas. SUPPLEMENTARY INFORMATION The online version contains supplementary material available at 10.1007/s42113-021-00112-3.
Collapse
|