1
|
Nigam T, Schwiedrzik CM. Predictions enable top-down pattern separation in the macaque face-processing hierarchy. Nat Commun 2024; 15:7196. [PMID: 39169024 PMCID: PMC11339276 DOI: 10.1038/s41467-024-51543-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2023] [Accepted: 08/07/2024] [Indexed: 08/23/2024] Open
Abstract
Distinguishing faces requires well distinguishable neural activity patterns. Contextual information may separate neural representations, leading to enhanced identity recognition. Here, we use functional magnetic resonance imaging to investigate how predictions derived from contextual information affect the separability of neural activity patterns in the macaque face-processing system, a 3-level processing hierarchy in ventral visual cortex. We find that in the presence of predictions, early stages of this hierarchy exhibit well separable and high-dimensional neural geometries resembling those at the top of the hierarchy. This is accompanied by a systematic shift of tuning properties from higher to lower areas, endowing lower areas with higher-order, invariant representations instead of their feedforward tuning properties. Thus, top-down signals dynamically transform neural representations of faces into separable and high-dimensional neural geometries. Our results provide evidence how predictive context transforms flexible representational spaces to optimally use the computational resources provided by cortical processing hierarchies for better and faster distinction of facial identities.
Collapse
Affiliation(s)
- Tarana Nigam
- Neural Circuits and Cognition Lab, European Neuroscience Institute Göttingen - A Joint Initiative of the University Medical Center Göttingen and the Max Planck Institute for Multidisciplinary Sciences, Grisebachstraße 5, 37077, Göttingen, Germany
- Perception and Plasticity Group, German Primate Center - Leibniz Institute for Primate Research, Kellnerweg 4, 37077, Göttingen, Germany
- Leibniz ScienceCampus 'Primate Cognition', Göttingen, Germany
- International Max Planck Research School 'Neurosciences', Georg August University Göttingen, Grisebachstraße 5, 37077, Göttingen, Germany
| | - Caspar M Schwiedrzik
- Neural Circuits and Cognition Lab, European Neuroscience Institute Göttingen - A Joint Initiative of the University Medical Center Göttingen and the Max Planck Institute for Multidisciplinary Sciences, Grisebachstraße 5, 37077, Göttingen, Germany.
- Perception and Plasticity Group, German Primate Center - Leibniz Institute for Primate Research, Kellnerweg 4, 37077, Göttingen, Germany.
- Leibniz ScienceCampus 'Primate Cognition', Göttingen, Germany.
| |
Collapse
|
2
|
Nazlı İ, Ferrari A, Huber-Huber C, de Lange FP. Forward and backward blocking in statistical learning. PLoS One 2024; 19:e0306797. [PMID: 39102398 PMCID: PMC11299817 DOI: 10.1371/journal.pone.0306797] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2023] [Accepted: 06/24/2024] [Indexed: 08/07/2024] Open
Abstract
Prediction errors have a prominent role in many forms of learning. For example, in reinforcement learning, agents learn by updating the association between states and outcomes as a function of the prediction error elicited by the event. One paradigm often used to study error-driven learning is blocking. In forward blocking, participants are first presented with stimulus A, followed by outcome X (A→X). In the second phase, A and B are presented together, followed by X (AB→X). Here, A→X blocks the formation of B→X, given that X is already fully predicted by A. In backward blocking, the order of phases is reversed. Here, the association between B and X that is formed during the first learning phase of AB→X is weakened when participants learn exclusively A→X in the second phase. The present study asked the question whether forward and backward blocking occur during visual statistical learning, i.e., the incidental learning of the statistical structure of the environment. In a series of studies, using both forward and backward blocking, we observed statistical learning of temporal associations among pairs of images. While we found no forward blocking, we observed backward blocking, thereby suggesting a retrospective revaluation process in statistical learning and supporting a functional similarity between statistical learning and reinforcement learning.
Collapse
Affiliation(s)
- İlayda Nazlı
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| | - Ambra Ferrari
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
- Max Plank Institute for Psycholinguistics, Nijmegen, The Netherlands
| | - Christoph Huber-Huber
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
- Center for Mind/Brain Sciences (CIMeC), University of Trento, Rovereto, Italy
| | - Floris P. de Lange
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| |
Collapse
|
3
|
Feuerriegel D. Adaptation in the visual system: Networked fatigue or suppressed prediction error signalling? Cortex 2024; 177:302-320. [PMID: 38905873 DOI: 10.1016/j.cortex.2024.06.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2024] [Revised: 05/10/2024] [Accepted: 06/04/2024] [Indexed: 06/23/2024]
Abstract
Our brains are constantly adapting to changes in our visual environments. Neural adaptation exerts a persistent influence on the activity of sensory neurons and our perceptual experience, however there is a lack of consensus regarding how adaptation is implemented in the visual system. One account describes fatigue-based mechanisms embedded within local networks of stimulus-selective neurons (networked fatigue models). Another depicts adaptation as a product of stimulus expectations (predictive coding models). In this review, I evaluate neuroimaging and psychophysical evidence that poses fundamental problems for predictive coding models of neural adaptation. Specifically, I discuss observations of distinct repetition and expectation effects, as well as incorrect predictions of repulsive adaptation aftereffects made by predictive coding accounts. Based on this evidence, I argue that networked fatigue models provide a more parsimonious account of adaptation effects in the visual system. Although stimulus expectations can be formed based on recent stimulation history, any consequences of these expectations are likely to co-occur (or interact) with effects of fatigue-based adaptation. I conclude by proposing novel, testable hypotheses relating to interactions between fatigue-based adaptation and other predictive processes, focusing on stimulus feature extrapolation phenomena.
Collapse
Affiliation(s)
- Daniel Feuerriegel
- Melbourne School of Psychological Sciences, The University of Melbourne, Australia.
| |
Collapse
|
4
|
Sáringer S, Kaposvári P, Benyhe A. Visual linguistic statistical learning is traceable through neural entrainment. Psychophysiology 2024; 61:e14575. [PMID: 38549442 DOI: 10.1111/psyp.14575] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2023] [Revised: 02/22/2024] [Accepted: 03/17/2024] [Indexed: 07/07/2024]
Abstract
The human brain can detect statistical regularities in the environment across a wide variety of contexts. The importance of this process is well-established not just in language acquisition but across different modalities; in addition, several neural correlates of statistical learning have been identified. A current technique for tracking the emergence of regularity learning and localizing its neural background is frequency tagging (FT). FT can detect neural entrainment not only to the frequency of stimulus presentation but also to that of a hidden structure. Auditory learning paradigms with linguistic and nonlinguistic stimuli, along with a visual paradigm using nonlinguistic stimuli, have already been tested with FT. To complete the picture, we conducted an FT experiment using written syllables as stimuli and a hidden triplet structure. Both behavioral and neural entrainment data showed evidence of structure learning. In addition, we localized two electrode clusters related to the process, which spread across the frontal and parieto-occipital areas, similar to previous findings. Accordingly, we conclude that fast-paced visual linguistic regularities can be acquired and are traceable through neural entrainment. In comparison with the literature, our findings support the view that statistical learning involves a domain-general network.
Collapse
Affiliation(s)
- Szabolcs Sáringer
- Department of Physiology, Albert Szent-Györgyi Medical School, University of Szeged, Szeged, Hungary
| | - Péter Kaposvári
- Department of Physiology, Albert Szent-Györgyi Medical School, University of Szeged, Szeged, Hungary
| | - András Benyhe
- Department of Physiology, Albert Szent-Györgyi Medical School, University of Szeged, Szeged, Hungary
| |
Collapse
|
5
|
Yamane Y. Adaptation of the inferior temporal neurons and efficient visual processing. Front Behav Neurosci 2024; 18:1398874. [PMID: 39132448 PMCID: PMC11310006 DOI: 10.3389/fnbeh.2024.1398874] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2024] [Accepted: 07/16/2024] [Indexed: 08/13/2024] Open
Abstract
Numerous studies examining the responses of individual neurons in the inferior temporal (IT) cortex have revealed their characteristics such as two-dimensional or three-dimensional shape tuning, objects, or category selectivity. While these basic selectivities have been studied assuming that their response to stimuli is relatively stable, physiological experiments have revealed that the responsiveness of IT neurons also depends on visual experience. The activity changes of IT neurons occur over various time ranges; among these, repetition suppression (RS), in particular, is robustly observed in IT neurons without any behavioral or task constraints. I observed a similar phenomenon in the ventral visual neurons in macaque monkeys while they engaged in free viewing and actively fixated on one consistent object multiple times. This observation indicates that the phenomenon also occurs in natural situations during which the subject actively views stimuli without forced fixation, suggesting that this phenomenon is an everyday occurrence and widespread across regions of the visual system, making it a default process for visual neurons. Such short-term activity modulation may be a key to understanding the visual system; however, the circuit mechanism and the biological significance of RS remain unclear. Thus, in this review, I summarize the observed modulation types in IT neurons and the known properties of RS. Subsequently, I discuss adaptation in vision, including concepts such as efficient and predictive coding, as well as the relationship between adaptation and psychophysical aftereffects. Finally, I discuss some conceptual implications of this phenomenon as well as the circuit mechanisms and the models that may explain adaptation as a fundamental aspect of visual processing.
Collapse
Affiliation(s)
- Yukako Yamane
- Neural Computation Unit, Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan
| |
Collapse
|
6
|
Stan PL, Smith MA. Recent visual experience reshapes V4 neuronal activity and improves perceptual performance. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.08.27.555026. [PMID: 37693510 PMCID: PMC10491105 DOI: 10.1101/2023.08.27.555026] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/12/2023]
Abstract
Recent visual experience heavily influences our visual perception, but how this is mediated by the reshaping of neuronal activity to alter and improve perceptual discrimination remains unknown. We recorded from populations of neurons in visual cortical area V4 while monkeys performed a natural image change detection task under different experience conditions. We found that maximizing the recent experience with a particular image led to an improvement in the ability to detect a change in that image. This improvement was associated with decreased neural responses to the image, consistent with neuronal changes previously seen in studies of adaptation and expectation. We found that the magnitude of behavioral improvement was correlated with the magnitude of response suppression. Furthermore, this suppression of activity led to an increase in signal separation, providing evidence that a reduction in activity can improve stimulus encoding. Within populations of neurons, greater recent experience was associated with decreased trial-to-trial shared variability, indicating that a reduction in variability is a key means by which experience influences perception. Taken together, the results of our study contribute to an understanding of how recent visual experience can shape our perception and behavior through modulating activity patterns in mid-level visual cortex.
Collapse
|
7
|
Sáringer S, Fehér Á, Sáry G, Kaposvári P. Perceptual Expectations Are Reflected by Early Alpha Power Reduction. J Cogn Neurosci 2024; 36:1282-1296. [PMID: 38652100 DOI: 10.1162/jocn_a_02169] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/25/2024]
Abstract
The predictability of a stimulus can be characterized by its transitional probability. Perceptual expectations derived from the transitional probability of the stimulus were found to modulate the early alpha oscillations in the sensory regions of the brain when neural responses to expected versus unexpected stimuli were compared. The objective of our study was to find out the extent to which this low-frequency oscillation reflects stimulus predictability. We aimed to detect the alpha-power difference with smaller differences in transitional probabilities by comparing expected stimuli with neutral ones. We studied the effect of expectation on perception by applying an unsupervised visual statistical learning paradigm with expected and neutral stimuli embedded in an image sequence while recording EEG. Time-frequency analysis showed that expected stimuli elicit lower alpha power in the window of 8-12 Hz and 0-400 msec after stimulus presentation, appearing in the centroparietal region. Comparing previous findings of expectancy-based alpha-band modulation with our results suggests that early alpha oscillation shows an inverse relationship with stimulus predictability. Although current data are insufficient to determine the origin of the alpha power reduction, this could be a potential sign of expectation suppression in cortical oscillatory activity.
Collapse
|
8
|
De Schrijver S, Decramer T, Janssen P. Simple visual stimuli are sufficient to drive responses in action observation and execution neurons in macaque ventral premotor cortex. PLoS Biol 2024; 22:e3002358. [PMID: 38768251 PMCID: PMC11142659 DOI: 10.1371/journal.pbio.3002358] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2023] [Revised: 05/31/2024] [Accepted: 04/26/2024] [Indexed: 05/22/2024] Open
Abstract
Neurons responding during action execution and action observation were discovered in the ventral premotor cortex 3 decades ago. However, the visual features that drive the responses of action observation/execution neurons (AOENs) have not been revealed at present. We investigated the neural responses of AOENs in ventral premotor area F5c of 4 macaques during the observation of action videos and crucial control stimuli. The large majority of AOENs showed highly phasic responses during the action videos, with a preference for the moment that the hand made contact with the object. They also responded to an abstract shape moving towards but not interacting with an object, even when the shape moved on a scrambled background, implying that most AOENs in F5c do not require the perception of causality or a meaningful action. Additionally, the majority of AOENs responded to static frames of the videos. Our findings show that very elementary stimuli, even without a grasping context, are sufficient to drive responses in F5c AOENs.
Collapse
Affiliation(s)
- Sofie De Schrijver
- Laboratory for Neuro- and Psychophysiology, Department of Neurosciences, KU Leuven and the Leuven Brain Institute, Leuven, Belgium
| | - Thomas Decramer
- Research group Experimental Neurosurgery and Neuroanatomy, KU Leuven and the Leuven Brain Institute, Leuven, Belgium
| | - Peter Janssen
- Laboratory for Neuro- and Psychophysiology, Department of Neurosciences, KU Leuven and the Leuven Brain Institute, Leuven, Belgium
| |
Collapse
|
9
|
Walsh K, McGovern DP, Dully J, Kelly SP, O'Connell RG. Prior probability cues bias sensory encoding with increasing task exposure. eLife 2024; 12:RP91135. [PMID: 38564237 PMCID: PMC10987094 DOI: 10.7554/elife.91135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/04/2024] Open
Abstract
When observers have prior knowledge about the likely outcome of their perceptual decisions, they exhibit robust behavioural biases in reaction time and choice accuracy. Computational modelling typically attributes these effects to strategic adjustments in the criterion amount of evidence required to commit to a choice alternative - usually implemented by a starting point shift - but recent work suggests that expectations may also fundamentally bias the encoding of the sensory evidence itself. Here, we recorded neural activity with EEG while participants performed a contrast discrimination task with valid, invalid, or neutral probabilistic cues across multiple testing sessions. We measured sensory evidence encoding via contrast-dependent steady-state visual-evoked potentials (SSVEP), while a read-out of criterion adjustments was provided by effector-selective mu-beta band activity over motor cortex. In keeping with prior modelling and neural recording studies, cues evoked substantial biases in motor preparation consistent with criterion adjustments, but we additionally found that the cues produced a significant modulation of the SSVEP during evidence presentation. While motor preparation adjustments were observed in the earliest trials, the sensory-level effects only emerged with extended task exposure. Our results suggest that, in addition to strategic adjustments to the decision process, probabilistic information can also induce subtle biases in the encoding of the evidence itself.
Collapse
Affiliation(s)
- Kevin Walsh
- School of Psychological Sciences, Monash UniversityMelbourneAustralia
| | | | - Jessica Dully
- Institute of Psychiatry, Psychology & Neuroscience, King's College LondonLondonUnited Kingdom
| | - Simon P Kelly
- School of Electrical Engineering, University College DublinDublinIreland
- Trinity College Institute of Neuroscience, Trinity College DublinDublinIreland
| | - Redmond G O'Connell
- Trinity College Institute of Neuroscience, Trinity College DublinDublinIreland
- School of Psychology, Trinity College DublinDublinIreland
| |
Collapse
|
10
|
Esmailpour H, Vogels R. Location-specific deviant responses to object sequences in macaque inferior temporal cortex. Sci Rep 2024; 14:3757. [PMID: 38355712 PMCID: PMC10866936 DOI: 10.1038/s41598-024-54298-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Accepted: 02/11/2024] [Indexed: 02/16/2024] Open
Abstract
Many species learn temporal regularities in their visual environment, demonstrating visual statistical learning. In this study, we explored the sensitivity of macaque inferior temporal (IT) cortical neurons to transition probabilities of sequentially presented visual images, presented at different locations in the visual field. We exposed monkeys to sequences of two images, where the first image was presented either foveally or peripherally, and the second image was consistently presented foveally. Following several weeks of exposure, we recorded IT responses to assess differences between the exposed (Fixed) and new, Deviant sequences, where the identity of the first image in a sequence differed from the exposure phase. While enhanced responses to Deviant sequences were observed when both images of a pair were foveally presented during exposure, no such deviant responses were present when the first image was presented peripherally. This finding challenges the notion that mere exposure to image sequences always leads to deviant responses in macaque IT. The results highlight the complexity of the mechanisms underlying statistical learning in primates, particularly in the context of peripheral image presentations, emphasizing the need for further investigation into the origins of these responses in the IT cortex.
Collapse
Affiliation(s)
- Hamideh Esmailpour
- Laboratorium Voor Neuro- en Psychofysiologie, Department of Neurosciences, KU Leuven, Leuven, Belgium
- Leuven Brain Institute, KU Leuven, Leuven, Belgium
| | - Rufin Vogels
- Laboratorium Voor Neuro- en Psychofysiologie, Department of Neurosciences, KU Leuven, Leuven, Belgium.
- Leuven Brain Institute, KU Leuven, Leuven, Belgium.
| |
Collapse
|
11
|
Mohanta S, Cleveland DM, Afrasiabi M, Rhone AE, Górska U, Cooper Borkenhagen M, Sanders RD, Boly M, Nourski KV, Saalmann YB. Traveling waves shape neural population dynamics enabling predictions and internal model updating. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.01.09.574848. [PMID: 38260606 PMCID: PMC10802392 DOI: 10.1101/2024.01.09.574848] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/24/2024]
Abstract
The brain generates predictions based on statistical regularities in our environment. However, it is unclear how predictions are optimized through iterative interactions with the environment. Because traveling waves (TWs) propagate across the cortex shaping neural excitability, they can carry information to serve predictive processing. Using human intracranial recordings, we show that anterior-to-posterior alpha TWs correlated with prediction strength. Learning about priors altered neural state space trajectories, and how much it altered correlated with trial-by-trial prediction strength. Learning involved mismatches between predictions and sensory evidence triggering alpha-phase resets in lateral temporal cortex, accompanied by stronger alpha phase-high gamma amplitude coupling and high-gamma power. The mismatch initiated posterior-to-anterior alpha TWs and change in the subsequent trial's state space trajectory, facilitating model updating. Our findings suggest a vital role of alpha TWs carrying both predictions to sensory cortex and mismatch signals to frontal cortex for trial-by-trial fine-tuning of predictive models.
Collapse
Affiliation(s)
- S Mohanta
- Department of Psychology, University of Wisconsin-Madison, WI, USA
| | - D M Cleveland
- Department of Psychology, University of Wisconsin-Madison, WI, USA
| | - M Afrasiabi
- Department of Psychology, University of Wisconsin-Madison, WI, USA
| | - A E Rhone
- Department of Neurosurgery, University of Iowa, IA, USA
| | - U Górska
- Department of Psychiatry, University of Wisconsin-Madison, WI, USA
| | | | - R D Sanders
- Specialty of Anaesthesia, University of Sydney, Camperdown, NSW, Australia and Department of Anaesthetics and Institute of Academic Surgery, Royal Prince Alfred Hospital, Camperdown, NSW, Australia
| | - M Boly
- Department of Psychiatry, University of Wisconsin-Madison, WI, USA
- Department of Neurology, University of Wisconsin-Madison, WI, USA
| | - K V Nourski
- Department of Neurosurgery, University of Iowa, IA, USA
- Iowa Neuroscience Institute, University of Iowa, IA, USA
| | - Y B Saalmann
- Department of Psychology, University of Wisconsin-Madison, WI, USA
| |
Collapse
|
12
|
Shipp S. Computational components of visual predictive coding circuitry. Front Neural Circuits 2024; 17:1254009. [PMID: 38259953 PMCID: PMC10800426 DOI: 10.3389/fncir.2023.1254009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2023] [Accepted: 12/13/2023] [Indexed: 01/24/2024] Open
Abstract
If a full visual percept can be said to be a 'hypothesis', so too can a neural 'prediction' - although the latter addresses one particular component of image content (such as 3-dimensional organisation, the interplay between lighting and surface colour, the future trajectory of moving objects, and so on). And, because processing is hierarchical, predictions generated at one level are conveyed in a backward direction to a lower level, seeking to predict, in fact, the neural activity at that prior stage of processing, and learning from errors signalled in the opposite direction. This is the essence of 'predictive coding', at once an algorithm for information processing and a theoretical basis for the nature of operations performed by the cerebral cortex. Neural models for the implementation of predictive coding invoke specific functional classes of neuron for generating, transmitting and receiving predictions, and for producing reciprocal error signals. Also a third general class, 'precision' neurons, tasked with regulating the magnitude of error signals contingent upon the confidence placed upon the prediction, i.e., the reliability and behavioural utility of the sensory data that it predicts. So, what is the ultimate source of a 'prediction'? The answer is multifactorial: knowledge of the current environmental context and the immediate past, allied to memory and lifetime experience of the way of the world, doubtless fine-tuned by evolutionary history too. There are, in consequence, numerous potential avenues for experimenters seeking to manipulate subjects' expectation, and examine the neural signals elicited by surprising, and less surprising visual stimuli. This review focuses upon the predictive physiology of mouse and monkey visual cortex, summarising and commenting on evidence to date, and placing it in the context of the broader field. It is concluded that predictive coding has a firm grounding in basic neuroscience and that, unsurprisingly, there remains much to learn.
Collapse
Affiliation(s)
- Stewart Shipp
- Institute of Ophthalmology, University College London, London, United Kingdom
| |
Collapse
|
13
|
Raman R, Bognár A, Nejad GG, Taubert N, Giese M, Vogels R. Bodies in motion: Unraveling the distinct roles of motion and shape in dynamic body responses in the temporal cortex. Cell Rep 2023; 42:113438. [PMID: 37995183 PMCID: PMC10783614 DOI: 10.1016/j.celrep.2023.113438] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Revised: 09/26/2023] [Accepted: 10/26/2023] [Indexed: 11/25/2023] Open
Abstract
The temporal cortex represents social stimuli, including bodies. We examine and compare the contributions of dynamic and static features to the single-unit responses to moving monkey bodies in and between a patch in the anterior dorsal bank of the superior temporal sulcus (dorsal patch [DP]) and patches in the anterior inferotemporal cortex (ventral patch [VP]), using fMRI guidance in macaques. The response to dynamics varies within both regions, being higher in DP. The dynamic body selectivity of VP neurons correlates with static features derived from convolutional neural networks and motion. DP neurons' dynamic body selectivity is not predicted by static features but is dominated by motion. Whereas these data support the dominance of motion in the newly proposed "dynamic social perception" stream, they challenge the traditional view that distinguishes DP and VP processing in terms of motion versus static features, underscoring the role of inferotemporal neurons in representing body dynamics.
Collapse
Affiliation(s)
- Rajani Raman
- Department of Neurosciences, KU Leuven, 3000 Leuven, Belgium; Leuven Brain Institute, KU Leuven, 3000 Leuven, Belgium
| | - Anna Bognár
- Department of Neurosciences, KU Leuven, 3000 Leuven, Belgium; Leuven Brain Institute, KU Leuven, 3000 Leuven, Belgium
| | - Ghazaleh Ghamkhari Nejad
- Department of Neurosciences, KU Leuven, 3000 Leuven, Belgium; Leuven Brain Institute, KU Leuven, 3000 Leuven, Belgium
| | - Nick Taubert
- Hertie Institute for Clinical Brain Research and Center for Integrative Neuroscience, University Clinic Tuebingen, 72074 Tuebingen, Germany
| | - Martin Giese
- Hertie Institute for Clinical Brain Research and Center for Integrative Neuroscience, University Clinic Tuebingen, 72074 Tuebingen, Germany
| | - Rufin Vogels
- Department of Neurosciences, KU Leuven, 3000 Leuven, Belgium; Leuven Brain Institute, KU Leuven, 3000 Leuven, Belgium.
| |
Collapse
|
14
|
Sáringer S, Fehér Á, Sáry G, Kaposvári P. Gamma oscillations in visual statistical learning correlate with individual behavioral differences. Front Behav Neurosci 2023; 17:1285773. [PMID: 38025386 PMCID: PMC10663268 DOI: 10.3389/fnbeh.2023.1285773] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Accepted: 10/23/2023] [Indexed: 12/01/2023] Open
Abstract
Statistical learning is assumed to be a fundamentally general sensory process across modalities, age, other cognitive functions, and even species. Despite this general role, behavioral testing on regularity acquisition shows great variance among individuals. The current study aimed to find neural correlates of visual statistical learning showing a correlation with behavioral results. Based on a pilot study, we conducted an EEG study where participants were exposed to associated stimulus pairs; the acquisition was tested through a familiarity test. We identified an oscillation in the gamma range (40-70 Hz, 0.5-0.75 s post-stimulus), which showed a positive correlation with the behavioral results. This change in activity was located in a left frontoparietal cluster. Based on its latency and location, this difference was identified as a late gamma activity, a correlate of model-based learning. Such learning is a summary of several top-down mechanisms that modulate the recollection of statistical relationships such as the capacity of working memory or attention. These results suggest that, during acquisition, individual behavioral variance is influenced by dominant learning processes which affect the recall of previously gained information.
Collapse
Affiliation(s)
| | | | | | - Péter Kaposvári
- Department of Physiology, Albert Szent-Gyögyi Medical School, University of Szeged, Szeged, Hungary
| |
Collapse
|
15
|
den Ouden C, Zhou A, Mepani V, Kovács G, Vogels R, Feuerriegel D. Stimulus expectations do not modulate visual event-related potentials in probabilistic cueing designs. Neuroimage 2023; 280:120347. [PMID: 37648120 DOI: 10.1016/j.neuroimage.2023.120347] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Revised: 08/10/2023] [Accepted: 08/23/2023] [Indexed: 09/01/2023] Open
Abstract
Humans and other animals can learn and exploit repeating patterns that occur within their environments. These learned patterns can be used to form expectations about future sensory events. Several influential predictive coding models have been proposed to explain how learned expectations influence the activity of stimulus-selective neurons in the visual system. These models specify reductions in neural response measures when expectations are fulfilled (termed expectation suppression) and increases following surprising sensory events. However, there is currently scant evidence for expectation suppression in the visual system when confounding factors are taken into account. Effects of surprise have been observed in blood oxygen level dependent (BOLD) signals, but not when using electrophysiological measures. To provide a strong test for expectation suppression and surprise effects we performed a predictive cueing experiment while recording electroencephalographic (EEG) data. Participants (n=48) learned cue-face associations during a training session and were then exposed to these cue-face pairs in a subsequent experiment. Using univariate analyses of face-evoked event-related potentials (ERPs) we did not observe any differences across expected (90% probability), neutral (50%) and surprising (10%) face conditions. Across these comparisons, Bayes factors consistently favoured the null hypothesis throughout the time-course of the stimulus-evoked response. When using multivariate pattern analysis we did not observe above-chance classification of expected and surprising face-evoked ERPs. By contrast, we found robust within- and across-trial stimulus repetition effects. Our findings do not support predictive coding-based accounts that specify reduced prediction error signalling when perceptual expectations are fulfilled. They instead highlight the utility of other types of predictive processing models that describe expectation-related phenomena in the visual system without recourse to prediction error signalling.
Collapse
Affiliation(s)
- Carla den Ouden
- Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, Australia
| | - Andong Zhou
- Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, Australia
| | - Vinay Mepani
- Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, Australia
| | - Gyula Kovács
- Institute of Psychology, Friedrich Schiller University Jena, Jena, Germany
| | - Rufin Vogels
- Laboratorium voor Neuro- en Psychofysiologie, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Daniel Feuerriegel
- Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, Australia.
| |
Collapse
|
16
|
Li C, Ficco L, Trapp S, Rostalski SM, Korn L, Kovács G. The effect of context congruency on fMRI repetition suppression for objects. Neuropsychologia 2023; 188:108603. [PMID: 37270029 DOI: 10.1016/j.neuropsychologia.2023.108603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Revised: 05/31/2023] [Accepted: 05/31/2023] [Indexed: 06/05/2023]
Abstract
The recognition of objects is strongly facilitated when they are presented in the context of other objects (Biederman, 1972). Such contexts facilitate perception and induce expectations of context-congruent objects (Trapp and Bar, 2015). The neural mechanisms underlying these facilitatory effects of context on object processing, however, are not yet fully understood. In the present study, we investigate how context-induced expectations affect subsequent object processing. We used functional magnetic resonance imaging and measured repetition suppression as a proxy for prediction error processing. Participants viewed pairs of alternating or repeated object images which were preceded by context-congruent, context-incongruent or neutral cues. We found a stronger repetition suppression in congruent as compared to incongruent or neutral cues in the object sensitive lateral occipital cortex. Interestingly, this stronger effect was driven by enhanced responses to alternating stimulus pairs in the congruent contexts, rather than by suppressed responses to repeated stimulus pairs, which emphasizes the contribution of surprise-related response enhancement for the context modulation on RS when expectations are violated. In addition, in the congruent condition, we discovered significant functional connectivity between object-responsive and frontal cortical regions, as well as between object-responsive regions and the fusiform gyrus. Our findings indicate that prediction errors, reflected in enhanced brain responses to violated contextual expectations, underlie the facilitating effect of context during object perception.
Collapse
Affiliation(s)
- Chenglin Li
- School of Psychology, Zhejiang Normal University, China; Department of Biological Psychology and Cognitive Neurosciences, Institute of Psychology, Friedrich-Schiller-Universität Jena, Germany
| | - Linda Ficco
- Department of General Psychology and Cognitive Neuroscience, Institute of Psychology, Friedrich-Schiller-Universität Jena, Germany; Department of Linguistics and Cultural Evolution, International Max Planck Research School for the Science of Human History, Jena, Germany
| | - Sabrina Trapp
- Macromedia University of Applied Sciences, Munich, Germany
| | - Sophie-Marie Rostalski
- Department of Biological Psychology and Cognitive Neurosciences, Institute of Psychology, Friedrich-Schiller-Universität Jena, Germany
| | - Lukas Korn
- Department of Biological Psychology and Cognitive Neurosciences, Institute of Psychology, Friedrich-Schiller-Universität Jena, Germany
| | - Gyula Kovács
- Department of Biological Psychology and Cognitive Neurosciences, Institute of Psychology, Friedrich-Schiller-Universität Jena, Germany.
| |
Collapse
|
17
|
Yan C, Ehinger BV, Pérez-Bellido A, Peelen MV, de Lange FP. Humans predict the forest, not the trees: statistical learning of spatiotemporal structure in visual scenes. Cereb Cortex 2023; 33:8300-8311. [PMID: 37005064 PMCID: PMC7614728 DOI: 10.1093/cercor/bhad115] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Revised: 03/11/2023] [Accepted: 03/13/2023] [Indexed: 04/04/2023] Open
Abstract
The human brain is capable of using statistical regularities to predict future inputs. In the real world, such inputs typically comprise a collection of objects (e.g. a forest constitutes numerous trees). The present study aimed to investigate whether perceptual anticipation relies on lower-level or higher-level information. Specifically, we examined whether the human brain anticipates each object in a scene individually or anticipates the scene as a whole. To explore this issue, we first trained participants to associate co-occurring objects within fixed spatial arrangements. Meanwhile, participants implicitly learned temporal regularities between these displays. We then tested how spatial and temporal violations of the structure modulated behavior and neural activity in the visual system using fMRI. We found that participants only showed a behavioral advantage of temporal regularities when the displays conformed to their previously learned spatial structure, demonstrating that humans form configuration-specific temporal expectations instead of predicting individual objects. Similarly, we found suppression of neural responses for temporally expected compared with temporally unexpected objects in lateral occipital cortex only when the objects were embedded within expected configurations. Overall, our findings indicate that humans form expectations about object configurations, demonstrating the prioritization of higher-level over lower-level information in temporal expectation.
Collapse
Affiliation(s)
- Chuyao Yan
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Kapittelweg 29, Nijmegen 6525 EN, The Netherlands
- School of Psychology, Nanjing Normal University, Nanjing 210098, China
| | - Benedikt V Ehinger
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Kapittelweg 29, Nijmegen 6525 EN, The Netherlands
- Stuttgart Center for Simulation Science, University of Stuttgart, Stuttgart 70049, Germany
| | - Alexis Pérez-Bellido
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Kapittelweg 29, Nijmegen 6525 EN, The Netherlands
- Department of Cognition, Development and Educational Psychology, University of Barcelona, Barcelona 17108035, Spain
- Institute of Neurosciences, University of Barcelona, Barcelona 17108035, Spain
| | - Marius V Peelen
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Kapittelweg 29, Nijmegen 6525 EN, The Netherlands
| | - Floris P de Lange
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Kapittelweg 29, Nijmegen 6525 EN, The Netherlands
| |
Collapse
|
18
|
Esmailpour H, Raman R, Vogels R. Inferior temporal cortex leads prefrontal cortex in response to a violation of a learned sequence. Cereb Cortex 2023; 33:3124-3141. [PMID: 35780398 DOI: 10.1093/cercor/bhac265] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2022] [Revised: 06/09/2022] [Accepted: 06/09/2022] [Indexed: 11/13/2022] Open
Abstract
Primates learn statistical regularities that are embedded in visual sequences, a form of statistical learning. Single-unit recordings in macaques showed that inferior temporal (IT) neurons are sensitive to statistical regularities in visual sequences. Here, we asked whether ventrolateral prefrontal cortex (VLPFC), which is connected to IT, is also sensitive to the transition probabilities in visual sequences and whether the statistical learning signal in IT originates in VLPFC. We recorded simultaneously multiunit activity (MUA) and local field potentials (LFPs) in IT and VLPFC after monkeys were exposed to triplets of images with a fixed presentation order. In both areas, the MUA was stronger to images that violated the learned sequence (deviants) compared to the same images presented in the learned triplets. The high-gamma and beta LFP power showed an enhanced and suppressed response, respectively, to the deviants in both areas. The enhanced response was present also for the image following the deviant, suggesting a sensitivity for temporal adjacent dependencies in IT and VLPFC. The increased response to the deviant occurred later in VLPFC than in IT, suggesting that the deviant response in IT was not inherited from VLPFC. These data support predictive coding theories that propose a feedforward flow of prediction errors.
Collapse
Affiliation(s)
- Hamideh Esmailpour
- Laboratorium voor Neuro-en Psychofysiologie, Department of Neurosciences, KU Leuven, Herestraat 49, 3000 Leuven, Belgium
- Leuven Brain Institute, KU Leuven, ON V Herestraat 49, 3000 Leuven, Belgium
| | - Rajani Raman
- Laboratorium voor Neuro-en Psychofysiologie, Department of Neurosciences, KU Leuven, Herestraat 49, 3000 Leuven, Belgium
- Leuven Brain Institute, KU Leuven, ON V Herestraat 49, 3000 Leuven, Belgium
| | - Rufin Vogels
- Laboratorium voor Neuro-en Psychofysiologie, Department of Neurosciences, KU Leuven, Herestraat 49, 3000 Leuven, Belgium
- Leuven Brain Institute, KU Leuven, ON V Herestraat 49, 3000 Leuven, Belgium
| |
Collapse
|
19
|
Benetti S, Ferrari A, Pavani F. Multimodal processing in face-to-face interactions: A bridging link between psycholinguistics and sensory neuroscience. Front Hum Neurosci 2023; 17:1108354. [PMID: 36816496 PMCID: PMC9932987 DOI: 10.3389/fnhum.2023.1108354] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Accepted: 01/11/2023] [Indexed: 02/05/2023] Open
Abstract
In face-to-face communication, humans are faced with multiple layers of discontinuous multimodal signals, such as head, face, hand gestures, speech and non-speech sounds, which need to be interpreted as coherent and unified communicative actions. This implies a fundamental computational challenge: optimally binding only signals belonging to the same communicative action while segregating signals that are not connected by the communicative content. How do we achieve such an extraordinary feat, reliably, and efficiently? To address this question, we need to further move the study of human communication beyond speech-centred perspectives and promote a multimodal approach combined with interdisciplinary cooperation. Accordingly, we seek to reconcile two explanatory frameworks recently proposed in psycholinguistics and sensory neuroscience into a neurocognitive model of multimodal face-to-face communication. First, we introduce a psycholinguistic framework that characterises face-to-face communication at three parallel processing levels: multiplex signals, multimodal gestalts and multilevel predictions. Second, we consider the recent proposal of a lateral neural visual pathway specifically dedicated to the dynamic aspects of social perception and reconceive it from a multimodal perspective ("lateral processing pathway"). Third, we reconcile the two frameworks into a neurocognitive model that proposes how multiplex signals, multimodal gestalts, and multilevel predictions may be implemented along the lateral processing pathway. Finally, we advocate a multimodal and multidisciplinary research approach, combining state-of-the-art imaging techniques, computational modelling and artificial intelligence for future empirical testing of our model.
Collapse
Affiliation(s)
- Stefania Benetti
- Centre for Mind/Brain Sciences, University of Trento, Trento, Italy,Interuniversity Research Centre “Cognition, Language, and Deafness”, CIRCLeS, Catania, Italy,*Correspondence: Stefania Benetti,
| | - Ambra Ferrari
- Max Planck Institute for Psycholinguistics, Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Nijmegen, Netherlands
| | - Francesco Pavani
- Centre for Mind/Brain Sciences, University of Trento, Trento, Italy,Interuniversity Research Centre “Cognition, Language, and Deafness”, CIRCLeS, Catania, Italy
| |
Collapse
|
20
|
Mancini F, Zhang S, Seymour B. Computational and neural mechanisms of statistical pain learning. Nat Commun 2022; 13:6613. [PMID: 36329014 PMCID: PMC9633765 DOI: 10.1038/s41467-022-34283-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Accepted: 10/11/2022] [Indexed: 11/06/2022] Open
Abstract
Pain invariably changes over time. These fluctuations contain statistical regularities which, in theory, could be learned by the brain to generate expectations and control responses. We demonstrate that humans learn to extract these regularities and explicitly predict the likelihood of forthcoming pain intensities in a manner consistent with optimal Bayesian inference with dynamic update of beliefs. Healthy participants received probabilistic, volatile sequences of low and high-intensity electrical stimuli to the hand during brain fMRI. The inferred frequency of pain correlated with activity in sensorimotor cortical regions and dorsal striatum, whereas the uncertainty of these inferences was encoded in the right superior parietal cortex. Unexpected changes in stimulus frequencies drove the update of internal models by engaging premotor, prefrontal and posterior parietal regions. This study extends our understanding of sensory processing of pain to include the generation of Bayesian internal models of the temporal statistics of pain.
Collapse
Affiliation(s)
- Flavia Mancini
- Department of Engineering, University of Cambridge, Trumpington Street, Cambridge, CB2 1PZ, UK.
| | - Suyi Zhang
- Wellcome Centre for Integrative Neuroimaging, John Radcliffe Hospital, Headington, Oxford, OX3 9DU, UK
| | - Ben Seymour
- Wellcome Centre for Integrative Neuroimaging, John Radcliffe Hospital, Headington, Oxford, OX3 9DU, UK
- Center for Information and Neural Networks (CiNet), 1-4 Yamadaoka, Suita City, Osaka, 565-0871, Japan
| |
Collapse
|
21
|
Online measurement of learning temporal statistical structure in categorization tasks. Mem Cognit 2022; 50:1530-1545. [PMID: 35377057 PMCID: PMC9508059 DOI: 10.3758/s13421-022-01302-5] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/10/2022] [Indexed: 11/08/2022]
Abstract
AbstractThe ability to grasp relevant patterns from a continuous stream of environmental information is called statistical learning. Although the representations that emerge during visual statistical learning (VSL) are well characterized, little is known about how they are formed. We developed a sensitive behavioral design to characterize the VSL trajectory during ongoing task performance. In sequential categorization tasks, we assessed two previously identified VSL markers: priming of the second predictable image in a pair manifested by a reduced reaction time (RT) and greater accuracy, and the anticipatory effect on the first image revealed by a longer RT. First, in Experiment 1A, we used an adapted paradigm and replicated these VSL markers; however, they appeared to be confounded by motor learning. Next, in Experiment 1B, we confirmed the confounding influence of motor learning. To assess VSL without motor learning, in Experiment 2 we (1) simplified the categorization task, (2) raised the number of subjects and image repetitions, and (3) increased the number of single unpaired images. Using linear mixed-effect modeling and estimated marginal means of linear trends, we found that the RT curves differed significantly between predictable paired and control single images. Further, the VSL curve fitted a logarithmic model, suggesting a rapid learning process. These results suggest that our paradigm in Experiment 2 seems to be a viable online tool to monitor the behavioral correlates of unsupervised implicit VSL.
Collapse
|
22
|
Yang X, Su Y, Yang F, Song Y, Yan J, Luo Y, Zeng J. Neurofunctional mapping of reward anticipation and outcome for major depressive disorder: a voxel-based meta-analysis. Psychol Med 2022; 52:1-14. [PMID: 36047042 DOI: 10.1017/s0033291722002707] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Aberrations in how people form expectations about rewards and how they respond to receiving rewards are thought to underlie major depressive disorder (MDD). However, the underlying mechanism linking the appetitive reward system, specifically anticipation and outcome, is still not fully understood. To examine the neural correlates of monetary anticipation and outcome in currently depressed subjects with MDD, we performed two separate voxel-wise meta-analyses of functional neuroimaging studies using the monetary incentive delay task. During reward anticipation, the depressed patients exhibited an increased response in the bilateral middle cingulate cortex (MCC) extending to the anterior cingulate cortex, the medial prefrontal cortex, the left inferior frontal gyrus (IFG), and the postcentral gyrus, but a reduced response in the mesolimbic circuit, including the left striatum, insula, amygdala, right cerebellum, striatum, and IFG, compared to controls. During the outcome stage, MDD showed higher activity in the left inferior temporal gyrus, and lower activity in the mesocortical pathway, including the bilateral MCC, left caudate nucleus, precentral gyrus, thalamus, cerebellum, right striatum, insula, IFG, middle frontal gyrus, and temporal pole. Our findings suggest that cMDD may be characterised by state-dependent hyper-responsivity in cortical regions during the anticipation phase, and hypo-responsivity of the mesocortico-limbic circuit across the two phases of the reward response. Our study showed dissociable neural circuit responses to monetary stimuli during reward anticipation and outcome, which help to understand the dysfunction in different aspects of reward processing, particularly motivational v. hedonic deficits in depression.
Collapse
Affiliation(s)
- Xun Yang
- School of Public Policy and Administration, Chongqing University, Chongqing, 400044, China
| | - Yueyue Su
- School of Public Policy and Administration, Chongqing University, Chongqing, 400044, China
| | - Fan Yang
- Department of Ultrasonography, West China Second University Hospital, Sichuan University, Chengdu, 610041, China
- Chengdu Chenghua District Maternal and Child Health Hospital, Sichuan University, Chengdu, 610041, China
| | - Yuan Song
- School of Public Policy and Administration, Chongqing University, Chongqing, 400044, China
| | - Jiangnan Yan
- School of Economics and Business Administration, Chongqing University, Chongqing, 400044, China
| | - Ya Luo
- Department of Psychiatry, State Key Lab of Biotherapy, West China Hospital of Sichuan University, Chengdu, 610041, China
| | - Jianguang Zeng
- School of Economics and Business Administration, Chongqing University, Chongqing, 400044, China
| |
Collapse
|
23
|
Burk DC, Sheinberg DL. Neurons in inferior temporal cortex are sensitive to motion trajectory during degraded object recognition. Cereb Cortex Commun 2022; 3:tgac034. [PMID: 36168516 PMCID: PMC9499820 DOI: 10.1093/texcom/tgac034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Revised: 08/03/2022] [Accepted: 08/05/2022] [Indexed: 11/30/2022] Open
Abstract
Our brains continuously acquire sensory information and make judgments even when visual information is limited. In some circumstances, an ambiguous object can be recognized from how it moves, such as an animal hopping or a plane flying overhead. Yet it remains unclear how movement is processed by brain areas involved in visual object recognition. Here we investigate whether inferior temporal (IT) cortex, an area known for its relevance in visual form processing, has access to motion information during recognition. We developed a matching task that required monkeys to recognize moving shapes with variable levels of shape degradation. Neural recordings in area IT showed that, surprisingly, some IT neurons responded stronger to degraded shapes than clear ones. Furthermore, neurons exhibited motion sensitivity at different times during the presentation of the blurry target. Population decoding analyses showed that motion patterns could be decoded from IT neuron pseudo-populations. Contrary to previous findings, these results suggest that neurons in IT can integrate visual motion and shape information, particularly when shape information is degraded, in a way that has been previously overlooked. Our results highlight the importance of using challenging multifeature recognition tasks to understand the role of area IT in naturalistic visual object recognition.
Collapse
Affiliation(s)
- Diana C Burk
- Department of Neuroscience, Brown University , Providence, RI 02912 , United States
| | - David L Sheinberg
- Department of Neuroscience, Brown University , Providence, RI 02912 , United States
- Carney Institute for Brain Science, Brown University , Providence, RI 02912 , United States
| |
Collapse
|
24
|
Wu S, Blanchard T, Meschke E, Aslin RN, Hayden BY, Kidd C. Macaques preferentially attend to intermediately surprising information. Biol Lett 2022; 18:20220144. [PMID: 35857891 PMCID: PMC9256086 DOI: 10.1098/rsbl.2022.0144] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022] Open
Abstract
Normative learning theories dictate that we should preferentially attend to informative sources, but only up to the point that our limited learning systems can process their content. Humans, including infants, show this predicted strategic deployment of attention. Here, we demonstrate that rhesus monkeys, much like humans, attend to events of moderate surprisingness over both more and less surprising events. They do this in the absence of any specific goal or contingent reward, indicating that the behavioural pattern is spontaneous. We suggest this U-shaped attentional preference represents an evolutionarily preserved strategy for guiding intelligent organisms toward material that is maximally useful for learning.
Collapse
Affiliation(s)
- Shengyi Wu
- Department of Psychology, University of California, Berkeley, 2121 Berkeley Way West, Berkeley, CA 94720, USA
| | | | - Emily Meschke
- Helen Wills Neuroscience Institute, University of California, Berkeley, 175 Li Ka Shing Center, MC 3370, Berkeley, CA 94720, USA
| | - Richard N Aslin
- Haskins Laboratories, Yale University, 300 George Street, New Haven, CT 06511, USA
| | - Benjamin Y Hayden
- Department of Neuroscience, University of Minnesota, 321 Church St SE, Minneapolis, MN 55455, USA
| | - Celeste Kidd
- Department of Psychology, University of California, Berkeley, 2121 Berkeley Way West, Berkeley, CA 94720, USA
| |
Collapse
|
25
|
He T, Richter D, Wang Z, de Lange FP. Spatial and Temporal Context Jointly Modulate the Sensory Response within the Ventral Visual Stream. J Cogn Neurosci 2021; 34:332-347. [PMID: 34964889 DOI: 10.1162/jocn_a_01792] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Both spatial and temporal context play an important role in visual perception and behavior. Humans can extract statistical regularities from both forms of context to help process the present and to construct expectations about the future. Numerous studies have found reduced neural responses to expected stimuli compared with unexpected stimuli, for both spatial and temporal regularities. However, it is largely unclear whether and how these forms of context interact. In the current fMRI study, 33 human volunteers were exposed to pairs of object stimuli that could be expected or surprising in terms of their spatial and temporal context. We found reliable independent contributions of both spatial and temporal context in modulating the neural response. Specifically, neural responses to stimuli in expected compared with unexpected contexts were suppressed throughout the ventral visual stream. These results suggest that both spatial and temporal context may aid sensory processing in a similar fashion, providing evidence on how different types of context jointly modulate perceptual processing.
Collapse
|
26
|
Boros M, Magyari L, Török D, Bozsik A, Deme A, Andics A. Neural processes underlying statistical learning for speech segmentation in dogs. Curr Biol 2021; 31:5512-5521.e5. [PMID: 34717832 DOI: 10.1016/j.cub.2021.10.017] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 07/23/2021] [Accepted: 10/07/2021] [Indexed: 10/20/2022]
Abstract
To learn words, humans extract statistical regularities from speech. Multiple species use statistical learning also to process speech, but the neural underpinnings of speech segmentation in non-humans remain largely unknown. Here, we investigated computational and neural markers of speech segmentation in dogs, a phylogenetically distant mammal that efficiently navigates humans' social and linguistic environment. Using electroencephalography (EEG), we compared event-related responses (ERPs) for artificial words previously presented in a continuous speech stream with different distributional statistics. Results revealed an early effect (220-470 ms) of transitional probability and a late component (590-790 ms) modulated by both word frequency and transitional probability. Using fMRI, we searched for brain regions sensitive to statistical regularities in speech. Structured speech elicited lower activity in the basal ganglia, a region involved in sequence learning, and repetition enhancement in the auditory cortex. Speech segmentation in dogs, similar to that of humans, involves complex computations, engaging both domain-general and modality-specific brain areas. VIDEO ABSTRACT.
Collapse
Affiliation(s)
- Marianna Boros
- MTA-ELTE "Lendület" Neuroethology of Communication Research Group, Hungarian Academy of Sciences - Eötvös Loránd University, 1117 Budapest, Pázmány Péter sétány 1/C, Hungary; Department of Ethology, Eötvös Loránd University, 1117 Budapest, Pázmány Péter sétány 1/C, Hungary.
| | - Lilla Magyari
- MTA-ELTE "Lendület" Neuroethology of Communication Research Group, Hungarian Academy of Sciences - Eötvös Loránd University, 1117 Budapest, Pázmány Péter sétány 1/C, Hungary; Department of Ethology, Eötvös Loránd University, 1117 Budapest, Pázmány Péter sétány 1/C, Hungary; Norwegian Reading Centre for Reading Education and Research, Faculty of Arts and Education, University of Stavanger, Professor Olav Hanssens vei 10, 4036 Stavanger, Norway
| | - Dávid Török
- MTA-ELTE "Lendület" Neuroethology of Communication Research Group, Hungarian Academy of Sciences - Eötvös Loránd University, 1117 Budapest, Pázmány Péter sétány 1/C, Hungary; Department of Ethology, Eötvös Loránd University, 1117 Budapest, Pázmány Péter sétány 1/C, Hungary
| | - Anett Bozsik
- MTA-ELTE "Lendület" Neuroethology of Communication Research Group, Hungarian Academy of Sciences - Eötvös Loránd University, 1117 Budapest, Pázmány Péter sétány 1/C, Hungary; Department of Anatomy and Histology, University of Veterinary Medicine, 1078 Budapest, István utca 2, Hungary
| | - Andrea Deme
- Department of Applied Linguistics and Phonetics, Eötvös Loránd University, 1088 Budapest, Múzeum krt. 4/A, Hungary; MTA-ELTE "Lendület" Lingual Articulation Research Group, 1088 Budapest, Múzeum krt. 4/A, Hungary
| | - Attila Andics
- MTA-ELTE "Lendület" Neuroethology of Communication Research Group, Hungarian Academy of Sciences - Eötvös Loránd University, 1117 Budapest, Pázmány Péter sétány 1/C, Hungary; Department of Ethology, Eötvös Loránd University, 1117 Budapest, Pázmány Péter sétány 1/C, Hungary.
| |
Collapse
|
27
|
Kimura M. Prediction, Suppression of Visual Response, and Modulation of Visual Perception: Insights From Visual Evoked Potentials and Representational Momentum. Front Hum Neurosci 2021; 15:730962. [PMID: 34512299 PMCID: PMC8425455 DOI: 10.3389/fnhum.2021.730962] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Accepted: 07/27/2021] [Indexed: 11/27/2022] Open
Abstract
When a visual object changes its position along with certain sequential regularities, the visual system rapidly and automatically forms a prediction regarding the future position of the object based on the regularities. Such prediction can drastically alter visual perception. A phenomenon called representational momentum (RM: a predictive displacement of the perceived final position of a visual object along its recent regular pattern) has provided extensive evidence for the predictive modulation of visual perception. The purpose of the present study was to identify neural effects that could explain individual differences in the strength of the predictive modulation of visual perception as measured by RM. For this purpose, in two experiments with a conventional RM paradigm where a bar was discretely presented in a regular rotation manner (with a step of 18° in Experiment 1 and a step of 20° in Experiment 2), visual evoked potentials (VEPs) in response to the regularly rotated bar were measured, and correlations between the magnitudes of RM and VEPs were examined. The results showed that the magnitudes of RM and central P2 were negatively correlated, consistently in both experiments; participants who showed a smaller central P2 tended to exhibit greater RM. Together with a previous proposal that central P2 would represent delayed reactivation of lower visual areas around the striate and prestriate cortices via reentrant feedback projections from higher areas, the present results suggest that greater suppression of delayed reactivation of lower visual areas (as indicated by smaller central P2) may underlie stronger predictive modulation of visual perception (as indicated by greater RM).
Collapse
Affiliation(s)
- Motohiro Kimura
- Department of Information Technology and Human Factors, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Japan
| |
Collapse
|
28
|
Feuerriegel D, Vogels R, Kovács G. Evaluating the evidence for expectation suppression in the visual system. Neurosci Biobehav Rev 2021; 126:368-381. [PMID: 33836212 DOI: 10.1016/j.neubiorev.2021.04.002] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2020] [Revised: 02/16/2021] [Accepted: 04/02/2021] [Indexed: 01/25/2023]
Abstract
Reports of expectation suppression have shaped the development of influential predictive coding-based theories of visual perception. However recent work has highlighted confounding factors that may mimic or inflate expectation suppression effects. In this review, we describe four confounds that are prevalent across experiments that tested for expectation suppression: effects of surprise, attention, stimulus repetition and adaptation, and stimulus novelty. With these confounds in mind we then critically review the evidence for expectation suppression across probabilistic cueing, statistical learning, oddball, action-outcome learning and apparent motion designs. We found evidence for expectation suppression within a specific subset of statistical learning designs that involved weeks of sequence learning prior to neural activity measurement. Across other experimental contexts, whereby stimulus appearance probabilities were learned within one or two testing sessions, there was inconsistent evidence for genuine expectation suppression. We discuss how an absence of expectation suppression could inform models of predictive processing, repetition suppression and perceptual decision-making. We also provide suggestions for designing experiments that may better test for expectation suppression in future work.
Collapse
Affiliation(s)
- Daniel Feuerriegel
- Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, Australia.
| | - Rufin Vogels
- Laboratorium voor Neuro- en Psychofysiologie, Department of Neurosciences, KU Leuven, Leuven, Belgium
| | - Gyula Kovács
- Institute of Psychology, Friedrich Schiller University Jena, Jena, Germany
| |
Collapse
|
29
|
Solomon SS, Tang H, Sussman E, Kohn A. Limited Evidence for Sensory Prediction Error Responses in Visual Cortex of Macaques and Humans. Cereb Cortex 2021; 31:3136-3152. [PMID: 33683317 DOI: 10.1093/cercor/bhab014] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2020] [Revised: 12/06/2020] [Accepted: 01/15/2021] [Indexed: 11/14/2022] Open
Abstract
A recent formulation of predictive coding theory proposes that a subset of neurons in each cortical area encodes sensory prediction errors, the difference between predictions relayed from higher cortex and the sensory input. Here, we test for evidence of prediction error responses in spiking responses and local field potentials (LFP) recorded in primary visual cortex and area V4 of macaque monkeys, and in complementary electroencephalographic (EEG) scalp recordings in human participants. We presented a fixed sequence of visual stimuli on most trials, and violated the expected ordering on a small subset of trials. Under predictive coding theory, pattern-violating stimuli should trigger robust prediction errors, but we found that spiking, LFP and EEG responses to expected and pattern-violating stimuli were nearly identical. Our results challenge the assertion that a fundamental computational motif in sensory cortex is to signal prediction errors, at least those based on predictions derived from temporal patterns of visual stimulation.
Collapse
Affiliation(s)
- Selina S Solomon
- Dominick P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY 10461, USA
| | - Huizhen Tang
- Dominick P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY 10461, USA
- Department of Otorhinolaryngology - Head & Neck Surgery, Albert Einstein College of Medicine, Bronx, NY 10461, USA
| | - Elyse Sussman
- Dominick P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY 10461, USA
- Department of Otorhinolaryngology - Head & Neck Surgery, Albert Einstein College of Medicine, Bronx, NY 10461, USA
| | - Adam Kohn
- Dominick P. Purpura Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY 10461, USA
- Department of Ophthalmology and Vision Sciences, Albert Einstein College of Medicine, Bronx, NY 10461, USA
- Department of Systems and Computational Biology, Albert Einstein College of Medicine, Bronx, NY 10461, USA
| |
Collapse
|
30
|
Prior expectations evoke stimulus-specific activity in the deep layers of the primary visual cortex. PLoS Biol 2020; 18:e3001023. [PMID: 33284791 PMCID: PMC7746273 DOI: 10.1371/journal.pbio.3001023] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2020] [Revised: 12/17/2020] [Accepted: 11/20/2020] [Indexed: 12/23/2022] Open
Abstract
The way we perceive the world is strongly influenced by our expectations. In line with this, much recent research has revealed that prior expectations strongly modulate sensory processing. However, the neural circuitry through which the brain integrates external sensory inputs with internal expectation signals remains unknown. In order to understand the computational architecture of the cortex, we need to investigate the way these signals flow through the cortical layers. This is crucial because the different cortical layers have distinct intra- and interregional connectivity patterns, and therefore determining which layers are involved in a cortical computation can inform us on the sources and targets of these signals. Here, we used ultra-high field (7T) functional magnetic resonance imaging (fMRI) to reveal that prior expectations evoke stimulus-specific activity selectively in the deep layers of the primary visual cortex (V1). These findings are in line with predictive processing theories proposing that neurons in the deep cortical layers represent perceptual hypotheses and thereby shed light on the computational architecture of cortex. The way we perceive the world is strongly influenced by our expectations, but the neural circuitry through which the brain achieves this remains unknown. A study using ultra-high field fMRI reveals that prior expectations evoke stimulus-specific signals in the deep layers of the primary visual cortex.
Collapse
|
31
|
Vergnieux V, Vogels R. Statistical Learning Signals for Complex Visual Images in Macaque Early Visual Cortex. Front Neurosci 2020; 14:789. [PMID: 32848562 PMCID: PMC7411161 DOI: 10.3389/fnins.2020.00789] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2020] [Accepted: 07/06/2020] [Indexed: 12/21/2022] Open
Abstract
Animals of several species, including primates, learn the statistical regularities of their environment. In particular, they learn the temporal regularities that occur in streams of visual images. Previous human neuroimaging studies reported discrepant effects of such statistical learning, ranging from stronger occipito-temporal activations for sequences in which image order was fixed, compared with sequences of randomly ordered images, to weaker activations for fixed-order sequences compared with sequences that violated the learned order. Several single-unit studies in macaque monkeys reported that after statistical learning of temporal regularities, inferior temporal (IT) neurons show reduced responses to learned fixed-order sequences of visual images compared with random or mispredicted sequences. However, it is unknown how other macaque brain areas respond to such temporal statistical regularities. To address this gap, we exposed rhesus monkeys (Macaca mulatta) to two types of sequences of complex images. The “regular” sequences consisted of a continuous stream of quartets, and within each quartet, the image order was fixed. The quartets themselves were displayed, uninterrupted, in a random order. The same monkeys were exposed to sequences of other images having a pseudorandomized order (“random” sequence). After exposure, both monkeys were scanned with functional MRI (fMRI) using a block design with three conditions: regular sequence, random sequence, and fixation-only blocks. A whole-brain analysis showed a reduced activation in mainly the occipito-temporal cortex for the regular compared to the random sequences. Marked response reductions for the regular sequence were observed in early extrastriate visual cortical areas, including area V2, despite the use of rather complex images of animals. These data suggest that statistical learning signals are already present in early visual areas of monkeys, even for complex visual images. These monkey fMRI data are in line with recent human fMRI studies that showed a reduced activation in early visual areas for predicted compared with mispredicted complex images.
Collapse
Affiliation(s)
- Victor Vergnieux
- Laboratorium voor Neuro- en Psychofysiologie, Department of Neurosciences, KU Leuven, Leuven, Belgium.,Leuven Brain Institute, KU Leuven, Leuven, Belgium
| | - Rufin Vogels
- Laboratorium voor Neuro- en Psychofysiologie, Department of Neurosciences, KU Leuven, Leuven, Belgium.,Leuven Brain Institute, KU Leuven, Leuven, Belgium
| |
Collapse
|
32
|
Walsh KS, McGovern DP, Clark A, O'Connell RG. Evaluating the neurophysiological evidence for predictive processing as a model of perception. Ann N Y Acad Sci 2020; 1464:242-268. [PMID: 32147856 PMCID: PMC7187369 DOI: 10.1111/nyas.14321] [Citation(s) in RCA: 109] [Impact Index Per Article: 27.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2019] [Revised: 01/21/2020] [Accepted: 02/03/2020] [Indexed: 12/12/2022]
Abstract
For many years, the dominant theoretical framework guiding research into the neural origins of perceptual experience has been provided by hierarchical feedforward models, in which sensory inputs are passed through a series of increasingly complex feature detectors. However, the long-standing orthodoxy of these accounts has recently been challenged by a radically different set of theories that contend that perception arises from a purely inferential process supported by two distinct classes of neurons: those that transmit predictions about sensory states and those that signal sensory information that deviates from those predictions. Although these predictive processing (PP) models have become increasingly influential in cognitive neuroscience, they are also criticized for lacking the empirical support to justify their status. This limited evidence base partly reflects the considerable methodological challenges that are presented when trying to test the unique predictions of these models. However, a confluence of technological and theoretical advances has prompted a recent surge in human and nonhuman neurophysiological research seeking to fill this empirical gap. Here, we will review this new research and evaluate the degree to which its findings support the key claims of PP.
Collapse
Affiliation(s)
- Kevin S. Walsh
- Trinity College Institute of Neuroscience and School of PsychologyTrinity College DublinDublinIreland
| | - David P. McGovern
- Trinity College Institute of Neuroscience and School of PsychologyTrinity College DublinDublinIreland
- School of PsychologyDublin City UniversityDublinIreland
| | - Andy Clark
- Department of PhilosophyUniversity of SussexBrightonUK
- Department of InformaticsUniversity of SussexBrightonUK
| | - Redmond G. O'Connell
- Trinity College Institute of Neuroscience and School of PsychologyTrinity College DublinDublinIreland
| |
Collapse
|
33
|
Zhou YJ, Pérez-Bellido A, Haegens S, de Lange FP. Perceptual Expectations Modulate Low-Frequency Activity: A Statistical Learning Magnetoencephalography Study. J Cogn Neurosci 2019; 32:691-702. [PMID: 31820679 DOI: 10.1162/jocn_a_01511] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/29/2023]
Abstract
Perceptual expectations can change how a visual stimulus is perceived. Recent studies have shown mixed results in terms of whether expectations modulate sensory representations. Here, we used a statistical learning paradigm to study the temporal characteristics of perceptual expectations. We presented participants with pairs of object images organized in a predictive manner and then recorded their brain activity with magnetoencephalography while they viewed expected and unexpected image pairs on the subsequent day. We observed stronger alpha-band (7-14 Hz) activity in response to unexpected compared with expected object images. Specifically, the alpha-band modulation occurred as early as the onset of the stimuli and was most pronounced in left occipito-temporal cortex. Given that the differential response to expected versus unexpected stimuli occurred in sensory regions early in time, our results suggest that expectations modulate perceptual decision-making by changing the sensory response elicited by the stimuli.
Collapse
|
34
|
Kok P, Rait LI, Turk-Browne NB. Content-based Dissociation of Hippocampal Involvement in Prediction. J Cogn Neurosci 2019; 32:527-545. [PMID: 31820676 DOI: 10.1162/jocn_a_01509] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
Recent work suggests that a key function of the hippocampus is to predict the future. This is thought to depend on its ability to bind inputs over time and space and to retrieve upcoming or missing inputs based on partial cues. In line with this, previous research has revealed prediction-related signals in the hippocampus for complex visual objects, such as fractals and abstract shapes. Implicit in such accounts is that these computations in the hippocampus reflect domain-general processes that apply across different types and modalities of stimuli. An alternative is that the hippocampus plays a more domain-specific role in predictive processing, with the type of stimuli being predicted determining its involvement. To investigate this, we compared hippocampal responses to auditory cues predicting abstract shapes (Experiment 1) versus oriented gratings (Experiment 2). We measured brain activity in male and female human participants using high-resolution fMRI, in combination with inverted encoding models to reconstruct shape and orientation information. Our results revealed that expectations about shape and orientation evoked distinct representations in the hippocampus. For complex shapes, the hippocampus represented which shape was expected, potentially serving as a source of top-down predictions. In contrast, for simple gratings, the hippocampus represented only unexpected orientations, more reminiscent of a prediction error. We discuss several potential explanations for this content-based dissociation in hippocampal function, concluding that the computational role of the hippocampus in predictive processing may depend on the nature and complexity of stimuli.
Collapse
Affiliation(s)
- Peter Kok
- Yale University.,University College London
| | | | | |
Collapse
|
35
|
A common probabilistic framework for perceptual and statistical learning. Curr Opin Neurobiol 2019; 58:218-228. [PMID: 31669722 DOI: 10.1016/j.conb.2019.09.007] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2019] [Revised: 08/24/2019] [Accepted: 09/09/2019] [Indexed: 11/20/2022]
Abstract
System-level learning of sensory information is traditionally divided into two domains: perceptual learning that focuses on acquiring knowledge suitable for fine discrimination between similar sensory inputs, and statistical learning that explores the mechanisms that develop complex representations of unfamiliar sensory experiences. The two domains have been typically treated in complete separation both in terms of the underlying computational mechanisms and the brain areas and processes implementing those computations. However, a number of recent findings in both domains call in question this strict separation. We interpret classical and more recent results in the general framework of probabilistic computation, provide a unifying view of how various aspects of the two domains are interlinked, and suggest how the probabilistic approach can also alleviate the problem of dealing with widely different types of neural correlates of learning. Finally, we outline several directions along which our proposed approach fosters new types of experiments that can promote investigations of natural learning in humans and other species.
Collapse
|
36
|
Han B, Mostert P, de Lange FP. Predictable tones elicit stimulus-specific suppression of evoked activity in auditory cortex. Neuroimage 2019; 200:242-249. [DOI: 10.1016/j.neuroimage.2019.06.033] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2019] [Revised: 05/11/2019] [Accepted: 06/16/2019] [Indexed: 10/26/2022] Open
|
37
|
Richter D, de Lange FP. Statistical learning attenuates visual activity only for attended stimuli. eLife 2019; 8:e47869. [PMID: 31442202 PMCID: PMC6731093 DOI: 10.7554/elife.47869] [Citation(s) in RCA: 46] [Impact Index Per Article: 9.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2019] [Accepted: 08/21/2019] [Indexed: 12/21/2022] Open
Abstract
Perception and behavior can be guided by predictions, which are often based on learned statistical regularities. Neural responses to expected stimuli are frequently found to be attenuated after statistical learning. However, whether this sensory attenuation following statistical learning occurs automatically or depends on attention remains unknown. In the present fMRI study, we exposed human volunteers to sequentially presented object stimuli, in which the first object predicted the identity of the second object. We observed a reliable attenuation of neural activity for expected compared to unexpected stimuli in the ventral visual stream. Crucially, this sensory attenuation was only apparent when stimuli were attended, and vanished when attention was directed away from the predictable objects. These results put important constraints on neurocomputational theories that cast perception as a process of probabilistic integration of prior knowledge and sensory information.
Collapse
Affiliation(s)
- David Richter
- Donders Institute for Brain, Cognition and BehaviourRadboud University NijmegenNijmegenNetherlands
| | - Floris P de Lange
- Donders Institute for Brain, Cognition and BehaviourRadboud University NijmegenNijmegenNetherlands
| |
Collapse
|
38
|
Tang MF, Smout CA, Arabzadeh E, Mattingley JB. Prediction error and repetition suppression have distinct effects on neural representations of visual information. eLife 2018; 7:33123. [PMID: 30547881 PMCID: PMC6312401 DOI: 10.7554/elife.33123] [Citation(s) in RCA: 31] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2017] [Accepted: 12/13/2018] [Indexed: 12/28/2022] Open
Abstract
Predictive coding theories argue that recent experience establishes expectations in the brain that generate prediction errors when violated. Prediction errors provide a possible explanation for repetition suppression, where evoked neural activity is attenuated across repeated presentations of the same stimulus. The predictive coding account argues repetition suppression arises because repeated stimuli are expected, whereas non-repeated stimuli are unexpected and thus elicit larger neural responses. Here, we employed electroencephalography in humans to test the predictive coding account of repetition suppression by presenting sequences of visual gratings with orientations that were expected either to repeat or change in separate blocks of trials. We applied multivariate forward modelling to determine how orientation selectivity was affected by repetition and prediction. Unexpected stimuli were associated with significantly enhanced orientation selectivity, whereas selectivity was unaffected for repeated stimuli. Our results suggest that repetition suppression and expectation have separable effects on neural representations of visual feature information.
Collapse
Affiliation(s)
- Matthew F Tang
- Queensland Brain Institute, The University of Queensland, St Lucia, Australia.,Australian Research Council Centre of Excellence for Integrative Brain Function, Victoria, Australia
| | - Cooper A Smout
- Queensland Brain Institute, The University of Queensland, St Lucia, Australia.,Australian Research Council Centre of Excellence for Integrative Brain Function, Victoria, Australia
| | - Ehsan Arabzadeh
- Australian Research Council Centre of Excellence for Integrative Brain Function, Victoria, Australia.,Eccles Institute of Neuroscience, John Curtin School of Medical Research, The Australian National University, Canberra, Australia
| | - Jason B Mattingley
- Queensland Brain Institute, The University of Queensland, St Lucia, Australia.,Australian Research Council Centre of Excellence for Integrative Brain Function, Victoria, Australia.,School of Psychology, The University of Queensland, St Lucia, Australia
| |
Collapse
|
39
|
Sleep Strengthens Predictive Sequence Coding. J Neurosci 2018; 38:8989-9000. [PMID: 30185464 DOI: 10.1523/jneurosci.1352-18.2018] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2018] [Revised: 08/23/2018] [Accepted: 08/24/2018] [Indexed: 02/05/2023] Open
Abstract
Predictive-coding theories assume that perception and action are based on internal models derived from previous experience. Such internal models require selection and consolidation to be stored over time. Sleep is known to support memory consolidation. We hypothesized that sleep supports both consolidation and abstraction of an internal task model that is subsequently used to predict upcoming stimuli. Human subjects (of either sex) were trained on deterministic visual sequences and tested with interleaved deviant stimuli after retention intervals of sleep or wakefulness. Adopting a predictive-coding approach, we found increased prediction strength after sleep, as expressed by increased error rates to deviant stimuli, but fewer errors for the immediately following standard stimuli. Sleep likewise enhanced the formation of an abstract sequence model, independent of the temporal context during training. Moreover, sleep increased confidence for sequence knowledge, reflecting enhanced metacognitive access to the model. Our results suggest that sleep supports the formation of internal models which can be used to predict upcoming events in different contexts.SIGNIFICANCE STATEMENT To efficiently interact with the ever-changing world, we predict upcoming events based on similar previous experiences. Sleep is known to benefit memory consolidation. However, it is not clear whether sleep specifically supports the transformation of past experience into predictions of future events. Here, we find that, when human subjects sleep after learning a sequence of predictable visual events, they make better predictions about upcoming events compared with subjects who stayed awake for an equivalent period of time. In addition, sleep supports the transfer of such knowledge between different temporal contexts (i.e., when sequences unfold at different speeds). Thus, sleep supports perception and action by enhancing the predictive utility of previous experiences.
Collapse
|
40
|
Manahova ME, Mostert P, Kok P, Schoffelen JM, de Lange FP. Stimulus Familiarity and Expectation Jointly Modulate Neural Activity in the Visual Ventral Stream. J Cogn Neurosci 2018; 30:1366-1377. [DOI: 10.1162/jocn_a_01281] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Prior knowledge about the visual world can change how a visual stimulus is processed. Two forms of prior knowledge are often distinguished: stimulus familiarity (i.e., whether a stimulus has been seen before) and stimulus expectation (i.e., whether a stimulus is expected to occur, based on the context). Neurophysiological studies in monkeys have shown suppression of spiking activity both for expected and for familiar items in object-selective inferotemporal cortex. It is an open question, however, if and how these types of knowledge interact in their modulatory effects on the sensory response. To address this issue and to examine whether previous findings generalize to noninvasively measured neural activity in humans, we separately manipulated stimulus familiarity and expectation while noninvasively recording human brain activity using magnetoencephalography. We observed independent suppression of neural activity by familiarity and expectation, specifically in the lateral occipital complex, the putative human homologue of monkey inferotemporal cortex. Familiarity also led to sharpened response dynamics, which was predominantly observed in early visual cortex. Together, these results show that distinct types of sensory knowledge jointly determine the amount of neural resources dedicated to object processing in the visual ventral stream.
Collapse
Affiliation(s)
- Mariya E. Manahova
- Donders Institute for Brain, Cognition, and Behaviour, Radboud University Nijmegen
| | - Pim Mostert
- Donders Institute for Brain, Cognition, and Behaviour, Radboud University Nijmegen
| | | | | | - Floris P. de Lange
- Donders Institute for Brain, Cognition, and Behaviour, Radboud University Nijmegen
| |
Collapse
|
41
|
Suppressed Sensory Response to Predictable Object Stimuli throughout the Ventral Visual Stream. J Neurosci 2018; 38:7452-7461. [PMID: 30030402 DOI: 10.1523/jneurosci.3421-17.2018] [Citation(s) in RCA: 52] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2017] [Revised: 06/21/2018] [Accepted: 07/09/2018] [Indexed: 11/21/2022] Open
Abstract
Prediction plays a crucial role in perception, as prominently suggested by predictive coding theories. However, the exact form and mechanism of predictive modulations of sensory processing remain unclear, with some studies reporting a downregulation of the sensory response for predictable input whereas others observed an enhanced response. In a similar vein, downregulation of the sensory response for predictable input has been linked to either sharpening or dampening of the sensory representation, which are opposite in nature. In the present study, we set out to investigate the neural consequences of perceptual expectation of object stimuli throughout the visual hierarchy, using fMRI in human volunteers. Participants of both sexes were exposed to pairs of sequentially presented object images in a statistical learning paradigm, in which the first object predicted the identity of the second object. Image transitions were not task relevant; thus, all learning of statistical regularities was incidental. We found strong suppression of neural responses to expected compared with unexpected stimuli throughout the ventral visual stream, including primary visual cortex, lateral occipital complex, and anterior ventral visual areas. Expectation suppression in lateral occipital complex scaled positively with image preference and voxel selectivity, lending support to the dampening account of expectation suppression in object perception.SIGNIFICANCE STATEMENT It has been suggested that the brain fundamentally relies on predictions and constructs models of the world to make sense of sensory information. Previous research on the neural basis of prediction has documented suppressed neural responses to expected compared with unexpected stimuli. In the present study, we demonstrate robust expectation suppression throughout the entire ventral visual stream, and underlying this suppression a dampening of the sensory representation in object-selective visual cortex, but not in primary visual cortex. Together, our results provide novel evidence in support of theories conceptualizing perception as an active inference process, which selectively dampens cortical representations of predictable objects. This dampening may support our ability to automatically filter out irrelevant, predictable objects.
Collapse
|
42
|
Associative Prediction of Visual Shape in the Hippocampus. J Neurosci 2018; 38:6888-6899. [PMID: 29986875 DOI: 10.1523/jneurosci.0163-18.2018] [Citation(s) in RCA: 57] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2018] [Revised: 04/29/2018] [Accepted: 06/20/2018] [Indexed: 11/21/2022] Open
Abstract
Perception can be cast as a process of inference, in which bottom-up signals are combined with top-down predictions in sensory systems. In line with this, neural activity in sensory cortex is strongly modulated by prior expectations. Such top-down predictions often arise from cross-modal associations, such as when a sound (e.g., bell or bark) leads to an expectation of the visual appearance of the corresponding object (e.g., bicycle or dog). We hypothesized that the hippocampus, which rapidly learns arbitrary relationships between stimuli over space and time, may be involved in forming such associative predictions. We exposed male and female human participants to auditory cues predicting visual shapes, while measuring high-resolution fMRI signals in visual cortex and the hippocampus. Using multivariate reconstruction methods, we discovered a dissociation between these regions: representations in visual cortex were dominated by whichever shape was presented, whereas representations in the hippocampus reflected only which shape was predicted by the cue. The strength of hippocampal predictions correlated across participants with the amount of expectation-related facilitation in visual cortex. These findings help bridge the gap between memory and sensory systems in the human brain.SIGNIFICANCE STATEMENT The way we perceive the world is to a great extent determined by our prior knowledge. Despite this intimate link between perception and memory, these two aspects of cognition have mostly been studied in isolation. Here we investigate their interaction by asking how memory systems that encode and retrieve associations can inform perception. We find that upon hearing a familiar auditory cue, the hippocampus represents visual information that had previously co-occurred with the cue, even when this expectation differs from what is currently visible. Furthermore, the strength of this hippocampal expectation correlates with facilitation of perceptual processing in visual cortex. These findings help bridge the gap between memory and sensory systems in the human brain.
Collapse
|
43
|
Repetition suppression to objects is modulated by stimulus-specific expectations. Sci Rep 2017; 7:8781. [PMID: 28821808 PMCID: PMC5562860 DOI: 10.1038/s41598-017-09374-z] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2016] [Accepted: 07/25/2017] [Indexed: 01/01/2023] Open
Abstract
Repeated exposure to the same stimulus results in an attenuated brain response in cortical regions that are activated during the processing of that stimulus. This phenomenon, called repetition suppression (RS), has been shown to be modulated by expectation. Typically, this is achieved by varying the probability of stimulus repetitions (Prep) between blocks of an experiment, generating an abstract expectation that ‘things will repeat’. Here, we examined whether stimulus-specific expectations also modulate RS. We designed a task where expectation and repetition are manipulated independently, using stimulus-specific expectations. We investigated to which extent such stimulus-specific expectations modulated the visual evoked response to objects in lateral occipital cortex (LOC) and primary visual cortex (V1), using functional magnetic resonance imaging (fMRI). In LOC, we found that RS interacted with expectation, such that repetition suppression was more pronounced for unexpected relative to expected stimuli. Additionally, we found that the response of stimulus-preferring voxels in V1 was generally decreased when stimuli were expected. These results suggest that stimulus-specific expectations about objects modulate LOC and propagate back to the earliest cortical station processing visual input.
Collapse
|
44
|
Ramachandran S, Meyer T, Olson CR. Prediction suppression and surprise enhancement in monkey inferotemporal cortex. J Neurophysiol 2017; 118:374-382. [PMID: 28424293 DOI: 10.1152/jn.00136.2017] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2017] [Revised: 04/06/2017] [Accepted: 04/06/2017] [Indexed: 11/22/2022] Open
Abstract
Exposing monkeys, over the course of days and weeks, to pairs of images presented in fixed sequence, so that each leading image becomes a predictor for the corresponding trailing image, affects neuronal visual responsiveness in area TE. At the end of the training period, neurons respond relatively weakly to a trailing image when it appears in a trained sequence and, thus, confirms prediction, whereas they respond relatively strongly to the same image when it appears in an untrained sequence and, thus, violates prediction. This effect could arise from prediction suppression (reduced firing in response to the occurrence of a probable event) or surprise enhancement (elevated firing in response to the omission of a probable event). To identify its cause, we compared firing under the prediction-confirming and prediction-violating conditions to firing under a prediction-neutral condition. The results provide strong evidence for prediction suppression and limited evidence for surprise enhancement.NEW & NOTEWORTHY In predictive coding models of the visual system, neurons carry signed prediction error signals. We show here that monkey inferotemporal neurons exhibit prediction-modulated firing, as posited by these models, but that the signal is unsigned. The response to a prediction-confirming image is suppressed, and the response to a prediction-violating image may be enhanced. These results are better explained by a model in which the visual system emphasizes unpredicted events than by a predictive coding model.
Collapse
Affiliation(s)
- Suchitra Ramachandran
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, Pennsylvania; .,Department of Biological Sciences, Carnegie Mellon University, Pittsburgh, Pennsylvania.,Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland
| | - Travis Meyer
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, Pennsylvania
| | - Carl R Olson
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, Pennsylvania.,Department of Biological Sciences, Carnegie Mellon University, Pittsburgh, Pennsylvania.,Department of Neuroscience, University of Pittsburgh, Pittsburgh, Pennsylvania; and
| |
Collapse
|