1
|
Dijkstra N, van Gaal S, Geerligs L, Bosch SE, van Gerven MAJ. No Evidence for Neural Overlap between Unconsciously Processed and Imagined Stimuli. eNeuro 2021; 8:ENEURO.0228-21.2021. [PMID: 34593516 PMCID: PMC8577044 DOI: 10.1523/eneuro.0228-21.2021] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2021] [Revised: 09/01/2021] [Accepted: 09/02/2021] [Indexed: 11/23/2022] Open
Abstract
Visual representations can be generated via feedforward or feedback processes. The extent to which these processes result in overlapping representations remains unclear. Previous work has shown that imagined stimuli elicit similar representations as perceived stimuli throughout the visual cortex. However, while representations during imagery are indeed only caused by feedback processing, neural processing during perception is an interplay of both feedforward and feedback processing. This means that any representational overlap could be because of overlap in feedback processes. In the current study, we aimed to investigate this issue by characterizing the overlap between feedforward- and feedback-initiated category representations during imagined stimuli, conscious perception, and unconscious processing using fMRI in humans of either sex. While all three conditions elicited stimulus representations in left lateral occipital cortex (LOC), significant similarities were observed only between imagery and conscious perception in this area. Furthermore, connectivity analyses revealed stronger connectivity between frontal areas and left LOC during conscious perception and in imagery compared with unconscious processing. Together, these findings can be explained by the idea that long-range feedback modifies visual representations, thereby reducing representational overlap between purely feedforward- and feedback-initiated stimulus representations measured by fMRI. Neural representations influenced by feedback, either stimulus driven (perception) or purely internally driven (imagery), are, however, relatively similar.
Collapse
Affiliation(s)
- Nadine Dijkstra
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, 6500 GL, Nijmegen, The Netherlands
- Wellcome Centre for Human Neuroimaging, University College London, London WC1N 3AR, United Kingdom
| | - Simon van Gaal
- Department of Psychology, Brain & Cognition, University of Amsterdam, 1000 GG, Amsterdam, The Netherlands
| | - Linda Geerligs
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, 6500 GL, Nijmegen, The Netherlands
| | - Sander E Bosch
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, 6500 GL, Nijmegen, The Netherlands
| | - Marcel A J van Gerven
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, 6500 GL, Nijmegen, The Netherlands
| |
Collapse
|
2
|
Thielen J, Bosch SE, van Leeuwen TM, van Gerven MAJ, van Lier R. Evidence for confounding eye movements under attempted fixation and active viewing in cognitive neuroscience. Sci Rep 2019; 9:17456. [PMID: 31767911 PMCID: PMC6877555 DOI: 10.1038/s41598-019-54018-z] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2019] [Accepted: 11/08/2019] [Indexed: 12/02/2022] Open
Abstract
Eye movements can have serious confounding effects in cognitive neuroscience experiments. Therefore, participants are commonly asked to fixate. Regardless, participants will make so-called fixational eye movements under attempted fixation, which are thought to be necessary to prevent perceptual fading. Neural changes related to these eye movements could potentially explain previously reported neural decoding and neuroimaging results under attempted fixation. In previous work, under attempted fixation and passive viewing, we found no evidence for systematic eye movements. Here, however, we show that participants' eye movements are systematic under attempted fixation when active viewing is demanded by the task. Since eye movements directly affect early visual cortex activity, commonly used for neural decoding, our findings imply alternative explanations for previously reported results in neural decoding.
Collapse
Affiliation(s)
- Jordy Thielen
- Radboud University, Donders Centre for Cognition, Nijmegen, 6525 HR, The Netherlands.
| | - Sander E Bosch
- Radboud University, Donders Centre for Cognition, Nijmegen, 6525 HR, The Netherlands
| | - Tessa M van Leeuwen
- Radboud University, Donders Centre for Cognition, Nijmegen, 6525 HR, The Netherlands
| | - Marcel A J van Gerven
- Radboud University, Donders Centre for Cognition, Nijmegen, 6525 HR, The Netherlands
| | - Rob van Lier
- Radboud University, Donders Centre for Cognition, Nijmegen, 6525 HR, The Netherlands
| |
Collapse
|
3
|
Dijkstra N, Hinne M, Bosch SE, van Gerven MAJ. Between-subject variability in the influence of mental imagery on conscious perception. Sci Rep 2019; 9:15658. [PMID: 31666592 PMCID: PMC6821778 DOI: 10.1038/s41598-019-52072-1] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2019] [Accepted: 10/08/2019] [Indexed: 11/24/2022] Open
Abstract
Mental imagery and visual perception rely on similar neural mechanisms, but the function of this overlap remains unclear. One idea is that imagery can influence perception. Previous research has shown that imagining a stimulus prior to binocular presentation of rivalling stimuli increases the chance of perceiving the imagined stimulus. In this study we investigated how this effect interacts with bottom-up sensory input by comparing psychometric response curves for congruent and incongruent imagery in humans. A Bayesian hierarchical model was used, allowing us to simultaneously study group-level effects as well as effects for individual participants. We found strong effects of both imagery as well as its interaction with sensory evidence within individual participants. However, the direction of these effects were highly variable between individuals, leading to weak effects at the group level. This highlights the heterogeneity of conscious perception and emphasizes the need for individualized investigation of such complex cognitive processes.
Collapse
Affiliation(s)
- N Dijkstra
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands.
| | - M Hinne
- University of Amsterdam, Amsterdam, The Netherlands
| | - S E Bosch
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| | - M A J van Gerven
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| |
Collapse
|
4
|
Quax SC, Dijkstra N, van Staveren MJ, Bosch SE, van Gerven MA. Eye movements explain decodability during perception and cued attention in MEG. Neuroimage 2019; 195:444-453. [DOI: 10.1016/j.neuroimage.2019.03.069] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2018] [Revised: 03/27/2019] [Accepted: 03/28/2019] [Indexed: 10/27/2022] Open
|
5
|
Dijkstra N, Bosch SE, van Gerven MA. Shared Neural Mechanisms of Visual Perception and Imagery. Trends Cogn Sci 2019; 23:423-434. [DOI: 10.1016/j.tics.2019.02.004] [Citation(s) in RCA: 86] [Impact Index Per Article: 17.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2018] [Revised: 02/07/2019] [Accepted: 02/20/2019] [Indexed: 12/16/2022]
|
6
|
Thielen J, Bosch SE, van Leeuwen TM, van Gerven MAJ, van Lier R. Neuroimaging Findings on Amodal Completion: A Review. Iperception 2019; 10:2041669519840047. [PMID: 31007887 PMCID: PMC6457032 DOI: 10.1177/2041669519840047] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2018] [Accepted: 02/20/2019] [Indexed: 12/03/2022] Open
Abstract
Amodal completion is the phenomenon of perceiving completed objects even though physically they are partially occluded. In this review, we provide an extensive overview of the results obtained from a variety of neuroimaging studies on the neural correlates of amodal completion. We discuss whether low-level and high-level cortical areas are implicated in amodal completion; provide an overview of how amodal completion unfolds over time while dissociating feedforward, recurrent, and feedback processes; and discuss how amodal completion is represented at the neuronal level. The involvement of low-level visual areas such as V1 and V2 is not yet clear, while several high-level structures such as the lateral occipital complex and fusiform face area seem invariant to occlusion of objects and faces, respectively, and several motor areas seem to code for object permanence. The variety of results on the timing of amodal completion hints to a mixture of feedforward, recurrent, and feedback processes. We discuss whether the invisible parts of the occluded object are represented as if they were visible, contrary to a high-level representation. While plenty of questions on amodal completion remain, this review presents an overview of the neuroimaging findings reported to date, summarizes several insights from computational models, and connects research of other perceptual completion processes such as modal completion. In all, it is suggested that amodal completion is the solution to deal with various types of incomplete retinal information, and highly depends on stimulus complexity and saliency, and therefore also give rise to a variety of observed neural patterns.
Collapse
Affiliation(s)
- Jordy Thielen
- Radboud University, Donders Institute for Brain,
Cognition and Behaviour, Nijmegen, the Netherlands
| | - Sander E. Bosch
- Radboud University, Donders Institute for Brain,
Cognition and Behaviour, Nijmegen, the Netherlands
| | - Tessa M. van Leeuwen
- Radboud University, Donders Institute for Brain,
Cognition and Behaviour, Nijmegen, the Netherlands
| | - Marcel A. J. van Gerven
- Radboud University, Donders Institute for Brain,
Cognition and Behaviour, Nijmegen, the Netherlands
| | - Rob van Lier
- Radboud University, Donders Institute for Brain,
Cognition and Behaviour, Nijmegen, the Netherlands
| |
Collapse
|
7
|
Seeliger K, Fritsche M, Güçlü U, Schoenmakers S, Schoffelen JM, Bosch SE, van Gerven MAJ. Convolutional neural network-based encoding and decoding of visual object recognition in space and time. Neuroimage 2017; 180:253-266. [PMID: 28723578 DOI: 10.1016/j.neuroimage.2017.07.018] [Citation(s) in RCA: 40] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2017] [Revised: 06/24/2017] [Accepted: 07/10/2017] [Indexed: 10/19/2022] Open
Abstract
Representations learned by deep convolutional neural networks (CNNs) for object recognition are a widely investigated model of the processing hierarchy in the human visual system. Using functional magnetic resonance imaging, CNN representations of visual stimuli have previously been shown to correspond to processing stages in the ventral and dorsal streams of the visual system. Whether this correspondence between models and brain signals also holds for activity acquired at high temporal resolution has been explored less exhaustively. Here, we addressed this question by combining CNN-based encoding models with magnetoencephalography (MEG). Human participants passively viewed 1,000 images of objects while MEG signals were acquired. We modelled their high temporal resolution source-reconstructed cortical activity with CNNs, and observed a feed-forward sweep across the visual hierarchy between 75 and 200 ms after stimulus onset. This spatiotemporal cascade was captured by the network layer representations, where the increasingly abstract stimulus representation in the hierarchical network model was reflected in different parts of the visual cortex, following the visual ventral stream. We further validated the accuracy of our encoding model by decoding stimulus identity in a left-out validation set of viewed objects, achieving state-of-the-art decoding accuracy.
Collapse
Affiliation(s)
- K Seeliger
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Montessorilaan 3, 6525 HR Nijmegen, The Netherlands.
| | - M Fritsche
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Montessorilaan 3, 6525 HR Nijmegen, The Netherlands
| | - U Güçlü
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Montessorilaan 3, 6525 HR Nijmegen, The Netherlands
| | - S Schoenmakers
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Montessorilaan 3, 6525 HR Nijmegen, The Netherlands
| | - J-M Schoffelen
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Montessorilaan 3, 6525 HR Nijmegen, The Netherlands
| | - S E Bosch
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Montessorilaan 3, 6525 HR Nijmegen, The Netherlands
| | - M A J van Gerven
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Montessorilaan 3, 6525 HR Nijmegen, The Netherlands
| |
Collapse
|
8
|
Backus AR, Bosch SE, Ekman M, Grabovetsky AV, Doeller CF. Mnemonic convergence in the human hippocampus. Nat Commun 2016; 7:11991. [PMID: 27325442 PMCID: PMC4919533 DOI: 10.1038/ncomms11991] [Citation(s) in RCA: 46] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2015] [Accepted: 05/19/2016] [Indexed: 11/21/2022] Open
Abstract
The ability to form associations between a multitude of events is the hallmark of episodic memory. Computational models have espoused the importance of the hippocampus as convergence zone, binding different aspects of an episode into a coherent representation, by integrating information from multiple brain regions. However, evidence for this long-held hypothesis is limited, since previous work has largely focused on representational and network properties of the hippocampus in isolation. Here we identify the hippocampus as mnemonic convergence zone, using a combination of multivariate pattern and graph-theoretical network analyses of functional magnetic resonance imaging data from humans performing an associative memory task. We observe overlap of conjunctive coding and hub-like network attributes in the hippocampus. These results provide evidence for mnemonic convergence in the hippocampus, underlying the integration of distributed information into episodic memory representations. The ability to form associations between events is the hallmark of episodic memory and is thought to involve the hippocampus. Here the authors use a combination of multivariate pattern and graph theoretical network analyses of functional imaging data in humans, and show conjunctive coding and hub-like network attributes in the hippocampus.
Collapse
Affiliation(s)
- Alexander R Backus
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen 6525 EN, The Netherlands
| | - Sander E Bosch
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen 6525 EN, The Netherlands
| | - Matthias Ekman
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen 6525 EN, The Netherlands
| | | | - Christian F Doeller
- Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen 6525 EN, The Netherlands
| |
Collapse
|