1
|
De Cesarei A, Sambuco N, D’Ascenzo S, Nicoletti R, Codispoti M. Delayed ocular disengagement from arousing scenes. Front Psychol 2023; 14:1297192. [PMID: 38179488 PMCID: PMC10764442 DOI: 10.3389/fpsyg.2023.1297192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Accepted: 12/04/2023] [Indexed: 01/06/2024] Open
Abstract
Visual exploration of the world is supported by eye movements which can be speeded up or delayed depending on bottom-up stimulation, top-down goals, and prior associations. Previous studies observed faster initiation of saccades toward emotional than neutral natural scenes; however, less is known concerning saccades which originate from emotional, compared with neutral, scenes. Here, we addressed this issue by examining a task in which participants continuously moved their gaze from and toward pictures (natural scenes), which could be emotional or neutral, and changed position in every trial. Saccades were initiated later when the starting picture was emotional compared to neutral, and this slowing was associated with the arousal value of the picture, suggesting that ocular disengagement does not vary with stimulus valence but is affected by engaging picture contents such as erotica and threat/injuries.
Collapse
Affiliation(s)
| | - Nicola Sambuco
- Department of Psychology, University of Bologna, Bologna, Italy
| | - Stefania D’Ascenzo
- Department of Philosophy and Communication Studies, University of Bologna, Bologna, Italy
| | - Roberto Nicoletti
- Department of Philosophy and Communication Studies, University of Bologna, Bologna, Italy
| | | |
Collapse
|
2
|
Greene MR, Trivedi D. Spatial Scene Memories Are Biased Towards a Fixed Amount of Semantic Information. Open Mind (Camb) 2023; 7:445-459. [PMID: 37637297 PMCID: PMC10449403 DOI: 10.1162/opmi_a_00088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Accepted: 06/03/2023] [Indexed: 08/29/2023] Open
Abstract
Scene memory has known spatial biases. Boundary extension is a well-known bias whereby observers remember visual information beyond an image's boundaries. While recent studies demonstrate that boundary contraction also reliably occurs based on intrinsic image properties, the specific properties that drive the effect are unknown. This study assesses the extent to which scene memory might have a fixed capacity for information. We assessed both visual and semantic information in a scene database using techniques from image processing and natural language processing, respectively. We then assessed how both types of information predicted memory errors for scene boundaries using a standard rapid serial visual presentation (RSVP) forced error paradigm. A linear regression model indicated that memories for scene boundaries were significantly predicted by semantic, but not visual, information and that this effect persisted when scene depth was considered. Boundary extension was observed for images with low semantic information, and contraction was observed for images with high semantic information. This suggests a cognitive process that normalizes the amount of semantic information held in memory.
Collapse
Affiliation(s)
- Michelle R. Greene
- Bates College, Program in Neuroscience
- Barnard College, Columbia University
| | | |
Collapse
|
3
|
Ignatious E, Azam S, Jonkman M, De Boer F. Frequency and Time Domain Analysis of EEG Based Auditory Evoked Potentials to Detect Binaural Hearing in Noise. J Clin Med 2023; 12:4487. [PMID: 37445522 DOI: 10.3390/jcm12134487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Revised: 06/27/2023] [Accepted: 06/30/2023] [Indexed: 07/15/2023] Open
Abstract
Hearing loss is a prevalent health issue that affects individuals worldwide. Binaural hearing refers to the ability to integrate information received simultaneously from both ears, allowing individuals to identify, locate, and separate sound sources. Auditory evoked potentials (AEPs) refer to the electrical responses that are generated within any part of the auditory system in response to auditory stimuli presented externally. Electroencephalography (EEG) is a non-invasive technology used for the monitoring of AEPs. This research aims to investigate the use of audiometric EEGs as an objective method to detect specific features of binaural hearing with frequency and time domain analysis techniques. Thirty-five subjects with normal hearing and a mean age of 27.35 participated in the research. The stimuli used in the current study were designed to investigate the impact of binaural phase shifts of the auditory stimuli in the presence of noise. The frequency domain and time domain analyses provided statistically significant and promising novel findings. The study utilized Blackman windowed 18 ms and 48 ms pure tones as stimuli, embedded in noise maskers, of frequencies 125 Hz, 250 Hz, 500 Hz, 750 Hz, 1000 Hz in homophasic (the same phase in both ears) and antiphasic (180-degree phase difference between the two ears) conditions. The study focuses on the effect of phase reversal of auditory stimuli in noise of the middle latency response (MLR) and late latency response (LLR) regions of the AEPs. The frequency domain analysis revealed a significant difference in the frequency bands of 20 to 25 Hz and 25 to 30 Hz when elicited by antiphasic and homophasic stimuli of 500 Hz for MLRs and 500 Hz and 250 Hz for LLRs. The time domain analysis identified the Na peak of the MLR for 500 Hz, the N1 peak of the LLR for 500 Hz stimuli and the P300 peak of the LLR for 250 Hz as significant potential markers in detecting binaural processing in the brain.
Collapse
Affiliation(s)
- Eva Ignatious
- College of Engineering and IT, Charles Darwin University, Casuarina 0810, Australia
| | - Sami Azam
- College of Engineering and IT, Charles Darwin University, Casuarina 0810, Australia
| | - Mirjam Jonkman
- College of Engineering and IT, Charles Darwin University, Casuarina 0810, Australia
| | - Friso De Boer
- College of Engineering and IT, Charles Darwin University, Casuarina 0810, Australia
| |
Collapse
|
4
|
Woo T, Liang X, Evans DA, Fernandez O, Kretschmer F, Reiter S, Laurent G. The dynamics of pattern matching in camouflaging cuttlefish. Nature 2023:10.1038/s41586-023-06259-2. [PMID: 37380772 PMCID: PMC10322717 DOI: 10.1038/s41586-023-06259-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2021] [Accepted: 05/22/2023] [Indexed: 06/30/2023]
Abstract
Many cephalopods escape detection using camouflage1. This behaviour relies on a visual assessment of the surroundings, on an interpretation of visual-texture statistics2-4 and on matching these statistics using millions of skin chromatophores that are controlled by motoneurons located in the brain5-7. Analysis of cuttlefish images proposed that camouflage patterns are low dimensional and categorizable into three pattern classes, built from a small repertoire of components8-11. Behavioural experiments also indicated that, although camouflage requires vision, its execution does not require feedback5,12,13, suggesting that motion within skin-pattern space is stereotyped and lacks the possibility of correction. Here, using quantitative methods14, we studied camouflage in the cuttlefish Sepia officinalis as behavioural motion towards background matching in skin-pattern space. An analysis of hundreds of thousands of images over natural and artificial backgrounds revealed that the space of skin patterns is high-dimensional and that pattern matching is not stereotyped-each search meanders through skin-pattern space, decelerating and accelerating repeatedly before stabilizing. Chromatophores could be grouped into pattern components on the basis of their covariation during camouflaging. These components varied in shapes and sizes, and overlay one another. However, their identities varied even across transitions between identical skin-pattern pairs, indicating flexibility of implementation and absence of stereotypy. Components could also be differentiated by their sensitivity to spatial frequency. Finally, we compared camouflage to blanching, a skin-lightening reaction to threatening stimuli. Pattern motion during blanching was direct and fast, consistent with open-loop motion in low-dimensional pattern space, in contrast to that observed during camouflage.
Collapse
Affiliation(s)
- Theodosia Woo
- Max Planck Institute for Brain Research, Frankfurt, Germany
| | - Xitong Liang
- Max Planck Institute for Brain Research, Frankfurt, Germany
- School of Life Sciences, Peking University, Beijing, China
| | | | | | | | - Sam Reiter
- Max Planck Institute for Brain Research, Frankfurt, Germany.
- Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan.
| | - Gilles Laurent
- Max Planck Institute for Brain Research, Frankfurt, Germany.
| |
Collapse
|
5
|
Henderson MM, Tarr MJ, Wehbe L. A Texture Statistics Encoding Model Reveals Hierarchical Feature Selectivity across Human Visual Cortex. J Neurosci 2023; 43:4144-4161. [PMID: 37127366 PMCID: PMC10255092 DOI: 10.1523/jneurosci.1822-22.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Revised: 03/21/2023] [Accepted: 03/26/2023] [Indexed: 05/03/2023] Open
Abstract
Midlevel features, such as contour and texture, provide a computational link between low- and high-level visual representations. Although the nature of midlevel representations in the brain is not fully understood, past work has suggested a texture statistics model, called the P-S model (Portilla and Simoncelli, 2000), is a candidate for predicting neural responses in areas V1-V4 as well as human behavioral data. However, it is not currently known how well this model accounts for the responses of higher visual cortex to natural scene images. To examine this, we constructed single-voxel encoding models based on P-S statistics and fit the models to fMRI data from human subjects (both sexes) from the Natural Scenes Dataset (Allen et al., 2022). We demonstrate that the texture statistics encoding model can predict the held-out responses of individual voxels in early retinotopic areas and higher-level category-selective areas. The ability of the model to reliably predict signal in higher visual cortex suggests that the representation of texture statistics features is widespread throughout the brain. Furthermore, using variance partitioning analyses, we identify which features are most uniquely predictive of brain responses and show that the contributions of higher-order texture features increase from early areas to higher areas on the ventral and lateral surfaces. We also demonstrate that patterns of sensitivity to texture statistics can be used to recover broad organizational axes within visual cortex, including dimensions that capture semantic image content. These results provide a key step forward in characterizing how midlevel feature representations emerge hierarchically across the visual system.SIGNIFICANCE STATEMENT Intermediate visual features, like texture, play an important role in cortical computations and may contribute to tasks like object and scene recognition. Here, we used a texture model proposed in past work to construct encoding models that predict the responses of neural populations in human visual cortex (measured with fMRI) to natural scene stimuli. We show that responses of neural populations at multiple levels of the visual system can be predicted by this model, and that the model is able to reveal an increase in the complexity of feature representations from early retinotopic cortex to higher areas of ventral and lateral visual cortex. These results support the idea that texture-like representations may play a broad underlying role in visual processing.
Collapse
Affiliation(s)
- Margaret M Henderson
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
- Department of Psychology
- Machine Learning Department, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
| | - Michael J Tarr
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
- Department of Psychology
- Machine Learning Department, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
| | - Leila Wehbe
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
- Department of Psychology
- Machine Learning Department, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
| |
Collapse
|
6
|
Orima T, Motoyoshi I. Analysis and Synthesis of Natural Texture Perception From Visual Evoked Potentials. Front Neurosci 2021; 15:698940. [PMID: 34381330 PMCID: PMC8350323 DOI: 10.3389/fnins.2021.698940] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2021] [Accepted: 06/21/2021] [Indexed: 11/13/2022] Open
Abstract
The primate visual system analyzes statistical information in natural images and uses it for the immediate perception of scenes, objects, and surface materials. To investigate the dynamical encoding of image statistics in the human brain, we measured visual evoked potentials (VEPs) for 166 natural textures and their synthetic versions, and performed a reverse-correlation analysis of the VEPs and representative texture statistics of the image. The analysis revealed occipital VEP components strongly correlated with particular texture statistics. VEPs correlated with low-level statistics, such as subband SDs, emerged rapidly from 100 to 250 ms in a spatial frequency dependent manner. VEPs correlated with higher-order statistics, such as subband kurtosis and cross-band correlations, were observed at slightly later times. Moreover, these robust correlations enabled us to inversely estimate texture statistics from VEP signals via linear regression and to reconstruct texture images that appear similar to those synthesized with the original statistics. Additionally, we found significant differences in VEPs at 200-300 ms between some natural textures and their Portilla-Simoncelli (PS) synthesized versions, even though they shared almost identical texture statistics. This differential VEP was related to the perceptual "unnaturalness" of PS-synthesized textures. These results suggest that the visual cortex rapidly encodes image statistics hidden in natural textures specifically enough to predict the visual appearance of a texture, while it also represents high-level information beyond image statistics, and that electroencephalography can be used to decode these cortical signals.
Collapse
Affiliation(s)
- Taiki Orima
- Department of Life Sciences, The University of Tokyo, Tokyo, Japan.,Japan Society for the Promotion of Science, Tokyo, Japan
| | - Isamu Motoyoshi
- Department of Life Sciences, The University of Tokyo, Tokyo, Japan
| |
Collapse
|
7
|
Papale P, Leo A, Handjaras G, Cecchetti L, Pietrini P, Ricciardi E. Shape coding in occipito-temporal cortex relies on object silhouette, curvature, and medial axis. J Neurophysiol 2020; 124:1560-1570. [PMID: 33052726 DOI: 10.1152/jn.00212.2020] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023] Open
Abstract
Object recognition relies on different transformations of the retinal input, carried out by the visual system, that range from local contrast to object shape and category. While some of those transformations are thought to occur at specific stages of the visual hierarchy, the features they represent are correlated (e.g., object shape and identity) and selectivity for the same feature overlaps in many brain regions. This may be explained either by collinearity across representations or may instead reflect the coding of multiple dimensions by the same cortical population. Moreover, orthogonal and shared components may differently impact distinctive stages of the visual hierarchy. We recorded functional MRI activity while participants passively attended to object images and employed a statistical approach that partitioned orthogonal and shared object representations to reveal their relative impact on brain processing. Orthogonal shape representations (silhouette, curvature, and medial axis) independently explained distinct and overlapping clusters of selectivity in the occitotemporal and parietal cortex. Moreover, we show that the relevance of shared representations linearly increases moving from posterior to anterior regions. These results indicate that the visual cortex encodes shared relations between different features in a topographic fashion and that object shape is encoded along different dimensions, each representing orthogonal features.NEW & NOTEWORTHY There are several possible ways of characterizing the shape of an object. Which shape description better describes our brain responses while we passively perceive objects? Here, we employed three competing shape models to explain brain representations when viewing real objects. We found that object shape is encoded in a multidimensional fashion and thus defined by the interaction of multiple features.
Collapse
Affiliation(s)
- Paolo Papale
- Molecular Mind Laboratory, IMT School for Advanced Studies Lucca, Italy.,Department of Vision and Cognition, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands
| | - Andrea Leo
- Molecular Mind Laboratory, IMT School for Advanced Studies Lucca, Italy.,Department of Translational Research and Advanced Technologies in Medicine and Surgery, University of Pisa, Pisa, Italy
| | - Giacomo Handjaras
- Molecular Mind Laboratory, IMT School for Advanced Studies Lucca, Italy
| | - Luca Cecchetti
- Molecular Mind Laboratory, IMT School for Advanced Studies Lucca, Italy
| | - Pietro Pietrini
- Molecular Mind Laboratory, IMT School for Advanced Studies Lucca, Italy
| | | |
Collapse
|
8
|
Bayet L, Zinszer BD, Reilly E, Cataldo JK, Pruitt Z, Cichy RM, Nelson CA, Aslin RN. Temporal dynamics of visual representations in the infant brain. Dev Cogn Neurosci 2020; 45:100860. [PMID: 32932205 PMCID: PMC7498752 DOI: 10.1016/j.dcn.2020.100860] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2020] [Revised: 08/21/2020] [Accepted: 09/04/2020] [Indexed: 12/12/2022] Open
Abstract
Tools from computational neuroscience have facilitated the investigation of the neural correlates of mental representations. However, access to the representational content of neural activations early in life has remained limited. We asked whether patterns of neural activity elicited by complex visual stimuli (animals, human body) could be decoded from EEG data gathered from 12-15-month-old infants and adult controls. We assessed pairwise classification accuracy at each time-point after stimulus onset, for individual infants and adults. Classification accuracies rose above chance in both groups, within 500 ms. In contrast to adults, neural representations in infants were not linearly separable across visual domains. Representations were similar within, but not across, age groups. These findings suggest a developmental reorganization of visual representations between the second year of life and adulthood and provide a promising proof-of-concept for the feasibility of decoding EEG data within-subject to assess how the infant brain dynamically represents visual objects.
Collapse
Affiliation(s)
- Laurie Bayet
- Department of Psychology, American University, Washington, DC, 20016, USA; Center for Neuroscience and Behavior, American University, Washington, DC, 20016, USA.
| | - Benjamin D Zinszer
- Department of Psychology, Princeton University, Princeton, NJ, 08540, USA; Department of Linguistics and Cognitive Science, University of Delaware, Newark, DE, 19716, USA
| | - Emily Reilly
- Boston Children's Hospital, Boston, MA, 02115, USA
| | | | - Zoe Pruitt
- Department of Brain and Cognitive Science, University of Rochester, Rochester, NY, 14627, USA
| | - Radoslaw M Cichy
- Department of Education and Psychology, Freie Universität Berlin, 14195, Berlin, Germany
| | - Charles A Nelson
- Boston Children's Hospital, Boston, MA, 02115, USA; Department of Pediatrics, Harvard Medical School, Boston, MA, 02115, USA; Graduate School of Education, Harvard, Cambridge, MA, 02138, USA
| | - Richard N Aslin
- Haskins Laboratories, 300 George Street, New Haven, CT, 06511, USA; Psychological Sciences Department, University of Connecticut, Storrs, CT, 06269, USA; Department of Psychology, Yale University, New Haven, CT, 06511, USA; Yale Child Study Center, School of Medicine, New Haven, CT, 06519, USA
| |
Collapse
|
9
|
Seijdel N, Jahfari S, Groen IIA, Scholte HS. Low-level image statistics in natural scenes influence perceptual decision-making. Sci Rep 2020; 10:10573. [PMID: 32601499 PMCID: PMC7324621 DOI: 10.1038/s41598-020-67661-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2020] [Accepted: 06/08/2020] [Indexed: 11/10/2022] Open
Abstract
A fundamental component of interacting with our environment is gathering and interpretation of sensory information. When investigating how perceptual information influences decision-making, most researchers have relied on manipulated or unnatural information as perceptual input, resulting in findings that may not generalize to real-world scenes. Unlike simplified, artificial stimuli, real-world scenes contain low-level regularities that are informative about the structural complexity, which the brain could exploit. In this study, participants performed an animal detection task on low, medium or high complexity scenes as determined by two biologically plausible natural scene statistics, contrast energy (CE) or spatial coherence (SC). In experiment 1, stimuli were sampled such that CE and SC both influenced scene complexity. Diffusion modelling showed that the speed of information processing was affected by low-level scene complexity. Experiment 2a/b refined these observations by showing how isolated manipulation of SC resulted in weaker but comparable effects, with an additional change in response boundary, whereas manipulation of only CE had no effect. Overall, performance was best for scenes with intermediate complexity. Our systematic definition quantifies how natural scene complexity interacts with decision-making. We speculate that CE and SC serve as an indication to adjust perceptual decision-making based on the complexity of the input.
Collapse
Affiliation(s)
- Noor Seijdel
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands. .,Amsterdam Brain and Cognition (ABC) Center, University of Amsterdam, Amsterdam, The Netherlands.
| | - Sara Jahfari
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands.,Spinoza Centre for Neuroimaging, Royal Netherlands Academy of Arts and Sciences (KNAW), Amsterdam, The Netherlands
| | - Iris I A Groen
- Department of Psychology, New York University, New York, USA
| | - H Steven Scholte
- Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands.,Amsterdam Brain and Cognition (ABC) Center, University of Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
10
|
Disentangling the Independent Contributions of Visual and Conceptual Features to the Spatiotemporal Dynamics of Scene Categorization. J Neurosci 2020; 40:5283-5299. [PMID: 32467356 DOI: 10.1523/jneurosci.2088-19.2020] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2019] [Revised: 04/18/2020] [Accepted: 04/23/2020] [Indexed: 11/21/2022] Open
Abstract
Human scene categorization is characterized by its remarkable speed. While many visual and conceptual features have been linked to this ability, significant correlations exist between feature spaces, impeding our ability to determine their relative contributions to scene categorization. Here, we used a whitening transformation to decorrelate a variety of visual and conceptual features and assess the time course of their unique contributions to scene categorization. Participants (both sexes) viewed 2250 full-color scene images drawn from 30 different scene categories while having their brain activity measured through 256-channel EEG. We examined the variance explained at each electrode and time point of visual event-related potential (vERP) data from nine different whitened encoding models. These ranged from low-level features obtained from filter outputs to high-level conceptual features requiring human annotation. The amount of category information in the vERPs was assessed through multivariate decoding methods. Behavioral similarity measures were obtained in separate crowdsourced experiments. We found that all nine models together contributed 78% of the variance of human scene similarity assessments and were within the noise ceiling of the vERP data. Low-level models explained earlier vERP variability (88 ms after image onset), whereas high-level models explained later variance (169 ms). Critically, only high-level models shared vERP variability with behavior. Together, these results suggest that scene categorization is primarily a high-level process, but reliant on previously extracted low-level features.SIGNIFICANCE STATEMENT In a single fixation, we glean enough information to describe a general scene category. Many types of features are associated with scene categories, ranging from low-level properties, such as colors and contours, to high-level properties, such as objects and attributes. Because these properties are correlated, it is difficult to understand each property's unique contributions to scene categorization. This work uses a whitening transformation to remove the correlations between features and examines the extent to which each feature contributes to visual event-related potentials over time. We found that low-level visual features contributed first but were not correlated with categorization behavior. High-level features followed 80 ms later, providing key insights into how the brain makes sense of a complex visual world.
Collapse
|
11
|
Harel A, Mzozoyana MW, Al Zoubi H, Nador JD, Noesen BT, Lowe MX, Cant JS. Artificially-generated scenes demonstrate the importance of global scene properties for scene perception. Neuropsychologia 2020; 141:107434. [PMID: 32179102 DOI: 10.1016/j.neuropsychologia.2020.107434] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2019] [Revised: 03/04/2020] [Accepted: 03/09/2020] [Indexed: 10/24/2022]
Abstract
Recent electrophysiological research highlights the significance of global scene properties (GSPs) for scene perception. However, since real-world scenes span a range of low-level stimulus properties and high-level contextual semantics, GSP effects may also reflect additional processing of such non-global factors. We examined this question by asking whether Event-Related Potentials (ERPs) to GSPs will still be observed when specific low- and high-level scene properties are absent from the scene. We presented participants with computer-based artificially-manipulated scenes varying in two GSPs (spatial expanse and naturalness) which minimized other sources of scene information (color and semantic object detail). We found that the peak amplitude of the P2 component was sensitive to the spatial expanse and naturalness of the artificially-generated scenes: P2 amplitude was higher to closed than open scenes, and in response to manmade than natural scenes. A control experiment showed that the effect of Naturalness on the P2 is not driven by local texture information, while earlier effects of naturalness, expressed as a modulation of the P1 and N1 amplitudes, are sensitive to texture information. Our results demonstrate that GSPs are processed robustly around 220 ms and that P2 can be used as an index of global scene perception.
Collapse
Affiliation(s)
- Assaf Harel
- Department of Psychology, Wright State University, Dayton, OH, USA.
| | - Mavuso W Mzozoyana
- Department of Neuroscience, Cell Biology and Physiology, Wright State University, Dayton, OH, USA
| | - Hamada Al Zoubi
- Department of Neuroscience, Cell Biology and Physiology, Wright State University, Dayton, OH, USA
| | - Jeffrey D Nador
- Department of Psychology, Wright State University, Dayton, OH, USA
| | - Birken T Noesen
- Department of Psychology, Wright State University, Dayton, OH, USA
| | - Matthew X Lowe
- Computer Science and Artificial Intelligence Lab, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Jonathan S Cant
- Department of Psychology, University of Toronto Scarborough, Toronto, ON, Canada
| |
Collapse
|
12
|
Scene Representations Conveyed by Cortical Feedback to Early Visual Cortex Can Be Described by Line Drawings. J Neurosci 2019; 39:9410-9423. [PMID: 31611306 PMCID: PMC6867807 DOI: 10.1523/jneurosci.0852-19.2019] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2019] [Revised: 08/27/2019] [Accepted: 09/23/2019] [Indexed: 11/25/2022] Open
Abstract
Human behavior is dependent on the ability of neuronal circuits to predict the outside world. Neuronal circuits in early visual areas make these predictions based on internal models that are delivered via non-feedforward connections. Despite our extensive knowledge of the feedforward sensory features that drive cortical neurons, we have a limited grasp on the structure of the brain's internal models. Progress in neuroscience therefore depends on our ability to replicate the models that the brain creates internally. Here we record human fMRI data while presenting partially occluded visual scenes. Visual occlusion allows us to experimentally control sensory input to subregions of visual cortex while internal models continue to influence activity in these regions. Because the observed activity is dependent on internal models, but not on sensory input, we have the opportunity to map visual features conveyed by the brain's internal models. Our results show that activity related to internal models in early visual cortex are more related to scene-specific features than to categorical or depth features. We further demonstrate that behavioral line drawings provide a good description of internal model structure representing scene-specific features. These findings extend our understanding of internal models, showing that line drawings provide a window into our brains' internal models of vision. SIGNIFICANCE STATEMENT We find that fMRI activity patterns corresponding to occluded visual information in early visual cortex fill in scene-specific features. Line drawings of the missing scene information correlate with our recorded activity patterns, and thus to internal models. Despite our extensive knowledge of the sensory features that drive cortical neurons, we have a limited grasp on the structure of our brains' internal models. These results therefore constitute an advance to the field of neuroscience by extending our knowledge about the models that our brains construct to efficiently represent and predict the world. Moreover, they link a behavioral measure to these internal models, which play an active role in many components of human behavior, including visual predictions, action planning, and decision making.
Collapse
|
13
|
Conscious perception of natural images is constrained by category-related visual features. Nat Commun 2019; 10:4106. [PMID: 31511514 PMCID: PMC6739304 DOI: 10.1038/s41467-019-12135-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2019] [Accepted: 08/22/2019] [Indexed: 11/08/2022] Open
Abstract
Conscious perception is crucial for adaptive behaviour yet access to consciousness varies for different types of objects. The visual system comprises regions with widely distributed category information and exemplar-level representations that cluster according to category. Does this categorical organisation in the brain provide insight into object-specific access to consciousness? We address this question using the Attentional Blink approach with visual objects as targets. We find large differences across categories in the attentional blink. We then employ activation patterns extracted from a deep convolutional neural network to reveal that these differences depend on mid- to high-level, rather than low-level, visual features. We further show that these visual features can be used to explain variance in performance across trials. Taken together, our results suggest that the specific organisation of the higher-tier visual system underlies important functions relevant for conscious perception of differing natural images.
Collapse
|
14
|
De Cesarei A, Cavicchi S, Micucci A, Codispoti M. Categorization Goals Modulate the Use of Natural Scene Statistics. J Cogn Neurosci 2019; 31:109-125. [DOI: 10.1162/jocn_a_01333] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Understanding natural scenes involves the contribution of bottom–up analysis and top–down modulatory processes. However, the interaction of these processes during the categorization of natural scenes is not well understood. In the current study, we approached this issue using ERPs and behavioral and computational data. We presented pictures of natural scenes and asked participants to categorize them in response to different questions (Is it an animal/vehicle? Is it indoors/outdoors? Are there one/two foreground elements?). ERPs for target scenes requiring a “yes” response began to differ from those of nontarget scenes, beginning at 250 msec from picture onset, and this ERP difference was unmodulated by the categorization questions. Earlier ERPs showed category-specific differences (e.g., between animals and vehicles), which were associated with the processing of scene statistics. From 180 msec after scene onset, these category-specific ERP differences were modulated by the categorization question that was asked. Categorization goals do not modulate only later stages associated with target/nontarget decision but also earlier perceptual stages, which are involved in the processing of scene statistics.
Collapse
|
15
|
Greene MR. The information content of scene categories. PSYCHOLOGY OF LEARNING AND MOTIVATION 2019. [DOI: 10.1016/bs.plm.2019.03.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
16
|
Groen IIA, Jahfari S, Seijdel N, Ghebreab S, Lamme VAF, Scholte HS. Scene complexity modulates degree of feedback activity during object detection in natural scenes. PLoS Comput Biol 2018; 14:e1006690. [PMID: 30596644 PMCID: PMC6329519 DOI: 10.1371/journal.pcbi.1006690] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2018] [Revised: 01/11/2019] [Accepted: 12/01/2018] [Indexed: 02/06/2023] Open
Abstract
Selective brain responses to objects arise within a few hundreds of milliseconds of neural processing, suggesting that visual object recognition is mediated by rapid feed-forward activations. Yet disruption of neural responses in early visual cortex beyond feed-forward processing stages affects object recognition performance. Here, we unite these discrepant findings by reporting that object recognition involves enhanced feedback activity (recurrent processing within early visual cortex) when target objects are embedded in natural scenes that are characterized by high complexity. Human participants performed an animal target detection task on natural scenes with low, medium or high complexity as determined by a computational model of low-level contrast statistics. Three converging lines of evidence indicate that feedback was selectively enhanced for high complexity scenes. First, functional magnetic resonance imaging (fMRI) activity in early visual cortex (V1) was enhanced for target objects in scenes with high, but not low or medium complexity. Second, event-related potentials (ERPs) evoked by target objects were selectively enhanced at feedback stages of visual processing (from ~220 ms onwards) for high complexity scenes only. Third, behavioral performance for high complexity scenes deteriorated when participants were pressed for time and thus less able to incorporate the feedback activity. Modeling of the reaction time distributions using drift diffusion revealed that object information accumulated more slowly for high complexity scenes, with evidence accumulation being coupled to trial-to-trial variation in the EEG feedback response. Together, these results suggest that while feed-forward activity may suffice to recognize isolated objects, the brain employs recurrent processing more adaptively in naturalistic settings, using minimal feedback for simple scenes and increasing feedback for complex scenes.
Collapse
Affiliation(s)
- Iris I. A. Groen
- New York University, Department of Psychology, New York, New York, United States of America
| | - Sara Jahfari
- Spinoza Centre for Neuroimaging, Royal Netherlands Academy of Arts and Sciences (KNAW), Amsterdam, The Netherlands
- University of Amsterdam, Department of Psychology, Section Brain and Cognition, Amsterdam, The Netherlands
| | - Noor Seijdel
- University of Amsterdam, Department of Psychology, Section Brain and Cognition, Amsterdam, The Netherlands
| | - Sennay Ghebreab
- University of Amsterdam, Department of Psychology, Section Brain and Cognition, Amsterdam, The Netherlands
- University of Amsterdam, Department of Informatics, Intelligent Systems Lab, Amsterdam, The Netherlands
| | - Victor A. F. Lamme
- University of Amsterdam, Department of Psychology, Section Brain and Cognition, Amsterdam, The Netherlands
| | - H. Steven Scholte
- University of Amsterdam, Department of Psychology, Section Brain and Cognition, Amsterdam, The Netherlands
| |
Collapse
|
17
|
The temporal evolution of conceptual object representations revealed through models of behavior, semantics and deep neural networks. Neuroimage 2018; 178:172-182. [DOI: 10.1016/j.neuroimage.2018.05.037] [Citation(s) in RCA: 42] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2017] [Revised: 05/10/2018] [Accepted: 05/14/2018] [Indexed: 11/18/2022] Open
|
18
|
Groen II, Greene MR, Baldassano C, Fei-Fei L, Beck DM, Baker CI. Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior. eLife 2018. [PMID: 29513219 PMCID: PMC5860866 DOI: 10.7554/elife.32962] [Citation(s) in RCA: 76] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022] Open
Abstract
Inherent correlations between visual and semantic features in real-world scenes make it difficult to determine how different scene properties contribute to neural representations. Here, we assessed the contributions of multiple properties to scene representation by partitioning the variance explained in human behavioral and brain measurements by three feature models whose inter-correlations were minimized a priori through stimulus preselection. Behavioral assessments of scene similarity reflected unique contributions from a functional feature model indicating potential actions in scenes as well as high-level visual features from a deep neural network (DNN). In contrast, similarity of cortical responses in scene-selective areas was uniquely explained by mid- and high-level DNN features only, while an object label model did not contribute uniquely to either domain. The striking dissociation between functional and DNN features in their contribution to behavioral and brain representations of scenes indicates that scene-selective cortex represents only a subset of behaviorally relevant scene information.
Collapse
Affiliation(s)
- Iris Ia Groen
- Laboratory of Brain and Cognition, National Institutes of Health, Bethesda, United States.,Department of Psychology, New York University, New York City, United States
| | | | | | - Li Fei-Fei
- Stanford Vision Lab, Stanford University, Stanford, United States
| | - Diane M Beck
- Department of Psychology, University of Illinois, Urbana-Champaign, United States.,Beckman Institute, University of Illinois, Urbana-Champaign, United States
| | - Chris I Baker
- Laboratory of Brain and Cognition, National Institutes of Health, Bethesda, United States
| |
Collapse
|
19
|
Mensen A, Marshall W, Tononi G. EEG Differentiation Analysis and Stimulus Set Meaningfulness. Front Psychol 2017; 8:1748. [PMID: 29056921 PMCID: PMC5635725 DOI: 10.3389/fpsyg.2017.01748] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2017] [Accepted: 09/21/2017] [Indexed: 11/13/2022] Open
Abstract
A set of images can be considered as meaningfully different for an observer if they can be distinguished phenomenally from one another. Each phenomenal difference must be supported by some neurophysiological differences. Differentiation analysis aims to quantify neurophysiological differentiation evoked by a given set of stimuli to assess its meaningfulness to the individual observer. As a proof of concept using high-density EEG, we show increased neurophysiological differentiation for a set of natural, meaningfully different images in contrast to another set of artificially generated, meaninglessly different images in nine participants. Stimulus-evoked neurophysiological differentiation (over 257 channels, 800 ms) was systematically greater for meaningful vs. meaningless stimulus categories both at the group level and for individual subjects. Spatial breakdown showed a central-posterior peak of differentiation, consistent with the visual nature of the stimulus sets. Temporal breakdown revealed an early peak of differentiation around 110 ms, prominent in the central-posterior region; and a later, longer-lasting peak at 300-500 ms that was spatially more distributed. The early peak of differentiation was not accompanied by changes in mean ERP amplitude, whereas the later peak was associated with a higher amplitude ERP for meaningful images. An ERP component similar to visual-awareness-negativity occurred during the nadir of differentiation across all image types. Control stimulus sets and further analysis indicate that changes in neurophysiological differentiation between meaningful and meaningless stimulus sets could not be accounted for by spatial properties of the stimuli or by stimulus novelty and predictability.
Collapse
Affiliation(s)
- Armand Mensen
- Center for Sleep and Consciousness, University of Wisconsin-Madison, Madison, WI, United States.,Department of Neurology, Inselspital Bern, Bern, Switzerland
| | - William Marshall
- Center for Sleep and Consciousness, University of Wisconsin-Madison, Madison, WI, United States
| | - Giulio Tononi
- Center for Sleep and Consciousness, University of Wisconsin-Madison, Madison, WI, United States
| |
Collapse
|
20
|
Goddard E, Klein C, Solomon SG, Hogendoorn H, Carlson TA. Interpreting the dimensions of neural feature representations revealed by dimensionality reduction. Neuroimage 2017; 180:41-67. [PMID: 28663068 DOI: 10.1016/j.neuroimage.2017.06.068] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2017] [Accepted: 06/23/2017] [Indexed: 10/19/2022] Open
Abstract
Recent progress in understanding the structure of neural representations in the cerebral cortex has centred around the application of multivariate classification analyses to measurements of brain activity. These analyses have proved a sensitive test of whether given brain regions provide information about specific perceptual or cognitive processes. An exciting extension of this approach is to infer the structure of this information, thereby drawing conclusions about the underlying neural representational space. These approaches rely on exploratory data-driven dimensionality reduction to extract the natural dimensions of neural spaces, including natural visual object and scene representations, semantic and conceptual knowledge, and working memory. However, the efficacy of these exploratory methods is unknown, because they have only been applied to representations in brain areas for which we have little or no secondary knowledge. One of the best-understood areas of the cerebral cortex is area MT of primate visual cortex, which is known to be important in motion analysis. To assess the effectiveness of dimensionality reduction for recovering neural representational space we applied several dimensionality reduction methods to multielectrode measurements of spiking activity obtained from area MT of marmoset monkeys, made while systematically varying the motion direction and speed of moving stimuli. Despite robust tuning at individual electrodes, and high classifier performance, dimensionality reduction rarely revealed dimensions for direction and speed. We use this example to illustrate important limitations of these analyses, and suggest a framework for how to best apply such methods to data where the structure of the neural representation is unknown.
Collapse
Affiliation(s)
- Erin Goddard
- McGill Vision Research, Dept of Ophthalmology, McGill University, Montreal, QC, H3G 1A4, Canada; School of Psychology, University of Sydney, Sydney, NSW, 2006, Australia; ARC Centre of Excellence in Cognition and Its Disorders (CCD), Macquarie University, Sydney, NSW, 2109, Australia.
| | - Colin Klein
- ARC Centre of Excellence in Cognition and Its Disorders (CCD), Macquarie University, Sydney, NSW, 2109, Australia; Department of Philosophy, Macquarie University, Sydney, NSW, 2109, Australia
| | - Samuel G Solomon
- Department of Experimental Psychology, University College London, Gower Street, London, WC1E 6BT, United Kingdom
| | - Hinze Hogendoorn
- School of Psychology, University of Sydney, Sydney, NSW, 2006, Australia; Helmholtz Institute, Neuroscience & Cognition Utrecht, Experimental Psychology Division, Utrecht University, Utrecht, The Netherlands
| | - Thomas A Carlson
- School of Psychology, University of Sydney, Sydney, NSW, 2006, Australia; ARC Centre of Excellence in Cognition and Its Disorders (CCD), Macquarie University, Sydney, NSW, 2109, Australia
| |
Collapse
|
21
|
De Cesarei A, Loftus GR, Mastria S, Codispoti M. Understanding natural scenes: Contributions of image statistics. Neurosci Biobehav Rev 2017; 74:44-57. [DOI: 10.1016/j.neubiorev.2017.01.012] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2016] [Revised: 01/05/2017] [Accepted: 01/09/2017] [Indexed: 10/20/2022]
|
22
|
Groen IIA, Silson EH, Baker CI. Contributions of low- and high-level properties to neural processing of visual scenes in the human brain. Philos Trans R Soc Lond B Biol Sci 2017; 372:rstb.2016.0102. [PMID: 28044013 DOI: 10.1098/rstb.2016.0102] [Citation(s) in RCA: 87] [Impact Index Per Article: 12.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/20/2016] [Indexed: 11/12/2022] Open
Abstract
Visual scene analysis in humans has been characterized by the presence of regions in extrastriate cortex that are selectively responsive to scenes compared with objects or faces. While these regions have often been interpreted as representing high-level properties of scenes (e.g. category), they also exhibit substantial sensitivity to low-level (e.g. spatial frequency) and mid-level (e.g. spatial layout) properties, and it is unclear how these disparate findings can be united in a single framework. In this opinion piece, we suggest that this problem can be resolved by questioning the utility of the classical low- to high-level framework of visual perception for scene processing, and discuss why low- and mid-level properties may be particularly diagnostic for the behavioural goals specific to scene perception as compared to object recognition. In particular, we highlight the contributions of low-level vision to scene representation by reviewing (i) retinotopic biases and receptive field properties of scene-selective regions and (ii) the temporal dynamics of scene perception that demonstrate overlap of low- and mid-level feature representations with those of scene category. We discuss the relevance of these findings for scene perception and suggest a more expansive framework for visual scene analysis.This article is part of the themed issue 'Auditory and visual scene analysis'.
Collapse
Affiliation(s)
- Iris I A Groen
- Laboratory of Brain and Cognition, National Institutes of Health, 10 Center Drive 10-3N228, Bethesda, MD, USA
| | - Edward H Silson
- Laboratory of Brain and Cognition, National Institutes of Health, 10 Center Drive 10-3N228, Bethesda, MD, USA
| | - Chris I Baker
- Laboratory of Brain and Cognition, National Institutes of Health, 10 Center Drive 10-3N228, Bethesda, MD, USA
| |
Collapse
|
23
|
Ghodrati M, Ghodousi M, Yoonessi A. Low-Level Contrast Statistics of Natural Images Can Modulate the Frequency of Event-Related Potentials (ERP) in Humans. Front Hum Neurosci 2016; 10:630. [PMID: 28018197 PMCID: PMC5145888 DOI: 10.3389/fnhum.2016.00630] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2016] [Accepted: 11/25/2016] [Indexed: 11/20/2022] Open
Abstract
Humans are fast and accurate in categorizing complex natural images. It is, however, unclear what features of visual information are exploited by brain to perceive the images with such speed and accuracy. It has been shown that low-level contrast statistics of natural scenes can explain the variance of amplitude of event-related potentials (ERP) in response to rapidly presented images. In this study, we investigated the effect of these statistics on frequency content of ERPs. We recorded ERPs from human subjects, while they viewed natural images each presented for 70 ms. Our results showed that Weibull contrast statistics, as a biologically plausible model, explained the variance of ERPs the best, compared to other image statistics that we assessed. Our time-frequency analysis revealed a significant correlation between these statistics and ERPs' power within theta frequency band (~3–7 Hz). This is interesting, as theta band is believed to be involved in context updating and semantic encoding. This correlation became significant at ~110 ms after stimulus onset, and peaked at 138 ms. Our results show that not only the amplitude but also the frequency of neural responses can be modulated with low-level contrast statistics of natural images and highlights their potential role in scene perception.
Collapse
Affiliation(s)
- Masoud Ghodrati
- Department of Physiology, Monash UniversityClayton, VIC, Australia; Neuroscience Program, Biomedicine Discovery Institute, Monash UniversityClayton, VIC, Australia
| | - Mahrad Ghodousi
- Department of Neuroscience, School of Advanced Technologies in Medicine, Tehran University of Medical Sciences Tehran, Iran
| | - Ali Yoonessi
- Department of Neuroscience, School of Advanced Technologies in Medicine, Tehran University of Medical Sciences Tehran, Iran
| |
Collapse
|
24
|
Zhu W, Drewes J, Peatfield NA, Melcher D. Differential Visual Processing of Animal Images, with and without Conscious Awareness. Front Hum Neurosci 2016; 10:513. [PMID: 27790106 PMCID: PMC5061858 DOI: 10.3389/fnhum.2016.00513] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2016] [Accepted: 09/27/2016] [Indexed: 12/02/2022] Open
Abstract
The human visual system can quickly and efficiently extract categorical information from a complex natural scene. The rapid detection of animals in a scene is one compelling example of this phenomenon, and it suggests the automatic processing of at least some types of categories with little or no attentional requirements (Li et al., 2002, 2005). The aim of this study is to investigate whether the remarkable capability to categorize complex natural scenes exist in the absence of awareness, based on recent reports that “invisible” stimuli, which do not reach conscious awareness, can still be processed by the human visual system (Pasley et al., 2004; Williams et al., 2004; Fang and He, 2005; Jiang et al., 2006, 2007; Kaunitz et al., 2011a). In two experiments, we recorded event-related potentials (ERPs) in response to animal and non-animal/vehicle stimuli in both aware and unaware conditions in a continuous flash suppression (CFS) paradigm. Our results indicate that even in the “unseen” condition, the brain responds differently to animal and non-animal/vehicle images, consistent with rapid activation of animal-selective feature detectors prior to, or outside of, suppression by the CFS mask.
Collapse
Affiliation(s)
- Weina Zhu
- School of Information Science, Yunnan UniversityKunming, China; Department of Psychology, Giessen UniversityGiessen, Germany; Center for Mind/Brain Sciences (CIMeC), University of TrentoRovereto, Italy; Kunming Institute of Zoology, Chinese Academy of SciencesKunming, China
| | - Jan Drewes
- Center for Mind/Brain Sciences (CIMeC), University of Trento Rovereto, Italy
| | - Nicholas A Peatfield
- Department of Biomedical Physiology and Kinesiology, Simon Fraser University Burnaby, BC, Canada
| | - David Melcher
- Center for Mind/Brain Sciences (CIMeC), University of Trento Rovereto, Italy
| |
Collapse
|
25
|
Making Sense of Real-World Scenes. Trends Cogn Sci 2016; 20:843-856. [PMID: 27769727 DOI: 10.1016/j.tics.2016.09.003] [Citation(s) in RCA: 82] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2016] [Revised: 09/06/2016] [Accepted: 09/06/2016] [Indexed: 11/23/2022]
Abstract
To interact with the world, we have to make sense of the continuous sensory input conveying information about our environment. A recent surge of studies has investigated the processes enabling scene understanding, using increasingly complex stimuli and sophisticated analyses to highlight the visual features and brain regions involved. However, there are two major challenges to producing a comprehensive framework for scene understanding. First, scene perception is highly dynamic, subserving multiple behavioral goals. Second, a multitude of different visual properties co-occur across scenes and may be correlated or independent. We synthesize the recent literature and argue that for a complete view of scene understanding, it is necessary to account for both differing observer goals and the contribution of diverse scene properties.
Collapse
|
26
|
The Temporal Dynamics of Scene Processing: A Multifaceted EEG Investigation. eNeuro 2016; 3:eN-NWR-0139-16. [PMID: 27699208 PMCID: PMC5037322 DOI: 10.1523/eneuro.0139-16.2016] [Citation(s) in RCA: 37] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2016] [Revised: 08/12/2016] [Accepted: 09/06/2016] [Indexed: 11/25/2022] Open
Abstract
Our remarkable ability to process complex visual scenes is supported by a network of scene-selective cortical regions. Despite growing knowledge about the scene representation in these regions, much less is known about the temporal dynamics with which these representations emerge. We conducted two experiments aimed at identifying and characterizing the earliest markers of scene-specific processing. In the first experiment, human participants viewed images of scenes, faces, and everyday objects while event-related potentials (ERPs) were recorded. We found that the first ERP component to evince a significantly stronger response to scenes than the other categories was the P2, peaking ∼220 ms after stimulus onset. To establish that the P2 component reflects scene-specific processing, in the second experiment, we recorded ERPs while the participants viewed diverse real-world scenes spanning the following three global scene properties: spatial expanse (open/closed), relative distance (near/far), and naturalness (man-made/natural). We found that P2 amplitude was sensitive to these scene properties at both the categorical level, distinguishing between open and closed natural scenes, as well as at the single-image level, reflecting both computationally derived scene statistics and behavioral ratings of naturalness and spatial expanse. Together, these results establish the P2 as an ERP marker for scene processing, and demonstrate that scene-specific global information is available in the neural response as early as 220 ms.
Collapse
|
27
|
Abstract
Naturalistic textures with an intermediate degree of statistical regularity can capture key structural features of natural images (Freeman and Simoncelli, 2011). V2 and later visual areas are sensitive to these features, while primary visual cortex is not (Freeman et al., 2013). Here we expand on this work by investigating a class of textures that have maximal formal regularity, the 17 crystallographic wallpaper groups (Fedorov, 1891). We used texture stimuli from four of the groups that differ in the maximum order of rotation symmetry they contain, and measured neural responses in human participants using functional MRI and high-density EEG. We found that cortical area V3 has a parametric representation of the rotation symmetries in the textures that is not present in either V1 or V2, the first discovery of a stimulus property that differentiates processing in V3 from that of lower-level areas. Parametric responses were also seen in higher-order ventral stream areas V4, VO1, and lateral occipital complex (LOC), but not in dorsal stream areas. The parametric response pattern was replicated in the EEG data, and source localization indicated that responses in V3 and V4 lead responses in LOC, which is consistent with a feedforward mechanism. Finally, we presented our stimuli to four well developed feedforward models and found that none of them were able to account for our results. Our results highlight structural regularity as an important stimulus dimension for distinguishing the early stages of visual processing, and suggest a previously unrecognized role for V3 in the visual form-processing hierarchy. Significance statement: Hierarchical processing is a fundamental organizing principle in visual neuroscience, with each successive processing stage being sensitive to increasingly complex stimulus properties. Here, we probe the encoding hierarchy in human visual cortex using a class of visual textures--wallpaper patterns--that are maximally regular. Through a combination of fMRI and EEG source imaging, we find specific responses to texture regularity that depend parametrically on the maximum order of rotation symmetry in the textures. These parametric responses are seen in several areas of the ventral visual processing stream, as well as in area V3, but not in V1 or V2. This is the first demonstration of a stimulus property that differentiates processing in V3 from that of lower-level visual areas.
Collapse
|
28
|
Perceptual similarity of visual patterns predicts dynamic neural activation patterns measured with MEG. Neuroimage 2016; 132:59-70. [DOI: 10.1016/j.neuroimage.2016.02.019] [Citation(s) in RCA: 68] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2015] [Revised: 02/05/2016] [Accepted: 02/09/2016] [Indexed: 12/14/2022] Open
|
29
|
Ramkumar P, Hansen BC, Pannasch S, Loschky LC. Visual information representation and rapid-scene categorization are simultaneous across cortex: An MEG study. Neuroimage 2016; 134:295-304. [PMID: 27001497 DOI: 10.1016/j.neuroimage.2016.03.027] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2015] [Revised: 03/04/2016] [Accepted: 03/13/2016] [Indexed: 11/17/2022] Open
Abstract
Perceiving the visual world around us requires the brain to represent the features of stimuli and to categorize the stimulus based on these features. Incorrect categorization can result either from errors in visual representation or from errors in processes that lead to categorical choice. To understand the temporal relationship between the neural signatures of such systematic errors, we recorded whole-scalp magnetoencephalography (MEG) data from human subjects performing a rapid-scene categorization task. We built scene category decoders based on (1) spatiotemporally resolved neural activity, (2) spatial envelope (SpEn) image features, and (3) behavioral responses. Using confusion matrices, we tracked how well the pattern of errors from neural decoders could be explained by SpEn decoders and behavioral errors, over time and across cortical areas. Across the visual cortex and the medial temporal lobe, we found that both SpEn and behavioral errors explained unique variance in the errors of neural decoders. Critically, these effects were nearly simultaneous, and most prominent between 100 and 250ms after stimulus onset. Thus, during rapid-scene categorization, neural processes that ultimately result in behavioral categorization are simultaneous and co-localized with neural processes underlying visual information representation.
Collapse
Affiliation(s)
- Pavan Ramkumar
- Brain Research Unit, O.V. Lounasmaa Laboratory, Aalto University School of Science, Espoo, Finland.
| | - Bruce C Hansen
- Department of Psychology and Neuroscience Program, Colgate University, Hamilton, NY, USA.
| | - Sebastian Pannasch
- Brain Research Unit, O.V. Lounasmaa Laboratory, Aalto University School of Science, Espoo, Finland; Department of Psychology, Technische Universtät Dresden, Dresden, Germany.
| | - Lester C Loschky
- Department of Psychological Sciences, Kansas State University, Manhattan, KS, USA.
| |
Collapse
|
30
|
Groen IIA, Ghebreab S, Lamme VAF, Scholte HS. The time course of natural scene perception with reduced attention. J Neurophysiol 2016; 115:931-46. [DOI: 10.1152/jn.00896.2015] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2015] [Accepted: 11/09/2015] [Indexed: 11/22/2022] Open
Abstract
Attention is thought to impose an informational bottleneck on vision by selecting particular information from visual scenes for enhanced processing. Behavioral evidence suggests, however, that some scene information is extracted even when attention is directed elsewhere. Here, we investigated the neural correlates of this ability by examining how attention affects electrophysiological markers of scene perception. In two electro-encephalography (EEG) experiments, human subjects categorized real-world scenes as manmade or natural (full attention condition) or performed tasks on unrelated stimuli in the center or periphery of the scenes (reduced attention conditions). Scene processing was examined in two ways: traditional trial averaging was used to assess the presence of a categorical manmade/natural distinction in event-related potentials, whereas single-trial analyses assessed whether EEG activity was modulated by scene statistics that are diagnostic of naturalness of individual scenes. The results indicated that evoked activity up to 250 ms was unaffected by reduced attention, showing intact categorical differences between manmade and natural scenes and strong modulations of single-trial activity by scene statistics in all conditions. Thus initial processing of both categorical and individual scene information remained intact with reduced attention. Importantly, however, attention did have profound effects on later evoked activity; full attention on the scene resulted in prolonged manmade/natural differences, increased neural sensitivity to scene statistics, and enhanced scene memory. These results show that initial processing of real-world scene information is intact with diminished attention but that the depth of processing of this information does depend on attention.
Collapse
Affiliation(s)
- Iris I. A. Groen
- Amsterdam Brain and Cognition Center, Department of Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands
| | - Sennay Ghebreab
- Amsterdam Brain and Cognition Center, Department of Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands
- Intelligent Systems Lab Amsterdam, Institute of Informatics, University of Amsterdam, Amsterdam, The Netherlands
| | - Victor A. F. Lamme
- Amsterdam Brain and Cognition Center, Department of Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands
| | - H. Steven Scholte
- Amsterdam Brain and Cognition Center, Department of Brain and Cognition, University of Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
31
|
Bieniek MM, Bennett PJ, Sekuler AB, Rousselet GA. A robust and representative lower bound on object processing speed in humans. Eur J Neurosci 2015; 44:1804-14. [PMID: 26469359 PMCID: PMC4982026 DOI: 10.1111/ejn.13100] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2015] [Revised: 10/06/2015] [Accepted: 10/10/2015] [Indexed: 11/29/2022]
Abstract
How early does the brain decode object categories? Addressing this question is critical to constrain the type of neuronal architecture supporting object categorization. In this context, much effort has been devoted to estimating face processing speed. With onsets estimated from 50 to 150 ms, the timing of the first face-sensitive responses in humans remains controversial. This controversy is due partially to the susceptibility of dynamic brain measurements to filtering distortions and analysis issues. Here, using distributions of single-trial event-related potentials (ERPs), causal filtering, statistical analyses at all electrodes and time points, and effective correction for multiple comparisons, we present evidence that the earliest categorical differences start around 90 ms following stimulus presentation. These results were obtained from a representative group of 120 participants, aged 18-81, who categorized images of faces and noise textures. The results were reliable across testing days, as determined by test-retest assessment in 74 of the participants. Furthermore, a control experiment showed similar ERP onsets for contrasts involving images of houses or white noise. Face onsets did not change with age, suggesting that face sensitivity occurs within 100 ms across the adult lifespan. Finally, the simplicity of the face-texture contrast, and the dominant midline distribution of the effects, suggest the face responses were evoked by relatively simple image properties and are not face specific. Our results provide a new lower benchmark for the earliest neuronal responses to complex objects in the human visual system.
Collapse
Affiliation(s)
- Magdalena M Bieniek
- Institute of Neuroscience and Psychology, College of Medical, Veterinary and Life Sciences, University of Glasgow, 58 Hillhead Street, Glasgow, G12 8QB, UK
| | - Patrick J Bennett
- Department of Psychology, Neuroscience and Behaviour, McMaster University, Hamilton, ON, Canada
| | - Allison B Sekuler
- Department of Psychology, Neuroscience and Behaviour, McMaster University, Hamilton, ON, Canada
| | - Guillaume A Rousselet
- Institute of Neuroscience and Psychology, College of Medical, Veterinary and Life Sciences, University of Glasgow, 58 Hillhead Street, Glasgow, G12 8QB, UK
| |
Collapse
|
32
|
Sofer I, Crouzet SM, Serre T. Explaining the Timing of Natural Scene Understanding with a Computational Model of Perceptual Categorization. PLoS Comput Biol 2015; 11:e1004456. [PMID: 26335683 PMCID: PMC4559373 DOI: 10.1371/journal.pcbi.1004456] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2014] [Accepted: 07/19/2015] [Indexed: 11/18/2022] Open
Abstract
Observers can rapidly perform a variety of visual tasks such as categorizing a scene as open, as outdoor, or as a beach. Although we know that different tasks are typically associated with systematic differences in behavioral responses, to date, little is known about the underlying mechanisms. Here, we implemented a single integrated paradigm that links perceptual processes with categorization processes. Using a large image database of natural scenes, we trained machine-learning classifiers to derive quantitative measures of task-specific perceptual discriminability based on the distance between individual images and different categorization boundaries. We showed that the resulting discriminability measure accurately predicts variations in behavioral responses across categorization tasks and stimulus sets. We further used the model to design an experiment, which challenged previous interpretations of the so-called "superordinate advantage." Overall, our study suggests that observed differences in behavioral responses across rapid categorization tasks reflect natural variations in perceptual discriminability.
Collapse
Affiliation(s)
- Imri Sofer
- Cognitive, Linguistic & Psychological Sciences Department, Brown Institute for Brain Science, Brown University, Providence, Rhode Island, United States of America
| | - Sébastien M. Crouzet
- Cognitive, Linguistic & Psychological Sciences Department, Brown Institute for Brain Science, Brown University, Providence, Rhode Island, United States of America
| | - Thomas Serre
- Cognitive, Linguistic & Psychological Sciences Department, Brown Institute for Brain Science, Brown University, Providence, Rhode Island, United States of America
| |
Collapse
|
33
|
Can the Outputs of LGN Y-Cells Support Emotion Recognition? A Computational Study. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2015; 2015:695921. [PMID: 26175756 PMCID: PMC4484845 DOI: 10.1155/2015/695921] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/08/2015] [Accepted: 05/26/2015] [Indexed: 11/22/2022]
Abstract
It has been suggested that emotional visual input is processed along both a slower cortical pathway and a faster subcortical pathway which comprises the lateral geniculate nucleus (LGN), the superior colliculus, the pulvinar, and finally the amygdala. However, anatomical as well as functional evidence concerning the subcortical route is lacking. Here, we adopt a computational approach in order to investigate whether the visual representation that is achieved in the LGN may support emotion recognition and emotional response along the subcortical route. In four experiments, we show that the outputs of LGN Y-cells support neither facial expression categorization nor the same/different expression matching by an artificial classificator. However, the same classificator is able to perform at an above chance level in a statistics-based categorization of scenes containing animals and scenes containing people and of light and dark patterns. It is concluded that the visual representation achieved in the LGN is insufficient to allow for the recognition of emotional facial expression.
Collapse
|
34
|
Boly M, Sasai S, Gosseries O, Oizumi M, Casali A, Massimini M, Tononi G. Stimulus set meaningfulness and neurophysiological differentiation: a functional magnetic resonance imaging study. PLoS One 2015; 10:e0125337. [PMID: 25970444 PMCID: PMC4430458 DOI: 10.1371/journal.pone.0125337] [Citation(s) in RCA: 48] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2014] [Accepted: 03/11/2015] [Indexed: 11/19/2022] Open
Abstract
A meaningful set of stimuli, such as a sequence of frames from a movie, triggers a set of different experiences. By contrast, a meaningless set of stimuli, such as a sequence of ‘TV noise’ frames, triggers always the same experience—of seeing ‘TV noise’—even though the stimuli themselves are as different from each other as the movie frames. We reasoned that the differentiation of cortical responses underlying the subject’s experiences, as measured by Lempel-Ziv complexity (incompressibility) of functional MRI images, should reflect the overall meaningfulness of a set of stimuli for the subject, rather than differences among the stimuli. We tested this hypothesis by quantifying the differentiation of brain activity patterns in response to a movie sequence, to the same movie scrambled in time, and to ‘TV noise’, where the pixels from each movie frame were scrambled in space. While overall cortical activation was strong and widespread in all conditions, the differentiation (Lempel-Ziv complexity) of brain activation patterns was correlated with the meaningfulness of the stimulus set, being highest in the movie condition, intermediate in the scrambled movie condition, and minimal for ‘TV noise’. Stimulus set meaningfulness was also associated with higher information integration among cortical regions. These results suggest that the differentiation of neural responses can be used to assess the meaningfulness of a given set of stimuli for a given subject, without the need to identify the features and categories that are relevant to the subject, nor the precise location of selective neural responses.
Collapse
Affiliation(s)
- Melanie Boly
- Department of Neurology, University of Wisconsin, Madison, Wisconsin, United States of America
- Department of Psychiatry, University of Wisconsin, Madison, Wisconsin, United States of America
- * E-mail: (MB); (GT)
| | - Shuntaro Sasai
- Department of Psychiatry, University of Wisconsin, Madison, Wisconsin, United States of America
| | - Olivia Gosseries
- Department of Psychiatry, University of Wisconsin, Madison, Wisconsin, United States of America
- Coma Science Group, Cyclotron Research Centre and Department of Neurology, University of Liege and CHU Sart Tilman Hospital, Liege, Belgium
| | - Masafumi Oizumi
- Department of Psychiatry, University of Wisconsin, Madison, Wisconsin, United States of America
- Riken Brain Science Institute, Tokyo, Japan
| | - Adenauer Casali
- Department of Biomedical and Clinical Sciences ‘Luigi Sacco’, University of Milan, Milan, Italy
| | - Marcello Massimini
- Department of Biomedical and Clinical Sciences ‘Luigi Sacco’, University of Milan, Milan, Italy
- Instituto Di Ricovero e Cura a Carattere Scientifico, Fondazione Don Carlo Gnocchi, Milan, Italy
| | - Giulio Tononi
- Department of Psychiatry, University of Wisconsin, Madison, Wisconsin, United States of America
- * E-mail: (MB); (GT)
| |
Collapse
|
35
|
Russ BE, Leopold DA. Functional MRI mapping of dynamic visual features during natural viewing in the macaque. Neuroimage 2015; 109:84-94. [PMID: 25579448 DOI: 10.1016/j.neuroimage.2015.01.012] [Citation(s) in RCA: 66] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2014] [Revised: 10/24/2014] [Accepted: 01/05/2015] [Indexed: 10/24/2022] Open
Abstract
The ventral visual pathway of the primate brain is specialized to respond to stimuli in certain categories, such as the well-studied face selective patches in the macaque inferotemporal cortex. To what extent does response selectivity determined using brief presentations of isolated stimuli predict activity during the free viewing of a natural, dynamic scene, where features are superimposed in space and time? To approach this question, we obtained fMRI activity from the brains of three macaques viewing extended video clips containing a range of social and nonsocial content and compared the fMRI time courses to a family of feature models derived from the movie content. Starting with more than two dozen feature models extracted from each movie, we created functional maps based on features whose time courses were nearly orthogonal, focusing primarily on faces, motion content, and contrast level. Activity mapping using the face feature model readily yielded functional regions closely resembling face patches obtained using a block design in the same animals. Overall, the motion feature model dominated responses in nearly all visually driven areas, including the face patches as well as ventral visual areas V4, TEO, and TE. Control experiments presenting dynamic movies, whose content was free of animals, demonstrated that biological movement critically contributed to the predominance of motion in fMRI responses. These results highlight the value of natural viewing paradigms for studying the brain's functional organization and also underscore the paramount contribution of magnocellular input to the ventral visual pathway during natural vision.
Collapse
Affiliation(s)
- Brian E Russ
- Section on Cognitive Neurophysiology and Imaging, Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD20892, United States.
| | - David A Leopold
- Section on Cognitive Neurophysiology and Imaging, Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD20892, United States; Neurophysiology Imaging Facility, National Institute of Mental Health, National Institute of Neurological Disorders and Stroke, National Eye Institute, National Institutes of Health, Bethesda, MD20892, United States
| |
Collapse
|
36
|
Groen II, Ghebreab S, Prins H, Lamme VA, Scholte HS. From image statistics to scene gist: evoked neural activity reveals transition from low-level natural image structure to scene category. J Neurosci 2013; 33:18814-24. [PMID: 24285888 PMCID: PMC6618700 DOI: 10.1523/jneurosci.3128-13.2013] [Citation(s) in RCA: 66] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2013] [Revised: 10/07/2013] [Accepted: 10/24/2013] [Indexed: 11/21/2022] Open
Abstract
The visual system processes natural scenes in a split second. Part of this process is the extraction of "gist," a global first impression. It is unclear, however, how the human visual system computes this information. Here, we show that, when human observers categorize global information in real-world scenes, the brain exhibits strong sensitivity to low-level summary statistics. Subjects rated a specific instance of a global scene property, naturalness, for a large set of natural scenes while EEG was recorded. For each individual scene, we derived two physiologically plausible summary statistics by spatially pooling local contrast filter outputs: contrast energy (CE), indexing contrast strength, and spatial coherence (SC), indexing scene fragmentation. We show that behavioral performance is directly related to these statistics, with naturalness rating being influenced in particular by SC. At the neural level, both statistics parametrically modulated single-trial event-related potential amplitudes during an early, transient window (100-150 ms), but SC continued to influence activity levels later in time (up to 250 ms). In addition, the magnitude of neural activity that discriminated between man-made versus natural ratings of individual trials was related to SC, but not CE. These results suggest that global scene information may be computed by spatial pooling of responses from early visual areas (e.g., LGN or V1). The increased sensitivity over time to SC in particular, which reflects scene fragmentation, suggests that this statistic is actively exploited to estimate scene naturalness.
Collapse
Affiliation(s)
- Iris I.A. Groen
- Cognitive Neuroscience Group, Department of Psychology
- Amsterdam Center for Brain and Cognition, Institute for Interdisciplinary Studies, and
| | - Sennay Ghebreab
- Amsterdam Center for Brain and Cognition, Institute for Interdisciplinary Studies, and
- Intelligent Systems Laboratory Amsterdam, Institute of Informatics, University of Amsterdam, 1018 WS, Amsterdam, The Netherlands
| | - Hielke Prins
- Amsterdam Center for Brain and Cognition, Institute for Interdisciplinary Studies, and
| | | | - H. Steven Scholte
- Cognitive Neuroscience Group, Department of Psychology
- Amsterdam Center for Brain and Cognition, Institute for Interdisciplinary Studies, and
| |
Collapse
|
37
|
Kriegeskorte N, Kievit RA. Representational geometry: integrating cognition, computation, and the brain. Trends Cogn Sci 2013; 17:401-12. [PMID: 23876494 PMCID: PMC3730178 DOI: 10.1016/j.tics.2013.06.007] [Citation(s) in RCA: 444] [Impact Index Per Article: 40.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2013] [Revised: 06/06/2013] [Accepted: 06/12/2013] [Indexed: 01/08/2023]
Abstract
Representational geometry is a framework that enables us to relate brain, computation, and cognition. Representations in brains and models can be characterized by representational distance matrices. Distance matrices can be readily compared to test computational models. We review recent insights into perception, cognition, memory, and action and discuss current challenges.
The cognitive concept of representation plays a key role in theories of brain information processing. However, linking neuronal activity to representational content and cognitive theory remains challenging. Recent studies have characterized the representational geometry of neural population codes by means of representational distance matrices, enabling researchers to compare representations across stages of processing and to test cognitive and computational theories. Representational geometry provides a useful intermediate level of description, capturing both the information represented in a neuronal population code and the format in which it is represented. We review recent insights gained with this approach in perception, memory, cognition, and action. Analyses of representational geometry can compare representations between models and the brain, and promise to explain brain computation as transformation of representational similarity structure.
Collapse
|
38
|
Predicting the human reaction time based on natural image statistics in a rapid categorization task. Vision Res 2013; 81:36-44. [PMID: 23419619 DOI: 10.1016/j.visres.2013.02.003] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2012] [Revised: 02/01/2013] [Accepted: 02/04/2013] [Indexed: 11/23/2022]
Abstract
The human visual system is developed by viewing natural scenes. In controlled experiments, natural stimuli therefore provide a realistic framework with which to study the underlying information processing steps involved in human vision. Studying the properties of natural images and their effects on the visual processing can help us to understand underlying mechanisms of visual system. In this study, we used a rapid animal vs. non-animal categorization task to assess the relationship between the reaction times of human subjects and the statistical properties of images. We demonstrated that statistical measures, such as the beta and gamma parameters of a Weibull, fitted to the edge histogram of an image, and the image entropy, are effective predictors of subject reaction times. Using these three parameters, we proposed a computational model capable of predicting the reaction times of human subjects.
Collapse
|