1
|
Ye Z, Wessel R, Franken TP. Brain-like border ownership signals support prediction of natural videos. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.08.11.607040. [PMID: 39185218 PMCID: PMC11343161 DOI: 10.1101/2024.08.11.607040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 08/27/2024]
Abstract
To make sense of visual scenes, the brain must segment foreground from background. This is thought to be facilitated by neurons in the primate visual system that encode border ownership (BOS), i.e. whether a local border is part of an object on one or the other side of the border. It is unclear how these signals emerge in neural networks without a teaching signal of what is foreground and background. In this study, we investigated whether BOS signals exist in PredNet, a self-supervised artificial neural network trained to predict the next image frame of natural video sequences. We found that a significant number of units in PredNet are selective for BOS. Moreover these units share several other properties with the BOS neurons in the brain, including robustness to scene variations that constitute common object transformations in natural videos, and hysteresis of BOS signals. Finally, we performed ablation experiments and found that BOS units contribute more to prediction than non-BOS units for videos with moving objects. Our findings indicate that BOS units are especially useful to predict future input in natural videos, even when networks are not required to segment foreground from background. This suggests that BOS neurons in the brain might be the result of evolutionary or developmental pressure to predict future input in natural, complex dynamic visual environments.
Collapse
Affiliation(s)
- Zeyuan Ye
- Department of Physics, Washington University in St. Louis, St. Louis, Missouri, USA
| | - Ralf Wessel
- Department of Physics, Washington University in St. Louis, St. Louis, Missouri, USA
| | - Tom P. Franken
- Department of Neuroscience, Washington University School of Medicine, St. Louis, Missouri, USA
| |
Collapse
|
2
|
Zhu S, Oh YJ, Trepka E, Chen X, Moore T. Dependence of Contextual Modulation in Macaque V1 on Interlaminar Signal Flow. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.18.590176. [PMID: 38659877 PMCID: PMC11042257 DOI: 10.1101/2024.04.18.590176] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/26/2024]
Abstract
In visual cortex, neural correlates of subjective perception can be generated by modulation of activity from beyond the classical receptive field (CRF). In macaque V1, activity generated by nonclassical receptive field (nCRF) stimulation involves different intracortical circuitry than activity generated by CRF stimulation, suggesting that interactions between neurons across V1 layers differ under CRF and nCRF stimulus conditions. We measured border ownership modulation within large populations of V1 neurons. We found that neurons in single columns preferred the same side of objects located outside of the CRF. In addition, we found that interactions between pairs of neurons situated across feedback/horizontal and input layers differed between CRF and nCRF stimulation. Furthermore, the magnitude of border ownership modulation was predicted by greater information flow from feedback/horizontal to input layers. These results demonstrate that the flow of signals between layers covaries with the degree to which neurons integrate information from beyond the CRF.
Collapse
|
3
|
Franken TP, Reynolds JH. Grouping cells in primate visual cortex. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.01.16.575953. [PMID: 38293172 PMCID: PMC10827172 DOI: 10.1101/2024.01.16.575953] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/01/2024]
Abstract
Our perception of how objects are laid out in visual scenes is remarkably stable, despite rapid shifts in the patterns of light that fall on the retina with each saccade. One mechanism that may help establish perceptual stability is border ownership assignment. Studies in macaque area V2 have identified border ownership neurons that signal which side of a border belongs to a foreground surface. This signal persists for hundreds of milliseconds after border ownership has been rendered ambiguous by deleting the stimulus features that distinguish foreground from background. Remarkably, this signal survives eye movements: border ownership neurons also exhibit border ownership signals de novo when an eye movement places the newly ambiguous border within their receptive field. The grouping cell hypothesis proposes the existence of hypothetical grouping cells in a downstream brain area. These cells would compute persistent proto-object representations and therefore have the properties to endow cells in upstream brain areas with selectivity for border ownership. Such grouping cells have been predicted to show a centripetal and persistent pattern of preferred side of ownership for a border placed parallel to the perimeter of their classical receptive field, and such a centripetal ownership preference pattern should also occur de novo in these same cells if an ambiguous border lands in their receptive field after a saccade. It is unknown if grouping cells exist. Here we used laminar multielectrodes in area V4 - the main source of feedback to V2 - of behaving macaques to determine whether such grouping cells exist. Consistent with the model prediction we find a substantial population of neurons with these properties, in all laminar compartments, and they exhibit a response latency that is short enough to act as the source that endows neurons in V2 with selectivity for border ownership. While grouping cell activity provides information about the location of foreground surfaces, these neurons are, counterintuitively, not as strongly tuned for luminance contrast polarity, a feature of those surfaces, as are border ownership cells. Our data suggest a division of labor in which these newly discovered grouping cells provide spatiotemporal continuity of segmented surfaces whereas border ownership cells link this location information with surface features such as luminance contrast.
Collapse
Affiliation(s)
- Tom P. Franken
- Systems Neurobiology Laboratory, The Salk Institute for Biological Studies, San Diego, California, USA
- Department of Neuroscience, Washington University School of Medicine, St. Louis, Missouri, USA
- Lead contact
| | - John H. Reynolds
- Systems Neurobiology Laboratory, The Salk Institute for Biological Studies, San Diego, California, USA
| |
Collapse
|
4
|
Cacciamani L, Skocypec RM, Flowers CS, Perez DC, Peterson MA. BOLD activation on the groundside of figures: More suppression of grounds that competed more for figural status. Cortex 2023; 158:96-109. [PMID: 36495732 DOI: 10.1016/j.cortex.2022.10.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2022] [Revised: 09/01/2022] [Accepted: 10/20/2022] [Indexed: 11/17/2022]
Abstract
A fundamental aspect of object detection is assigning a border to one (figure) side but not the other (ground) side. Figures are shaped; grounds appear shapeless near the figure border. Accumulating evidence supports the view that the mechanism of figure assignment is inhibitory competition with the figure perceived on the winning side. Suppression has been observed on the groundside of figure borders. One prediction is that more suppression will be observed when the groundside competes more for figural status. We tested this prediction by assessing BOLD activation on the groundside of two types of stimuli with articulated borders: AEnov and AEfam stimuli. In both stimulus types, multiple image-based priors (symmetry, closure, small area, enclosure by a larger region) favored the inside as the figure. In AEfam but not AEnov stimuli, the figural prior of familiar configuration present on the outside competes for figural status. Observers perceived the insides of both types of stimuli as novel figures and the outsides as shapeless grounds. Previously, we observed lower BOLD activation in early visual areas representing the grounds of AEfam than AEnov stimuli, although unexpectedly, activation was above baseline. With articulated borders, it can be difficult to exclude figure activation from ground ROIs. Here, our ground ROIs better excluded figure activation; we also added straight-edge (SE) control stimuli and increased the sample size. In early visual areas representing the grounds, we observed lower BOLD activation on the groundside of AEfam than AEnov stimuli and below-baseline BOLD activation on the groundside of SE and AEfam stimuli. These results, indicating that greater suppression is applied to groundsides that competed more for figural status but lost the competition, support a Bayesian model of figure assignment in which proto-objects activated at both low and high levels where image features and familiar configurations are represented, respectively, compete for figural status.
Collapse
Affiliation(s)
- Laura Cacciamani
- Department of Psychology & Child Development, California Polytechnic State University, San Luis Obispo, CA, USA.
| | | | - Colin S Flowers
- Department of Psychology, University of Arizona, Tucson, AZ, USA
| | - Diana C Perez
- Department of Psychology, University of Arizona, Tucson, AZ, USA
| | - Mary A Peterson
- Department of Psychology, University of Arizona, Tucson, AZ, USA; Cognitive Science Program, University of Arizona, Tucson, AZ, USA
| |
Collapse
|
5
|
Wagatsuma N, Shimomura H, Nobukawa S. Disinhibitory circuit mediated by connections from vasoactive intestinal polypeptide to somatostatin interneurons underlies the paradoxical decrease in spike synchrony with increased border ownership selective neuron firing rate. Front Comput Neurosci 2022; 16:988715. [PMID: 36405781 PMCID: PMC9672816 DOI: 10.3389/fncom.2022.988715] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 10/13/2022] [Indexed: 11/06/2022] Open
Abstract
The activity of border ownership selective (BOS) neurons in intermediate-level visual areas indicates which side of a contour owns a border relative to its classical receptive field and provides a fundamental component of figure-ground segregation. A physiological study reported that selective attention facilitates the activity of BOS neurons with a consistent border ownership preference, defined as two neurons tuned to respond to the same visual object. However, spike synchrony between this pair is significantly suppressed by selective attention. These neurophysiological findings are derived from a biologically-plausible microcircuit model consisting of spiking neurons including two subtypes of inhibitory interneurons, somatostatin (SOM) and vasoactive intestinal polypeptide (VIP) interneurons, and excitatory BOS model neurons. In our proposed model, BOS neurons and SOM interneurons cooperate and interact with each other. VIP interneurons not only suppress SOM interneuron responses but also are activated by feedback signals mediating selective attention, which leads to disinhibition of BOS neurons when they are directing selective attention toward an object. Our results suggest that disinhibition arising from the synaptic connections from VIP to SOM interneurons plays a critical role in attentional modulation of neurons in intermediate-level visual areas.
Collapse
Affiliation(s)
- Nobuhiko Wagatsuma
- Department of Information Science, Faculty of Science, Toho University, Funabashi, Japan
- *Correspondence: Nobuhiko Wagatsuma,
| | - Haruka Shimomura
- Department of Information Science, Faculty of Science, Toho University, Funabashi, Japan
| | - Sou Nobukawa
- Department of Computer Science, Chiba Institute of Technology, Narashino, Japan
- Department of Preventive Intervention for Psychiatric Disorders, National Center of Neurology and Psychiatry, Kodaira, Japan
| |
Collapse
|
6
|
Adaptive processing and perceptual learning in visual cortical areas V1 and V4. Proc Natl Acad Sci U S A 2022; 119:e2213080119. [PMID: 36223395 PMCID: PMC9586333 DOI: 10.1073/pnas.2213080119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
Neurons in visual cortical areas primary visual cortex (V1) and V4 are adaptive processors, influenced by perceptual task. This is reflected in their ability to segment the visual scene into task-relevant and task-irrelevant stimulus components and by changing their tuning to task-relevant stimulus properties according to the current top-down instruction. Differences between the information represented in each area were seen. While V1 represented detailed stimulus characteristics, V4 filtered the input from V1 to carry the binary information required for the two-alternative judgement task. Neurons in V1 were activated at locations where the behaviorally relevant stimulus was placed well outside the grating-mapped receptive field. By systematically following the development of the task-dependent signals over the course of perceptual learning, we found that neuronal selectivity for task-relevant information was initially seen in V4 and, over a period of weeks, subsequently in V1. Once the learned information was represented in V1, on any given trial, task-relevant information appeared initially in V1 responses, followed by a 12-ms delay in V4. We propose that the shifting representation of learned information constitutes a mechanism for systems consolidation of memory.
Collapse
|
7
|
Franken TP, Reynolds JH. Columnar processing of border ownership in primate visual cortex. eLife 2021; 10:72573. [PMID: 34845986 PMCID: PMC8631947 DOI: 10.7554/elife.72573] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2021] [Accepted: 10/28/2021] [Indexed: 11/26/2022] Open
Abstract
To understand a visual scene, the brain segregates figures from background by assigning borders to foreground objects. Neurons in primate visual cortex encode which object owns a border (border ownership), but the underlying circuitry is not understood. Here, we used multielectrode probes to record from border ownership-selective units in different layers in macaque visual area V4 to study the laminar organization and timing of border ownership selectivity. We find that border ownership selectivity occurs first in deep layer units, in contrast to spike latency for small stimuli in the classical receptive field. Units on the same penetration typically share the preferred side of border ownership, also across layers, similar to orientation preference. Units are often border ownership-selective for a range of border orientations, where the preferred sides of border ownership are systematically organized in visual space. Together our data reveal a columnar organization of border ownership in V4 where the earliest border ownership signals are not simply inherited from upstream areas, but computed by neurons in deep layers, and may thus be part of signals fed back to upstream cortical areas or the oculomotor system early after stimulus onset. The finding that preferred border ownership is clustered and can cover a wide range of spatially contiguous locations suggests that the asymmetric context integrated by these neurons is provided in a systematically clustered manner, possibly through corticocortical feedback and horizontal connections. To understand a visual scene, the brain needs to identify objects and distinguish them from background. A border marks the transition from object to background, but to differentiate which side of the border belongs to the object and which to background, the brain must integrate information across space. An early signature of this computation is that brain cells signal which side of a border is ‘owned’ by an object, also known as border ownership. But how the brain computes border ownership remains unknown. The optic nerve is a cable-like group of nerve cells that transmits information from the eye to the brain’s visual processing areas and into the visual cortex. This flow of information is often described as traveling in a feedforward direction, away from the eyes to progressively more specialized areas in the visual cortex. However, there are also numerous feedback connections in the brain, running backward from more specialized to less specialized cortical areas. To better understand the role of these feedforward and feedback circuits in the visual processing of object borders, Franken and Reynolds made use of their stereotyped projection patterns across the cortex layers. Feedforward connections terminate in the middle layers of a cortical area, whereas feedback connections terminate in upper and lower layers. Since time is required for information to traverse the cortical layers, dissecting the timing of border ownership signals may reveal if border ownership is computed in a feedforward or feedback manner. To find out more, electrodes were used to record neural activity in the upper, middle and lower layers of the visual cortex of two rhesus monkeys as they were presented with a set of abstract scenes composed of simple shapes on a background. This revealed that cells signaling border ownership in deep layers of the cortex did so before the signals appeared in the middle layer. This suggests that feedback rather than feedforward is required to compute border ownership. Moreover, Franken and Reynolds found evidence that cells that prefer the same side of border ownership are clustered in columns, showing how these neural circuits are organized within the visual cortex. In summary, Franken and Reynolds found that the circuits of the primate brain that compute border ownership occur as columns, in which cells in deep layers signal border ownership first, suggesting that border ownership relies on feedback from more specialized areas. A better understanding of how feedback in the brain works to process visual information helps us appreciate what happens when these systems are impaired.
Collapse
Affiliation(s)
- Tom P Franken
- Systems Neurobiology Laboratory, The Salk Institute for Biological Studies, La Jolla, United States
| | - John H Reynolds
- Systems Neurobiology Laboratory, The Salk Institute for Biological Studies, La Jolla, United States
| |
Collapse
|
8
|
Early recurrence enables figure border ownership. Vision Res 2021; 186:23-33. [PMID: 34023589 DOI: 10.1016/j.visres.2021.04.009] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2020] [Revised: 04/23/2021] [Accepted: 04/28/2021] [Indexed: 11/19/2022]
Abstract
Rubin's face-vase illusion demonstrates how one can switch back and forth between two different interpretations depending on how the figure outlines are assigned. In the primate visual system, assigning ownership along figure borders is encoded by neurons called the border ownership (BO) cells. Studies show that the responses of these neurons not only depend on the local features within their receptive fields, but also on contextual information. Despite two decades of studies on BO neurons, the ownership assignment mechanism in the brain is still unknown. Here, we propose a hierarchical recurrent model grounded on the hypothesis that neurons in the dorsal stream provide the context required for ownership assignment. Our proposed model incorporates early recurrence from the dorsal pathway as well as lateral modulations within the ventral stream. While dorsal modulations initiate the response difference to figure on either side of the border, lateral modulations enhance the difference. We found responses of our dorsally-modulated BO cells, similar to their biological counterparts, are invariant to size, position and solid/outlined figures. Moreover, our model BO cells exhibit comparable levels of reliability in the ownership signal to biological BO neurons. We found dorsal modulations result in high levels of accuracy and robustness for BO assignments in complex scenes compared to previous models based on ventral feedback. Finally, our experiments with illusory contours suggest that BO encoding could explain the perception of such contours in higher processing stages in the brain.
Collapse
|
9
|
Wagatsuma N, Hu B, von der Heydt R, Niebur E. Analysis of spiking synchrony in visual cortex reveals distinct types of top-down modulation signals for spatial and object-based attention. PLoS Comput Biol 2021; 17:e1008829. [PMID: 33765007 PMCID: PMC8023487 DOI: 10.1371/journal.pcbi.1008829] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2020] [Revised: 04/06/2021] [Accepted: 02/22/2021] [Indexed: 11/19/2022] Open
Abstract
The activity of a border ownership selective (BOS) neuron indicates where a foreground object is located relative to its (classical) receptive field (RF). A population of BOS neurons thus provides an important component of perceptual grouping, the organization of the visual scene into objects. In previous theoretical work, it has been suggested that this grouping mechanism is implemented by a population of dedicated grouping (“G”) cells that integrate the activity of the distributed feature cells representing an object and, by feedback, modulate the same cells, thus making them border ownership selective. The feedback modulation by G cells is thought to also provide the mechanism for object-based attention. A recent modeling study showed that modulatory common feedback, implemented by synapses with N-methyl-D-aspartate (NMDA)-type glutamate receptors, accounts for the experimentally observed synchrony in spike trains of BOS neurons and the shape of cross-correlations between them, including its dependence on the attentional state. However, that study was limited to pairs of BOS neurons with consistent border ownership preferences, defined as two neurons tuned to respond to the same visual object, in which attention decreases synchrony. But attention has also been shown to increase synchrony in neurons with inconsistent border ownership selectivity. Here we extend the computational model from the previous study to fully understand these effects of attention. We postulate the existence of a second type of G-cell that represents spatial attention by modulating the activity of all BOS cells in a spatially defined area. Simulations of this model show that a combination of spatial and object-based mechanisms fully accounts for the observed pattern of synchrony between BOS neurons. Our results suggest that modulatory feedback from G-cells may underlie both spatial and object-based attention. Vision allows us to make sense out of a very complex signal, the patterns of light rays reaching our eyes. Two mechanisms are essential for this: perceptual organization which structures the input into meaningful visual objects, and attention which selects only the most important parts in the input. Prior work suggests that both of these mechanisms are implemented by neurons called grouping cells. These organize the object features into coherent entities (perceptual grouping) and access them as needed (selective attention). For technical reasons it is difficult to observe grouping cells but their effect can be seen in the influence they have on responses of other classes of cells. These responses have been measured experimentally and it was found that they depend in unexpected ways on where the subject was attending. Using a computational model, we here demonstrate that the responses can be understood in terms of the interaction between two kinds of selective attention, both of which are known to occur in primate perception. One is attention to a specific area in the environment, the other is to specific objects. A model including both of these attentional mechanisms generates neuronal responses in agreement with the observed patterns of neural activity.
Collapse
Affiliation(s)
| | - Brian Hu
- Allen Institute for Brain Science, Seattle, Washington, United States of America
| | - Rüdiger von der Heydt
- Zanvyl Krieger Mind/Brain Institute, and Solomon Snyder Department of Neuroscience, Johns Hopkins University, Baltimore, Maryland, United States of America
| | - Ernst Niebur
- Zanvyl Krieger Mind/Brain Institute, and Solomon Snyder Department of Neuroscience, Johns Hopkins University, Baltimore, Maryland, United States of America
| |
Collapse
|
10
|
Zooming-in on higher-level vision: High-resolution fMRI for understanding visual perception and awareness. Prog Neurobiol 2021; 207:101998. [PMID: 33497652 DOI: 10.1016/j.pneurobio.2021.101998] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2020] [Revised: 11/11/2020] [Accepted: 01/16/2021] [Indexed: 12/24/2022]
Abstract
One of the central questions in visual neuroscience is how the sparse retinal signals leaving our eyes are transformed into a rich subjective visual experience of the world. Invasive physiology studies, which offers the highest spatial resolution, have revealed many facts about the processing of simple visual features like contrast, color, and orientation, focusing on the early visual areas. At the same time, standard human fMRI studies with comparably coarser spatial resolution have revealed more complex, functionally specialized, and category-selective responses in higher visual areas. Although the visual system is the best understood among the sensory modalities, these two areas of research remain largely segregated. High-resolution fMRI opens up a possibility for linking them. On the one hand, it allows studying how the higher-level visual functions affect the fine-scale activity in early visual areas. On the other hand, it allows discovering the fine-scale functional organization of higher visual areas and exploring their functional connectivity with visual areas lower in the hierarchy. In this review, I will discuss examples of successful work undertaken in these directions using high-resolution fMRI and discuss where this method could be applied in the future to advance our understanding of the complexity of higher-level visual processing.
Collapse
|
11
|
Abstract
The natural environment and our interaction with it are essentially multisensory, where we may deploy visual, tactile and/or auditory senses to perceive, learn and interact with our environment. Our objective in this study is to develop a scene analysis algorithm using multisensory information, specifically vision and audio. We develop a proto-object-based audiovisual saliency map (AVSM) for the analysis of dynamic natural scenes. A specialized audiovisual camera with 360∘ field of view, capable of locating sound direction, is used to collect spatiotemporally aligned audiovisual data. We demonstrate that the performance of a proto-object-based audiovisual saliency map in detecting and localizing salient objects/events is in agreement with human judgment. In addition, the proto-object-based AVSM that we compute as a linear combination of visual and auditory feature conspicuity maps captures a higher number of valid salient events compared to unisensory saliency maps. Such an algorithm can be useful in surveillance, robotic navigation, video compression and related applications.
Collapse
|
12
|
Population coding of figure and ground in natural image patches by V4 neurons. PLoS One 2020; 15:e0235128. [PMID: 32589671 PMCID: PMC7319327 DOI: 10.1371/journal.pone.0235128] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2020] [Accepted: 06/08/2020] [Indexed: 01/09/2023] Open
Abstract
Segmentation of a natural scene into objects and background is a fundamental but challenging task for recognizing objects. Investigating intermediate-level visual cortical areas with a focus on local information is a crucial step towards understanding the formation of the cortical representations of figure and ground. We examined the activity of a population of macaque V4 neurons during the presentation of natural image patches and their respective variations. The natural image patches were optimized to exclude the influence of global context but included various characteristics of local stimulus. Around one fourth of the patch-responsive V4 neurons exhibited significant modulation of firing activity that was dependent on the positional relation between the figural region of the stimulus and the classical receptive field of the neuron. However, the individual neurons showed low consistency in figure-ground modulation across a variety of image patches (55–62%), indicating that individual neurons were capable of correctly signaling figure and ground only for a limited number of stimuli. We examined whether integration of the activity of multiple neurons enabled higher consistency across a variety of natural patches by training a support vector machine to classify figure and ground of the stimuli from the population firing activity. The integration of the activity of a few tens of neurons yielded discrimination accuracy much greater than that of single neurons (up to 85%), suggesting a crucial role of population coding for figure-ground discrimination in natural images.
Collapse
|
13
|
Zhu SD, Zhang LA, von der Heydt R. Searching for object pointers in the visual cortex. J Neurophysiol 2020; 123:1979-1994. [PMID: 32292110 DOI: 10.1152/jn.00112.2020] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
We perceive objects as permanent and stable despite frequent occlusions and eye movements, but their representation in the visual cortex is neither permanent nor stable. Feature selective cells respond only as long as objects are visible, and their responses depend on eye position. We explored the hypothesis that the system maintains object pointers that provide permanence and stability. Pointers should send facilitatory signals to the feature cells of an object, and these signals should persist across temporary occlusions and remap to compensate for image displacements caused by saccades. Here, we searched for such signals in monkey areas V2 and V4 (Macaca mulatta). We developed a new paradigm in which a monkey freely inspects an array of objects in search for reward while some of the objects are being occluded temporarily by opaque drifting strips. Two types of objects were used to manipulate attention. The results were as follows. 1) Eye movements indicated a robust representation of location and type of the occluded objects; 2) in neurons of V4, but not V2, occluded objects produced elevated activity relative to blank condition; 3) the elevation of activity was reduced for objects that had been fixated immediately before the current fixation ('inhibition of return'); and 4) when attended, or when the target of a saccade, visible objects produced enhanced responses in V4, but occluded objects produced no modulation. Although results 1-3 confirm the hypothesis, the absence of modulation under occlusion is not consistent. Further experiments are needed to resolve this discrepancy.NEW & NOTEWORTHY The way we perceive objects as permanent contrasts with the short-lived responses of visual cortical neurons. A theory postulates pointers that give objects continuity, predicting a class of neurons that respond not only to visual objects but also when an occluded object moves into their receptive field. Here, we tested this theory with a novel paradigm in which a monkey freely scans an array of objects while some of them are transiently occluded.
Collapse
Affiliation(s)
- Shude D Zhu
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, Maryland
| | - Li Alex Zhang
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, Maryland
| | - Rüdiger von der Heydt
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, Maryland.,Department of Neuroscience, Johns Hopkins University, School of Medicine, Baltimore, Maryland
| |
Collapse
|
14
|
Walsh KS, McGovern DP, Clark A, O'Connell RG. Evaluating the neurophysiological evidence for predictive processing as a model of perception. Ann N Y Acad Sci 2020; 1464:242-268. [PMID: 32147856 PMCID: PMC7187369 DOI: 10.1111/nyas.14321] [Citation(s) in RCA: 109] [Impact Index Per Article: 27.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2019] [Revised: 01/21/2020] [Accepted: 02/03/2020] [Indexed: 12/12/2022]
Abstract
For many years, the dominant theoretical framework guiding research into the neural origins of perceptual experience has been provided by hierarchical feedforward models, in which sensory inputs are passed through a series of increasingly complex feature detectors. However, the long-standing orthodoxy of these accounts has recently been challenged by a radically different set of theories that contend that perception arises from a purely inferential process supported by two distinct classes of neurons: those that transmit predictions about sensory states and those that signal sensory information that deviates from those predictions. Although these predictive processing (PP) models have become increasingly influential in cognitive neuroscience, they are also criticized for lacking the empirical support to justify their status. This limited evidence base partly reflects the considerable methodological challenges that are presented when trying to test the unique predictions of these models. However, a confluence of technological and theoretical advances has prompted a recent surge in human and nonhuman neurophysiological research seeking to fill this empirical gap. Here, we will review this new research and evaluate the degree to which its findings support the key claims of PP.
Collapse
Affiliation(s)
- Kevin S. Walsh
- Trinity College Institute of Neuroscience and School of PsychologyTrinity College DublinDublinIreland
| | - David P. McGovern
- Trinity College Institute of Neuroscience and School of PsychologyTrinity College DublinDublinIreland
- School of PsychologyDublin City UniversityDublinIreland
| | - Andy Clark
- Department of PhilosophyUniversity of SussexBrightonUK
- Department of InformaticsUniversity of SussexBrightonUK
| | - Redmond G. O'Connell
- Trinity College Institute of Neuroscience and School of PsychologyTrinity College DublinDublinIreland
| |
Collapse
|
15
|
Wandeto JM, Dresp-Langley B. The quantization error in a Self-Organizing Map as a contrast and colour specific indicator of single-pixel change in large random patterns. Neural Netw 2019; 120:116-128. [PMID: 31610898 DOI: 10.1016/j.neunet.2019.09.017] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
The quantization error in a fixed-size Self-Organizing Map (SOM) with unsupervised winner-take-all learning has previously been used successfully to detect, in minimal computation time, highly meaningful changes across images in medical time series and in time series of satellite images. Here, the functional properties of the quantization error in SOM are explored further to show that the metric is capable of reliably discriminating between the finest differences in local contrast intensities and contrast signs. While this capability of the QE is akin to functional characteristics of a specific class of retinal ganglion cells (the so-called Y-cells) in the visual systems of the primate and the cat, the sensitivity of the QE surpasses the capacity limits of human visual detection. Here, the quantization error in the SOM is found to reliably signal changes in contrast or colour when contrast information is removed from or added to the image, but not when the amount and relative weight of contrast information is constant and only the local spatial position of contrast elements in the pattern changes. While the RGB Mean reflects coarser changes in colour or contrast well enough, the SOM-QE is shown to outperform the RGB Mean in the detection of single-pixel changes in images with up to five million pixels. This could have important implications in the context of unsupervised image learning and computational building block approaches to large sets of image data (big data), including deep learning blocks, and automatic detection of contrast change at the nanoscale in Transmission or Scanning Electron Micrographs (TEM, SEM), or at the subpixel level in multispectral and hyper-spectral imaging data.
Collapse
Affiliation(s)
- John M Wandeto
- Dedan Kimathi University of Technology, Department of Information Technology, Nyeri, Kenya
| | - Birgitta Dresp-Langley
- Centre National de la Recherche Scientifique (CNRS), UMR 7357 ICube Lab, University of Strasbourg, France.
| |
Collapse
|
16
|
Grossberg S. The resonant brain: How attentive conscious seeing regulates action sequences that interact with attentive cognitive learning, recognition, and prediction. Atten Percept Psychophys 2019; 81:2237-2264. [PMID: 31218601 PMCID: PMC6848053 DOI: 10.3758/s13414-019-01789-2] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
This article describes mechanistic links that exist in advanced brains between processes that regulate conscious attention, seeing, and knowing, and those that regulate looking and reaching. These mechanistic links arise from basic properties of brain design principles such as complementary computing, hierarchical resolution of uncertainty, and adaptive resonance. These principles require conscious states to mark perceptual and cognitive representations that are complete, context sensitive, and stable enough to control effective actions. Surface-shroud resonances support conscious seeing and action, whereas feature-category resonances support learning, recognition, and prediction of invariant object categories. Feedback interactions between cortical areas such as peristriate visual cortical areas V2, V3A, and V4, and the lateral intraparietal area (LIP) and inferior parietal sulcus (IPS) of the posterior parietal cortex (PPC) control sequences of saccadic eye movements that foveate salient features of attended objects and thereby drive invariant object category learning. Learned categories can, in turn, prime the objects and features that are attended and searched. These interactions coordinate processes of spatial and object attention, figure-ground separation, predictive remapping, invariant object category learning, and visual search. They create a foundation for learning to control motor-equivalent arm movement sequences, and for storing these sequences in cognitive working memories that can trigger the learning of cognitive plans with which to read out skilled movement sequences. Cognitive-emotional interactions that are regulated by reinforcement learning can then help to select the plans that control actions most likely to acquire valued goal objects in different situations. Many interdisciplinary psychological and neurobiological data about conscious and unconscious behaviors in normal individuals and clinical patients have been explained in terms of these concepts and mechanisms.
Collapse
Affiliation(s)
- Stephen Grossberg
- Center for Adaptive Systems, Room 213, Graduate Program in Cognitive and Neural Systems, Departments of Mathematics & Statistics, Psychological & Brain Sciences, and Biomedical Engineering, Boston University, 677 Beacon Street, Boston, MA, 02215, USA.
| |
Collapse
|
17
|
Wandeto JM, Dresp-Langley B. The quantization error in a Self-Organizing Map as a contrast and colour specific indicator of single-pixel change in large random patterns. Neural Netw 2019; 119:273-285. [PMID: 31473578 DOI: 10.1016/j.neunet.2019.08.014] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2019] [Revised: 07/08/2019] [Accepted: 08/09/2019] [Indexed: 10/26/2022]
Abstract
The quantization error in a fixed-size Self-Organizing Map (SOM) with unsupervised winner-take-all learning has previously been used successfully to detect, in minimal computation time, highly meaningful changes across images in medical time series and in time series of satellite images. Here, the functional properties of the quantization error in SOM are explored further to show that the metric is capable of reliably discriminating between the finest differences in local contrast intensities and contrast signs. While this capability of the QE is akin to functional characteristics of a specific class of retinal ganglion cells (the so-called Y-cells) in the visual systems of the primate and the cat, the sensitivity of the QE surpasses the capacity limits of human visual detection. Here, the quantization error in the SOM is found to reliably signal changes in contrast or colour when contrast information is removed from or added to the image, but not when the amount and relative weight of contrast information is constant and only the local spatial position of contrast elements in the pattern changes. While the RGB Mean reflects coarser changes in colour or contrast well enough, the SOM-QE is shown to outperform the RGB Mean in the detection of single-pixel changes in images with up to five million pixels. This could have important implications in the context of unsupervised image learning and computational building block approaches to large sets of image data (big data), including deep learning blocks, and automatic detection of contrast change at the nanoscale in Transmission or Scanning Electron Micrographs (TEM, SEM), or at the subpixel level in multispectral and hyper-spectral imaging data.
Collapse
Affiliation(s)
- John M Wandeto
- Dedan Kimathi University of Technology, Department of Information Technology, Nyeri, Kenya
| | - Birgitta Dresp-Langley
- Centre National de la Recherche Scientifique (CNRS), UMR 7357 ICube Lab, University of Strasbourg, France.
| |
Collapse
|
18
|
Neural dynamics of spreading attentional labels in mental contour tracing. Neural Netw 2019; 119:113-138. [PMID: 31404805 DOI: 10.1016/j.neunet.2019.07.016] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2018] [Revised: 07/12/2019] [Accepted: 07/21/2019] [Indexed: 11/22/2022]
Abstract
Behavioral and neural data suggest that visual attention spreads along contour segments to bind them into a unified object representation. Such attentional labeling segregates the target contour from distractors in a process known as mental contour tracing. A recurrent competitive map is developed to simulate the dynamics of mental contour tracing. In the model, local excitation opposes global inhibition and enables enhanced activity to propagate on the path offered by the contour. The extent of local excitatory interactions is modulated by the output of the multi-scale contour detection network, which constrains the speed of activity spreading in a scale-dependent manner. Furthermore, an L-junction detection network enables tracing to switch direction at the L-junctions, but not at the X- or T-junctions, thereby preventing spillover to a distractor contour. Computer simulations reveal that the model exhibits a monotonic increase in tracing time as a function of the distance to be traced. Also, the speed of tracing increases with decreasing proximity to the distractor contour and with the reduced curvature of the contours. The proposed model demonstrated how an elaborated version of the winner-takes-all network can implement a complex cognitive operation such as contour tracing.
Collapse
|
19
|
Hu B, von der Heydt R, Niebur E. Figure-Ground Organization in Natural Scenes: Performance of a Recurrent Neural Model Compared with Neurons of Area V2. eNeuro 2019; 6:ENEURO.0479-18.2019. [PMID: 31167850 PMCID: PMC6635809 DOI: 10.1523/eneuro.0479-18.2019] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2018] [Revised: 04/15/2019] [Accepted: 05/07/2019] [Indexed: 12/02/2022] Open
Abstract
A crucial step in understanding visual input is its organization into meaningful components, in particular object contours and partially occluded background structures. This requires that all contours are assigned to either the foreground or the background (border ownership assignment). While earlier studies showed that neurons in primate extrastriate cortex signal border ownership for simple geometric shapes, recent studies show consistent border ownership coding also for complex natural scenes. In order to understand how the brain performs this task, we developed a biologically plausible recurrent neural network that is fully image computable. Our model uses local edge detector ( B ) cells and grouping ( G ) cells whose activity represents proto-objects based on the integration of local feature information. G cells send modulatory feedback connections to those B cells that caused their activation, making the B cells border ownership selective. We found close agreement between our model and neurophysiological results in terms of the timing of border ownership signals (BOSs) as well as the consistency of BOSs across scenes. We also benchmarked our model on the Berkeley Segmentation Dataset and achieved performance comparable to recent state-of-the-art computer vision approaches. Our proposed model provides insight into the cortical mechanisms of figure-ground organization.
Collapse
Affiliation(s)
- Brian Hu
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD 21218
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21205
| | - Rüdiger von der Heydt
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD 21218
- Solomon Snyder Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21205
| | - Ernst Niebur
- Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, Baltimore, MD 21218
- Solomon Snyder Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21205
| |
Collapse
|
20
|
Issa EB, Cadieu CF, DiCarlo JJ. Neural dynamics at successive stages of the ventral visual stream are consistent with hierarchical error signals. eLife 2018; 7:42870. [PMID: 30484773 PMCID: PMC6296785 DOI: 10.7554/elife.42870] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2018] [Accepted: 11/27/2018] [Indexed: 12/02/2022] Open
Abstract
Ventral visual stream neural responses are dynamic, even for static image presentations. However, dynamical neural models of visual cortex are lacking as most progress has been made modeling static, time-averaged responses. Here, we studied population neural dynamics during face detection across three cortical processing stages. Remarkably,~30 milliseconds after the initially evoked response, we found that neurons in intermediate level areas decreased their responses to typical configurations of their preferred face parts relative to their response for atypical configurations even while neurons in higher areas achieved and maintained a preference for typical configurations. These hierarchical neural dynamics were inconsistent with standard feedforward circuits. Rather, recurrent models computing prediction errors between stages captured the observed temporal signatures. This model of neural dynamics, which simply augments the standard feedforward model of online vision, suggests that neural responses to static images may encode top-down prediction errors in addition to bottom-up feature estimates.
Collapse
Affiliation(s)
- Elias B Issa
- Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, United States
| | - Charles F Cadieu
- Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, United States
| | - James J DiCarlo
- Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, United States
| |
Collapse
|
21
|
Wagatsuma N. Saliency model based on a neural population for integrating figure direction and organizing Border Ownership. Neural Netw 2018; 110:33-46. [PMID: 30481686 DOI: 10.1016/j.neunet.2018.10.015] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2018] [Revised: 08/31/2018] [Accepted: 10/30/2018] [Indexed: 11/24/2022]
Abstract
Attentional selection is a function of the brain that allocates computational resources momentarily to the most important part of a visual scene. Saliency map models have been used to predict the location of attentional selection and gaze. Border Ownership (BO) indicates the direction of the figure with respect to the border. I here propose a biologically plausible saliency model based on neural population for integrating the activities of intermediate-level visual areas with neurons selective to BO. A variety of BO organizations produces a population of model neurons that represent the grouping structure. In the model I propose, the interactions and the population responses of these model neurons underlie the determination of saliency and the accurate prediction of gaze location. I tested 100 patterns for BO organizations and found that the proposed saliency model not only reproduced the characteristics of perceptual organization but also captured object locations in natural images. Furthermore, the saliency model based on the population responses of the BO organization significantly improved the gaze prediction accuracy compared with previous saliency-based models. These results suggest a crucial role for a wide variety of BO organizations and neural population coding to determine saliency mediating attentional selection and to predict gaze location.
Collapse
Affiliation(s)
- Nobuhiko Wagatsuma
- Toho University, Faculty of Sciences, Miyama 2-2-1, Funabashi, Chiba 274-8510, Japan; University of Tsukuba, Department of Computer Science, Tennodai, 1-1-1, Tsukuba, Ibaraki 305-8573, Japan.
| |
Collapse
|
22
|
Wagatsuma N, Urabe M, Sakai K. Interactions Elicited by the Contradiction Between Figure Direction Discrimination and Figure-Ground Segregation. Front Psychol 2018; 9:1681. [PMID: 30237781 PMCID: PMC6135913 DOI: 10.3389/fpsyg.2018.01681] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2017] [Accepted: 08/21/2018] [Indexed: 11/13/2022] Open
Abstract
Figure-ground (FG) segregation that separates an object from the rest of the image is a fundamental problem in vision science. A majority of neurons in monkey V2 showed the selectivity to border ownership (BO) that indicates which side of a contour owns the border. Although BO could be a precursor of FG segregation, the contribution of BO to FG segregation has not been clarified. Because FG segregation is the perception of the global region that belongs to an object, whereas BO determination provides the local direction of figure (DOF) along a contour, a spatial integration of BO might be expected for the generation of FG. To understand the mechanisms underlying the perception of figural regions, we investigated the interaction between the local BO determination and the global FG segregation through the quantitative analysis of the visual perception and the spatiotemporal characteristics of eye movements. We generated a set of novel stimuli in which translucency induces local DOF along the contour and global FG independently so that DOF and FG could be either consistent or contradictory. The perceptual responses showed better performance in DOF discrimination than FG segregation, supporting distinct mechanisms for the DOF discrimination and the FG segregation. We examined whether the contradiction between DOF and FG modulates the eye movement while participants judged DOF and FG. The duration of the first eye fixation was modulated by the contradiction during FG segregation but not DOF discrimination, suggesting a sequential processing from the BO determination to the FG segregation. These results of human perception and eye fixation provide important clues for understanding the visual processing for FG segregation.
Collapse
Affiliation(s)
- Nobuhiko Wagatsuma
- Faculty of Sciences, Toho University, Funabashi, Japan
- Department of Computer Science, University of Tsukuba, Tsukuba, Japan
| | - Mika Urabe
- Department of Computer Science, University of Tsukuba, Tsukuba, Japan
| | - Ko Sakai
- Department of Computer Science, University of Tsukuba, Tsukuba, Japan
| |
Collapse
|
23
|
von der Heydt R, Zhang NR. Figure and ground: how the visual cortex integrates local cues for global organization. J Neurophysiol 2018; 120:3085-3098. [PMID: 30044171 DOI: 10.1152/jn.00125.2018] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Inferring figure-ground organization in two-dimensional images may require different complementary strategies. For isolated objects, it has been shown that mechanisms in visual cortex exploit the overall distribution of contours, but in images of cluttered scenes where the grouping of contours is not obvious, that strategy would fail. However, natural scenes contain local features, specifically contour junctions, that may contribute to the definition of object regions. To study the role of local features in the assignment of border ownership, we recorded single-cell activity from visual cortex in awake behaving Macaca mulatta. We tested configurations perceived as two overlapping figures in which T- and L-junctions depend on the direction of overlap, whereas the overall distribution of contours provides no valid information. While recording responses to the occluding contour, we varied direction of overlap and variably masked some of the critical contour features to determine their influences and their interactions. On average, most features influenced the responses consistently, producing either enhancement or suppression depending on border ownership. Different feature types could have opposite effects even at the same location. Features far from the receptive field produced effects as strong as near features and with the same short latency. Summation was highly nonlinear: any single feature produced more than two-thirds of the effect of all features together. These findings reveal fast and highly specific organization mechanisms, supporting a previously proposed model in which "grouping cells" integrate widely distributed edge signals with specific end-stopped signals to modulate the original edge signals by feedback. NEW & NOTEWORTHY Seeing objects seems effortless, but defining objects in a scene requires sophisticated neural mechanisms. For isolated objects, the visual cortex groups contours based on overall distribution, but this strategy does not work for cluttered scenes. Here, we demonstrate mechanisms that integrate local contour features like T- and L-junctions to resolve clutter. The process is fast, evaluates widely distributed features, and gives any single feature a decisive influence on figure-ground representation.
Collapse
Affiliation(s)
- Rüdiger von der Heydt
- Department of Neuroscience, Johns Hopkins University School of Medicine , Baltimore, Maryland.,Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University , Baltimore, Maryland
| | - Nan R Zhang
- Department of Neuroscience, Johns Hopkins University School of Medicine , Baltimore, Maryland
| |
Collapse
|
24
|
Nelson R, Hebda N. Figure-Ground Processing: A Reassessment of Gelb and Granit. Perception 2017; 47:344-354. [PMID: 29285993 DOI: 10.1177/0301006617750046] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
In 1923, Adhemar Gelb and Ragnar Granit, two prominent researchers in early Gestalt perceptual theory, reported a lower threshold for detection of a target (a small colored dot) on the ground region of an image than on an adjacent figural region. Although their results had a wide influence on the understanding of figure-ground perception, they are at odds with more recent investigations in which figural regions appear to have a processing advantage over ground regions. The two present studies replicated Gelb and Granit's experiment using a similar figure-ground stimulus albeit with a two-alternative forced choice procedure rather than their original method of adjustment. Experiment 1 found that, contrary to Gelb and Granit's findings, a detection advantage was found for the figural over the ground region. Experiment 2 indicated that explicit contours might have played a role in detection.
Collapse
Affiliation(s)
- Rolf Nelson
- Psychology Department, 6097 Wheaton College , Norton, MA, USA
| | - Nicholas Hebda
- Psychology Department, 6097 Wheaton College , Norton, MA, USA
| |
Collapse
|
25
|
Hu B, Niebur E. A recurrent neural model for proto-object based contour integration and figure-ground segregation. J Comput Neurosci 2017; 43:227-242. [PMID: 28924628 PMCID: PMC5693639 DOI: 10.1007/s10827-017-0659-3] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2017] [Revised: 06/22/2017] [Accepted: 09/08/2017] [Indexed: 12/01/2022]
Abstract
Visual processing of objects makes use of both feedforward and feedback streams of information. However, the nature of feedback signals is largely unknown, as is the identity of the neuronal populations in lower visual areas that receive them. Here, we develop a recurrent neural model to address these questions in the context of contour integration and figure-ground segregation. A key feature of our model is the use of grouping neurons whose activity represents tentative objects ("proto-objects") based on the integration of local feature information. Grouping neurons receive input from an organized set of local feature neurons, and project modulatory feedback to those same neurons. Additionally, inhibition at both the local feature level and the object representation level biases the interpretation of the visual scene in agreement with principles from Gestalt psychology. Our model explains several sets of neurophysiological results (Zhou et al. Journal of Neuroscience, 20(17), 6594-6611 2000; Qiu et al. Nature Neuroscience, 10(11), 1492-1499 2007; Chen et al. Neuron, 82(3), 682-694 2014), and makes testable predictions about the influence of neuronal feedback and attentional selection on neural responses across different visual areas. Our model also provides a framework for understanding how object-based attention is able to select both objects and the features associated with them.
Collapse
Affiliation(s)
- Brian Hu
- Zanvyl Krieger Mind/Brain Institute and Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21218, USA, Tel.: +1 410 516-8640, Fax.: +1 410 516-8648,
| | - Ernst Niebur
- Zanvyl Krieger Mind/Brain Institute and Solomon Snyder Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21218, USA,
| |
Collapse
|
26
|
Ko HK, von der Heydt R. Figure-ground organization in the visual cortex: does meaning matter? J Neurophysiol 2017; 119:160-176. [PMID: 28978761 DOI: 10.1152/jn.00131.2017] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Figure-ground organization in the visual cortex is generally assumed to be based partly on general rules and partly on specific influences of object recognition in higher centers as found in the temporal lobe. To see if shape familiarity influences figure-ground organization, we tested border ownership-selective neurons in monkey V1/V2 with silhouettes of human and monkey face profiles and "nonsense" silhouettes constructed by mirror-reversing the front part of the profile. We found no superiority of face silhouettes compared with nonsense shapes in eliciting border-ownership signals overall. However, in some neurons, border-ownership signals differed strongly between the two categories consistently across many different profile shapes. Surprisingly, this category selectivity appeared as early as 70 ms after stimulus onset, which is earlier than the typical latency of shape-selective responses but compatible with the earliest face-selective responses in the inferior temporal lobe. Although our results provide no evidence for a delayed top-down influence from object recognition centers, they indicate sophisticated shape categorization mechanisms that are much faster than generally assumed. NEW & NOTEWORTHY A long-standing question is whether low-level sensory representations in cortex are influenced by cognitive "top-down" signals. We studied figure-ground organization in the visual cortex by comparing border-ownership signals for face profiles and matched nonsense shapes. We found no sign of "face superiority" in the population border-ownership signal. However, some neurons consistently differentiated between the face and nonsense categories early on, indicating the presence of shape classification mechanisms that are much faster than previously assumed.
Collapse
Affiliation(s)
- Hee-Kyoung Ko
- Krieger Mind/Brain Institute, Johns Hopkins University , Baltimore, Maryland
| | - Rüdiger von der Heydt
- Krieger Mind/Brain Institute, Johns Hopkins University , Baltimore, Maryland.,Department of Neuroscience, Johns Hopkins University School of Medicine , Baltimore, Maryland
| |
Collapse
|
27
|
Sawada T, Petrov AA. The divisive normalization model of V1 neurons: a comprehensive comparison of physiological data and model predictions. J Neurophysiol 2017; 118:3051-3091. [PMID: 28835531 DOI: 10.1152/jn.00821.2016] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2016] [Revised: 08/21/2017] [Accepted: 08/21/2017] [Indexed: 01/24/2023] Open
Abstract
The physiological responses of simple and complex cells in the primary visual cortex (V1) have been studied extensively and modeled at different levels. At the functional level, the divisive normalization model (DNM; Heeger DJ. Vis Neurosci 9: 181-197, 1992) has accounted for a wide range of single-cell recordings in terms of a combination of linear filtering, nonlinear rectification, and divisive normalization. We propose standardizing the formulation of the DNM and implementing it in software that takes static grayscale images as inputs and produces firing rate responses as outputs. We also review a comprehensive suite of 30 empirical phenomena and report a series of simulation experiments that qualitatively replicate dozens of key experiments with a standard parameter set consistent with physiological measurements. This systematic approach identifies novel falsifiable predictions of the DNM. We show how the model simultaneously satisfies the conflicting desiderata of flexibility and falsifiability. Our key idea is that, while adjustable parameters are needed to accommodate the diversity across neurons, they must be fixed for a given individual neuron. This requirement introduces falsifiable constraints when this single neuron is probed with multiple stimuli. We also present mathematical analyses and simulation experiments that explicate some of these constraints.
Collapse
Affiliation(s)
- Tadamasa Sawada
- School of Psychology, National Research University Higher School of Economics, Moscow, Russia; and
| | | |
Collapse
|
28
|
Grossberg S. Towards solving the hard problem of consciousness: The varieties of brain resonances and the conscious experiences that they support. Neural Netw 2016; 87:38-95. [PMID: 28088645 DOI: 10.1016/j.neunet.2016.11.003] [Citation(s) in RCA: 45] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2016] [Revised: 10/21/2016] [Accepted: 11/20/2016] [Indexed: 10/20/2022]
Abstract
The hard problem of consciousness is the problem of explaining how we experience qualia or phenomenal experiences, such as seeing, hearing, and feeling, and knowing what they are. To solve this problem, a theory of consciousness needs to link brain to mind by modeling how emergent properties of several brain mechanisms interacting together embody detailed properties of individual conscious psychological experiences. This article summarizes evidence that Adaptive Resonance Theory, or ART, accomplishes this goal. ART is a cognitive and neural theory of how advanced brains autonomously learn to attend, recognize, and predict objects and events in a changing world. ART has predicted that "all conscious states are resonant states" as part of its specification of mechanistic links between processes of consciousness, learning, expectation, attention, resonance, and synchrony. It hereby provides functional and mechanistic explanations of data ranging from individual spikes and their synchronization to the dynamics of conscious perceptual, cognitive, and cognitive-emotional experiences. ART has reached sufficient maturity to begin classifying the brain resonances that support conscious experiences of seeing, hearing, feeling, and knowing. Psychological and neurobiological data in both normal individuals and clinical patients are clarified by this classification. This analysis also explains why not all resonances become conscious, and why not all brain dynamics are resonant. The global organization of the brain into computationally complementary cortical processing streams (complementary computing), and the organization of the cerebral cortex into characteristic layers of cells (laminar computing), figure prominently in these explanations of conscious and unconscious processes. Alternative models of consciousness are also discussed.
Collapse
Affiliation(s)
- Stephen Grossberg
- Center for Adaptive Systems, Boston University, 677 Beacon Street, Boston, MA 02215, USA; Graduate Program in Cognitive and Neural Systems, Departments of Mathematics & Statistics, Psychological & Brain Sciences, and Biomedical Engineering Boston University, 677 Beacon Street, Boston, MA 02215, USA.
| |
Collapse
|
29
|
Williford JR, von der Heydt R. Figure-Ground Organization in Visual Cortex for Natural Scenes. eNeuro 2016; 3:ENEURO.0127-16.2016. [PMID: 28058269 PMCID: PMC5197405 DOI: 10.1523/eneuro.0127-16.2016] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2016] [Revised: 11/30/2016] [Accepted: 12/02/2016] [Indexed: 11/21/2022] Open
Abstract
Figure-ground organization and border-ownership assignment are essential for understanding natural scenes. It has been shown that many neurons in the macaque visual cortex signal border-ownership in displays of simple geometric shapes such as squares, but how well these neurons resolve border-ownership in natural scenes is not known. We studied area V2 neurons in behaving macaques with static images of complex natural scenes. We found that about half of the neurons were border-ownership selective for contours in natural scenes, and this selectivity originated from the image context. The border-ownership signals emerged within 70 ms after stimulus onset, only ∼30 ms after response onset. A substantial fraction of neurons were highly consistent across scenes. Thus, the cortical mechanisms of figure-ground organization are fast and efficient even in images of complex natural scenes. Understanding how the brain performs this task so fast remains a challenge.
Collapse
|
30
|
Dresp-Langley B, Grossberg S. Neural Computation of Surface Border Ownership and Relative Surface Depth from Ambiguous Contrast Inputs. Front Psychol 2016; 7:1102. [PMID: 27516746 PMCID: PMC4963386 DOI: 10.3389/fpsyg.2016.01102] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2015] [Accepted: 07/07/2016] [Indexed: 11/13/2022] Open
Abstract
The segregation of image parts into foreground and background is an important aspect of the neural computation of 3D scene perception. To achieve such segregation, the brain needs information about border ownership; that is, the belongingness of a contour to a specific surface represented in the image. This article presents psychophysical data derived from 3D percepts of figure and ground that were generated by presenting 2D images composed of spatially disjoint shapes that pointed inward or outward relative to the continuous boundaries that they induced along their collinear edges. The shapes in some images had the same contrast (black or white) with respect to the background gray. Other images included opposite contrasts along each induced continuous boundary. Psychophysical results demonstrate conditions under which figure-ground judgment probabilities in response to these ambiguous displays are determined by the orientation of contrasts only, not by their relative contrasts, despite the fact that many border ownership cells in cortical area V2 respond to a preferred relative contrast. Studies are also reviewed in which both polarity-specific and polarity-invariant properties obtain. The FACADE and 3D LAMINART models are used to explain these data.
Collapse
Affiliation(s)
- Birgitta Dresp-Langley
- Centre National de la Recherche Scientifique, ICube UMR 7357, University of Strasbourg Strasbourg, France
| | - Stephen Grossberg
- Center for Adaptive Systems, Graduate Program in Cognitive and Neural Systems, Department of Mathematics, Boston University, Boston MA, USA
| |
Collapse
|
31
|
Grossberg S. Cortical Dynamics of Figure-Ground Separation in Response to 2D Pictures and 3D Scenes: How V2 Combines Border Ownership, Stereoscopic Cues, and Gestalt Grouping Rules. Front Psychol 2016; 6:2054. [PMID: 26858665 PMCID: PMC4726768 DOI: 10.3389/fpsyg.2015.02054] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2015] [Accepted: 12/24/2015] [Indexed: 11/20/2022] Open
Abstract
The FACADE model, and its laminar cortical realization and extension in the 3D LAMINART model, have explained, simulated, and predicted many perceptual and neurobiological data about how the visual cortex carries out 3D vision and figure-ground perception, and how these cortical mechanisms enable 2D pictures to generate 3D percepts of occluding and occluded objects. In particular, these models have proposed how border ownership occurs, but have not yet explicitly explained the correlation between multiple properties of border ownership neurons in cortical area V2 that were reported in a remarkable series of neurophysiological experiments by von der Heydt and his colleagues; namely, border ownership, contrast preference, binocular stereoscopic information, selectivity for side-of-figure, Gestalt rules, and strength of attentional modulation, as well as the time course during which such properties arise. This article shows how, by combining 3D LAMINART properties that were discovered in two parallel streams of research, a unified explanation of these properties emerges. This explanation proposes, moreover, how these properties contribute to the generation of consciously seen 3D surfaces. The first research stream models how processes like 3D boundary grouping and surface filling-in interact in multiple stages within and between the V1 interblob—V2 interstripe—V4 cortical stream and the V1 blob—V2 thin stripe—V4 cortical stream, respectively. Of particular importance for understanding figure-ground separation is how these cortical interactions convert computationally complementary boundary and surface mechanisms into a consistent conscious percept, including the critical use of surface contour feedback signals from surface representations in V2 thin stripes to boundary representations in V2 interstripes. Remarkably, key figure-ground properties emerge from these feedback interactions. The second research stream shows how cells that compute absolute disparity in cortical area V1 are transformed into cells that compute relative disparity in cortical area V2. Relative disparity is a more invariant measure of an object's depth and 3D shape, and is sensitive to figure-ground properties.
Collapse
Affiliation(s)
- Stephen Grossberg
- Center for Adaptive Systems, Graduate Program in Cognitive and Neural Systems, Center for Computational Neuroscience and Neural Technology, Boston UniversityBoston, MA, USA; Department of Mathematics, Boston UniversityBoston, MA, USA
| |
Collapse
|
32
|
Sakai K, Matsuoka S, Kurematsu K, Hatori Y. Perceptual representation and effectiveness of local figure-ground cues in natural contours. Front Psychol 2015; 6:1685. [PMID: 26579057 PMCID: PMC4630503 DOI: 10.3389/fpsyg.2015.01685] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2015] [Accepted: 10/19/2015] [Indexed: 11/15/2022] Open
Abstract
A contour shape strongly influences the perceptual segregation of a figure from the ground. We investigated the contribution of local contour shape to figure–ground segregation. Although previous studies have reported local contour features that evoke figure–ground perception, they were often image features and not necessarily perceptual features. First, we examined whether contour features, specifically, convexity, closure, and symmetry, underlie the perceptual representation of natural contour shapes. We performed similarity tests between local contours, and examined the contribution of the contour features to the perceptual similarities between the contours. The local contours were sampled from natural contours so that their distribution was uniform in the space composed of the three contour features. This sampling ensured the equal appearance frequency of the factors and a wide variety of contour shapes including those comprised of contradictory factors that induce figure in the opposite directions. This sampling from natural contours is advantageous in order to randomly pickup a variety of contours that satisfy a wide range of cue combinations. Multidimensional scaling analyses showed that the combinations of convexity, closure, and symmetry contribute to perceptual similarity, thus they are perceptual quantities. Second, we examined whether the three features contribute to local figure–ground perception. We performed psychophysical experiments to judge the direction of the figure along the local contours, and examined the contribution of the features to the figure–ground judgment. Multiple linear regression analyses showed that closure was a significant factor, but that convexity and symmetry were not. These results indicate that closure is dominant in the local figure–ground perception with natural contours when the other cues coexist with equal probability including contradictory cases.
Collapse
Affiliation(s)
- Ko Sakai
- Computational Vision Science Laboratory, Department of Computer Science, University of Tsukuba Tsukuba, Japan
| | - Shouhei Matsuoka
- Computational Vision Science Laboratory, Department of Computer Science, University of Tsukuba Tsukuba, Japan
| | - Ken Kurematsu
- Computational Vision Science Laboratory, Department of Computer Science, University of Tsukuba Tsukuba, Japan
| | - Yasuhiro Hatori
- Computational Vision Science Laboratory, Department of Computer Science, University of Tsukuba Tsukuba, Japan
| |
Collapse
|
33
|
von der Heydt R. Figure-ground organization and the emergence of proto-objects in the visual cortex. Front Psychol 2015; 6:1695. [PMID: 26579062 PMCID: PMC4630502 DOI: 10.3389/fpsyg.2015.01695] [Citation(s) in RCA: 36] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2015] [Accepted: 10/20/2015] [Indexed: 11/13/2022] Open
Abstract
A long history of studies of perception has shown that the visual system organizes the incoming information early on, interpreting the 2D image in terms of a 3D world and producing a structure that provides perceptual continuity and enables object-based attention. Recordings from monkey visual cortex show that many neurons, especially in area V2, are selective for border ownership. These neurons are edge selective and have ordinary classical receptive fields (CRF), but in addition their responses are modulated (enhanced or suppressed) depending on the location of a 'figure' relative to the edge in their receptive field. Each neuron has a fixed preference for location on one side or the other. This selectivity is derived from the image context far beyond the CRF. This paper reviews evidence indicating that border ownership selectivity reflects the formation of early object representations ('proto-objects'). The evidence includes experiments showing (1) reversal of border ownership signals with change of perceived object structure, (2) border ownership specific enhancement of responses in object-based selective attention, (3) persistence of border ownership signals in accordance with continuity of object perception, and (4) remapping of border ownership signals across saccades and object movements. Findings 1 and 2 can be explained by hypothetical grouping circuits that sum contour feature signals in search of objectness, and, via recurrent projections, enhance the corresponding low-level feature signals. Findings 3 and 4 might be explained by assuming that the activity of grouping circuits persists and can be remapped. Grouping, persistence, and remapping are fundamental operations of vision. Finding these operations manifest in low-level visual areas challenges traditional views of visual processing. New computational models need to be developed for a comprehensive understanding of the function of the visual cortex.
Collapse
|
34
|
Dresp-Langley B. Principles of perceptual grouping: implications for image-guided surgery. Front Psychol 2015; 6:1565. [PMID: 26539134 PMCID: PMC4611091 DOI: 10.3389/fpsyg.2015.01565] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2015] [Accepted: 09/28/2015] [Indexed: 11/18/2022] Open
Affiliation(s)
- Birgitta Dresp-Langley
- ICube UMR 7357 Centre National de la Recherche Scientifique, University of Strasbourg Strasbourg, France
| |
Collapse
|
35
|
Abstract
Neurons at early stages of the visual cortex signal elemental features, such as pieces of contour, but how these signals are organized into perceptual objects is unclear. Theories have proposed that spiking synchrony between these neurons encodes how features are grouped (binding-by-synchrony), but recent studies did not find the predicted increase in synchrony with binding. Here we propose that features are grouped to "proto-objects" by intrinsic feedback circuits that enhance the responses of the participating feature neurons. This hypothesis predicts synchrony exclusively between feature neurons that receive feedback from the same grouping circuit. We recorded from neurons in macaque visual cortex and used border-ownership selectivity, an intrinsic property of the neurons, to infer whether or not two neurons are part of the same grouping circuit. We found that binding produced synchrony between same-circuit neurons, but not between other pairs of neurons, as predicted by the grouping hypothesis. In a selective attention task, synchrony emerged with ignored as well as attended objects, and higher synchrony was associated with faster behavioral responses, as would be expected from early grouping mechanisms that provide the structure for object-based processing. Thus, synchrony could be produced by automatic activation of intrinsic grouping circuits. However, the binding-related elevation of synchrony was weak compared with its random fluctuations, arguing against synchrony as a code for binding. In contrast, feedback grouping circuits encode binding by modulating the response strength of related feature neurons. Thus, our results suggest a novel coding mechanism that might underlie the proto-objects of perception.
Collapse
|
36
|
Empathy and contextual social cognition. COGNITIVE AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2014; 14:407-25. [PMID: 23955101 DOI: 10.3758/s13415-013-0205-3] [Citation(s) in RCA: 60] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
Empathy is a highly flexible and adaptive process that allows for the interplay of prosocial behavior in many different social contexts. Empathy appears to be a very situated cognitive process, embedded with specific contextual cues that trigger different automatic and controlled responses. In this review, we summarize relevant evidence regarding social context modulation of empathy for pain. Several contextual factors, such as stimulus reality and personal experience, affectively link with other factors, emotional cues, threat information, group membership, and attitudes toward others to influence the affective, sensorimotor, and cognitive processing of empathy. Thus, we propose that the frontoinsular-temporal network, the so-called social context network model (SCNM), is recruited during the contextual processing of empathy. This network would (1) update the contextual cues and use them to construct fast predictions (frontal regions), (2) coordinate the internal (body) and external milieus (insula), and (3) consolidate the context-target associative learning of empathic processes (temporal sites). Furthermore, we propose these context-dependent effects of empathy in the framework of the frontoinsular-temporal network and examine the behavioral and neural evidence of three neuropsychiatric conditions (Asperger syndrome, schizophrenia, and the behavioral variant of frontotemporal dementia), which simultaneously present with empathy and contextual integration impairments. We suggest potential advantages of a situated approach to empathy in the assessment of these neuropsychiatric disorders, as well as their relationship with the SCNM.
Collapse
|
37
|
Orban GA, Zhu Q, Vanduffel W. The transition in the ventral stream from feature to real-world entity representations. Front Psychol 2014; 5:695. [PMID: 25071663 PMCID: PMC4079243 DOI: 10.3389/fpsyg.2014.00695] [Citation(s) in RCA: 32] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2014] [Accepted: 06/16/2014] [Indexed: 11/29/2022] Open
Abstract
We propose that the ventral visual pathway of human and non-human primates is organized into three levels: (1) ventral retinotopic cortex including what is known as TEO in the monkey but corresponds to V4A and PITd/v, and the phPIT cluster in humans, (2) area TE in the monkey and its homolog LOC and neighboring fusiform regions, and more speculatively, (3) TGv in the monkey and its possible human equivalent, the temporal pole. We attribute to these levels the visual representations of features, partial real-world entities (RWEs), and known, complete RWEs, respectively. Furthermore, we propose that the middle level, TE and its homolog, is organized into three parallel substreams, lower bank STS, dorsal convexity of TE, and ventral convexity of TE, as are their corresponding human regions. These presumably process shape in depth, 2D shape and material properties, respectively, to construct RWE representations.
Collapse
Affiliation(s)
- Guy A Orban
- Department of Neuroscience, University of Parma Parma, Italy
| | - Qi Zhu
- Laboratorium voor Neuro-en Psychofysiologie, Department of Neuroscience KU Leuven, Leuven, Belgium
| | - Wim Vanduffel
- Laboratorium voor Neuro-en Psychofysiologie, Department of Neuroscience KU Leuven, Leuven, Belgium
| |
Collapse
|
38
|
|
39
|
Chen M, Yan Y, Gong X, Gilbert CD, Liang H, Li W. Incremental integration of global contours through interplay between visual cortical areas. Neuron 2014; 82:682-94. [PMID: 24811385 DOI: 10.1016/j.neuron.2014.03.023] [Citation(s) in RCA: 118] [Impact Index Per Article: 11.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/03/2014] [Indexed: 11/29/2022]
Abstract
The traditional view on visual processing emphasizes a hierarchy: local line segments are first linked into global contours, which in turn are assembled into more complex forms. Distinct from this bottom-up viewpoint, here we provide evidence for a theoretical framework whereby objects and their parts are processed almost concurrently in a bidirectional cortico-cortical loop. By simultaneous recordings from V1 and V4 in awake monkeys, we found that information about global contours in a cluttered background emerged initially in V4, started ∼40 ms later in V1, and continued to develop in parallel in both areas. Detailed analysis of neuronal response properties implicated contour integration to emerge from both bottom-up and reentrant processes. Our results point to an incremental integration mechanism: feedforward assembling accompanied by feedback disambiguating to define and enhance the global contours and to suppress background noise. The consequence is a parallel accumulation of contour information over multiple cortical areas.
Collapse
Affiliation(s)
- Minggui Chen
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, and Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing 100875, China
| | - Yin Yan
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, and Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing 100875, China
| | - Xiajing Gong
- School of Biomedical Engineering, Drexel University, Philadelphia, PA 19104, USA
| | - Charles D Gilbert
- Laboratory of Neurobiology, The Rockefeller University, New York, NY 10065, USA
| | - Hualou Liang
- School of Biomedical Engineering, Drexel University, Philadelphia, PA 19104, USA
| | - Wu Li
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, and Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing 100875, China.
| |
Collapse
|
40
|
Roth ZN, Zohary E. Fingerprints of Learned Object Recognition Seen in the fMRI Activation Patterns of Lateral Occipital Complex. Cereb Cortex 2014; 25:2427-39. [PMID: 24692511 DOI: 10.1093/cercor/bhu042] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
One feature of visual processing in the ventral stream is that cortical responses gradually depart from the physical aspects of the visual stimulus and become correlated with perceptual experience. Thus, unlike early retinotopic areas, the responses in the object-related lateral occipital complex (LOC) are typically immune to parameter changes (e.g., contrast, location, etc.) when these do not affect recognition. Here, we use a complementary approach to highlight changes in brain activity following a shift in the perceptual state (in the absence of any alteration in the physical image). Specifically, we focus on LOC and early visual cortex (EVC) and compare their functional magnetic resonance imaging (fMRI) responses to degraded object images, before and after fast perceptual learning that renders initially unrecognized objects identifiable. Using 3 complementary analyses, we find that, in LOC, unlike EVC, learned recognition is associated with a change in the multivoxel response pattern to degraded object images, such that the response becomes significantly more correlated with that evoked by the intact version of the same image. This provides further evidence that the coding in LOC reflects the recognition of visual objects.
Collapse
Affiliation(s)
- Zvi N Roth
- Department of Neurobiology Interdisciplinary Center for Neural Computation Edmond & Lily Safra Center for Brain Sciences, Hebrew University, Jerusalem 91904, Israel
| | - Ehud Zohary
- Department of Neurobiology Interdisciplinary Center for Neural Computation Edmond & Lily Safra Center for Brain Sciences, Hebrew University, Jerusalem 91904, Israel
| |
Collapse
|
41
|
Hatori Y, Sakai K. Early representation of shape by onset synchronization of border-ownership-selective cells in the V1-V2 network. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2014; 31:716-729. [PMID: 24695133 DOI: 10.1364/josaa.31.000716] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
Construction of surface is a crucial step toward the representation of shape through the integration of local information. Physiological studies have reported that the primary visual cortex (V1) codes the medial axis (MA) that is a skeletal structure equidistant from nearby contours, suggesting the early representation of surface in V1. Although the neural basis of surface construction has not been clarified, the onset synchronization of border ownership (BO)-selective cells is a plausible candidate for the generation of surface. We investigated computationally the representation of surface in a biophysically detailed model of primate V1-V2 networks. The simulation results showed that the simultaneous arrival of signals from BO-selective cells evoked strong responses of V1 cells located around the MA. The simulation results lead to a prediction that the perception of the direction of figure (DOF) depends on the degree of synchronous presentation of contour. We conducted a psychophysical experiment and showed that the perception of the DOF is biased toward a highly synchronized contour. These results suggest a crucial role of the onset synchronization of BO-selective cells for the construction of early representation of shape.
Collapse
|
42
|
Abstract
Illusory contours can appear as interpolation between edges of the stimulus, as in the Kanizsa triangle, or run orthogonal to the inducing elements, as in the Ehrenstein illusion. Single-cell recordings from monkey visual cortex suggest that both are produced by the same mechanism. Neural border ownership coding, on the other hand, which shows a much larger range of context integration, might involve a different mechanism.
Collapse
|
43
|
Kogo N, Drożdżewska A, Zaenen P, Alp N, Wagemans J. Depth perception of illusory surfaces. Vision Res 2014; 96:53-64. [PMID: 24462748 DOI: 10.1016/j.visres.2013.12.018] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2013] [Revised: 12/20/2013] [Accepted: 12/21/2013] [Indexed: 12/01/2022]
Abstract
The perception of an illusory surface, a subjectively perceived surface that is not given in the image, is one of the most intriguing phenomena in vision. It strongly influences the perception of some fundamental properties, namely, depth, lightness and contours. Recently, we suggested (1) that the context-sensitive mechanism of depth computation plays a key role in creating the illusion, (2) that the illusory lightness perception can be explained by an influence of depth perception on the lightness computation, and (3) that the perception of variations of the Kanizsa figure can be well-reproduced by implementing these principles in a model (Kogo, Strecha, et al., 2010). However, depth perception, lightness perception, contour perception, and their interactions can be influenced by various factors. It is essential to measure the differences between the variation figures in these aspects separately to further understand the mechanisms. As a first step, we report here the results of a new experimental paradigm to compare the depth perception of the Kanizsa figure and its variations. One of the illusory figures was presented side-by-side with a non-illusory variation whose stereo disparities were varied. Participants had to decide in which of these two figures the central region appeared closer. The results indicate that the depth perception of the illusory surface was indeed different in the variation figures. Furthermore, there was a non-linear interaction between the occlusion cues and stereo disparity cues. Implications of the results for the neuro-computational mechanisms are discussed.
Collapse
Affiliation(s)
- Naoki Kogo
- Laboratory of Experimental Psychology, University of Leuven (KU Leuven), Tiensestraat 102, Box 3711, BE-3000 Leuven, Belgium.
| | - Anna Drożdżewska
- Laboratory of Experimental Psychology, University of Leuven (KU Leuven), Tiensestraat 102, Box 3711, BE-3000 Leuven, Belgium
| | - Peter Zaenen
- Laboratory of Experimental Psychology, University of Leuven (KU Leuven), Tiensestraat 102, Box 3711, BE-3000 Leuven, Belgium
| | - Nihan Alp
- Laboratory of Experimental Psychology, University of Leuven (KU Leuven), Tiensestraat 102, Box 3711, BE-3000 Leuven, Belgium
| | - Johan Wagemans
- Laboratory of Experimental Psychology, University of Leuven (KU Leuven), Tiensestraat 102, Box 3711, BE-3000 Leuven, Belgium
| |
Collapse
|
44
|
Russell AF, Mihalaş S, von der Heydt R, Niebur E, Etienne-Cummings R. A model of proto-object based saliency. Vision Res 2014; 94:1-15. [PMID: 24184601 PMCID: PMC3902215 DOI: 10.1016/j.visres.2013.10.005] [Citation(s) in RCA: 51] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2013] [Revised: 08/06/2013] [Accepted: 10/04/2013] [Indexed: 10/26/2022]
Abstract
Organisms use the process of selective attention to optimally allocate their computational resources to the instantaneously most relevant subsets of a visual scene, ensuring that they can parse the scene in real time. Many models of bottom-up attentional selection assume that elementary image features, like intensity, color and orientation, attract attention. Gestalt psychologists, however, argue that humans perceive whole objects before they analyze individual features. This is supported by recent psychophysical studies that show that objects predict eye-fixations better than features. In this report we present a neurally inspired algorithm of object based, bottom-up attention. The model rivals the performance of state of the art non-biologically plausible feature based algorithms (and outperforms biologically plausible feature based algorithms) in its ability to predict perceptual saliency (eye fixations and subjective interest points) in natural scenes. The model achieves this by computing saliency as a function of proto-objects that establish the perceptual organization of the scene. All computational mechanisms of the algorithm have direct neural correlates, and our results provide evidence for the interface theory of attention.
Collapse
Affiliation(s)
- Alexander F Russell
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, United States
| | - Stefan Mihalaş
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21218, United States; Zanvyl-Krieger Mind Brain Institute, Johns Hopkins University, Baltimore, MD 21218, United States
| | - Rudiger von der Heydt
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21218, United States; Zanvyl-Krieger Mind Brain Institute, Johns Hopkins University, Baltimore, MD 21218, United States
| | - Ernst Niebur
- Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21218, United States; Zanvyl-Krieger Mind Brain Institute, Johns Hopkins University, Baltimore, MD 21218, United States
| | - Ralph Etienne-Cummings
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, United States.
| |
Collapse
|
45
|
Ramachandra CA, Mel BW. Computing local edge probability in natural scenes from a population of oriented simple cells. J Vis 2013; 13:13.14.19. [PMID: 24381295 DOI: 10.1167/13.14.19] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
A key computation in visual cortex is the extraction of object contours, where the first stage of processing is commonly attributed to V1 simple cells. The standard model of a simple cell-an oriented linear filter followed by a divisive normalization-fits a wide variety of physiological data, but is a poor performing local edge detector when applied to natural images. The brain's ability to finely discriminate edges from nonedges therefore likely depends on information encoded by local simple cell populations. To gain insight into the corresponding decoding problem, we used Bayes's rule to calculate edge probability at a given location/orientation in an image based on a surrounding filter population. Beginning with a set of ∼ 100 filters, we culled out a subset that were maximally informative about edges, and minimally correlated to allow factorization of the joint on- and off-edge likelihood functions. Key features of our approach include a new, efficient method for ground-truth edge labeling, an emphasis on achieving filter independence, including a focus on filters in the region orthogonal rather than tangential to an edge, and the use of a customized parametric model to represent the individual filter likelihood functions. The resulting population-based edge detector has zero parameters, calculates edge probability based on a sum of surrounding filter influences, is much more sharply tuned than the underlying linear filters, and effectively captures fine-scale edge structure in natural scenes. Our findings predict nonmonotonic interactions between cells in visual cortex, wherein a cell may for certain stimuli excite and for other stimuli inhibit the same neighboring cell, depending on the two cells' relative offsets in position and orientation, and their relative activation levels.
Collapse
|
46
|
Mijović B, De Vos M, Vanderperren K, Machilsen B, Sunaert S, Van Huffel S, Wagemans J. The dynamics of contour integration: A simultaneous EEG-fMRI study. Neuroimage 2013; 88:10-21. [PMID: 24269572 DOI: 10.1016/j.neuroimage.2013.11.032] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2013] [Revised: 11/03/2013] [Accepted: 11/14/2013] [Indexed: 11/24/2022] Open
Abstract
To study the dynamics of contour integration in the human brain, we simultaneously acquired EEG and fMRI data while participants were engaged in a passive viewing task. The stimuli were Gabor arrays with some Gabor elements positioned on the contour of an embedded shape, in three conditions: with local and global structure (perfect contour alignment), with global structure only (orthogonal orientations interrupting the alignment), or without contour. By applying JointICA to the EEG and fMRI responses of the subjects, new insights could be obtained that cannot be derived from unimodal recordings. In particular, only in the global structure condition, an ERP peak around 300ms was identified that involved a loop from LOC to the early visual areas. This component can be interpreted as being related to the verification of the consistency of the different local elements with the globally defined shape, which is necessary when perfect local-to-global alignment is absent. By modifying JointICA, a quantitative comparison of brain regions and the time-course of their interplay were obtained between different conditions. More generally, we provide additional support for the presence of feedback loops from higher areas to lower level sensory regions.
Collapse
Affiliation(s)
- Bogdan Mijović
- KU Leuven, Department of Electrical Engineering, STADIUS, Leuven, Belgium; KU Leuven, iMinds Future Health Department, Leuven, Belgium.
| | - Maarten De Vos
- KU Leuven, Department of Electrical Engineering, STADIUS, Leuven, Belgium; Oldenburg University, Department of Psychology, Neuropsychology Lab, Oldenburg, Germany
| | - Katrien Vanderperren
- KU Leuven, Department of Electrical Engineering, STADIUS, Leuven, Belgium; KU Leuven, iMinds Future Health Department, Leuven, Belgium
| | - Bart Machilsen
- KU Leuven, Laboratory of Experimental Psychology, Leuven, Belgium
| | | | - Sabine Van Huffel
- KU Leuven, Department of Electrical Engineering, STADIUS, Leuven, Belgium; KU Leuven, iMinds Future Health Department, Leuven, Belgium
| | - Johan Wagemans
- KU Leuven, Laboratory of Experimental Psychology, Leuven, Belgium
| |
Collapse
|
47
|
Flevaris AV, Martínez A, Hillyard SA. Neural substrates of perceptual integration during bistable object perception. J Vis 2013; 13:17. [PMID: 24246467 DOI: 10.1167/13.13.17] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
The way we perceive an object depends both on feedforward, bottom-up processing of its physical stimulus properties and on top-down factors such as attention, context, expectation, and task relevance. Here we compared neural activity elicited by varying perceptions of the same physical image--a bistable moving image in which perception spontaneously alternates between dissociated fragments and a single, unified object. A time-frequency analysis of EEG changes associated with the perceptual switch from object to fragment and vice versa revealed a greater decrease in alpha (8-12 Hz) accompanying the switch to object percept than to fragment percept. Recordings of event-related potentials elicited by irrelevant probes superimposed on the moving image revealed an enhanced positivity between 184 and 212 ms when the probes were contained within the boundaries of the perceived unitary object. The topography of the positivity (P2) in this latency range elicited by probes during object perception was distinct from the topography elicited by probes during fragment perception, suggesting that the neural processing of probes differed as a function of perceptual state. Two source localization algorithms estimated the neural generator of this object-related difference to lie in the lateral occipital cortex, a region long associated with object perception. These data suggest that perceived objects attract attention, incorporate visual elements occurring within their boundaries into unified object representations, and enhance the visual processing of elements occurring within their boundaries. Importantly, the perceived object in this case emerged as a function of the fluctuating perceptual state of the viewer.
Collapse
|
48
|
Receptive field focus of visual area V4 neurons determines responses to illusory surfaces. Proc Natl Acad Sci U S A 2013; 110:17095-100. [PMID: 24085849 DOI: 10.1073/pnas.1310806110] [Citation(s) in RCA: 43] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
Illusory figures demonstrate the visual system's ability to infer surfaces under conditions of fragmented sensory input. To investigate the role of midlevel visual area V4 in visual surface completion, we used multielectrode arrays to measure spiking responses to two types of visual stimuli: Kanizsa patterns that induce the perception of an illusory surface and physically similar control stimuli that do not. Neurons in V4 exhibited stronger and sometimes rhythmic spiking responses for the illusion-promoting configurations compared with controls. Moreover, this elevated response depended on the precise alignment of the neuron's peak visual field sensitivity (receptive field focus) with the illusory surface itself. Neurons whose receptive field focus was over adjacent inducing elements, less than 1.5° away, did not show response enhancement to the illusion. Neither receptive field sizes nor fixational eye movements could account for this effect, which was present in both single-unit signals and multiunit activity. These results suggest that the active perceptual completion of surfaces and shapes, which is a fundamental problem in natural visual experience, draws upon the selective enhancement of activity within a distinct subpopulation of neurons in cortical area V4.
Collapse
|
49
|
van den Boomen C, Lamme VA, Kemner C. Parallel development of ERP and behavioural measurements of visual segmentation. Dev Sci 2013; 17:1-10. [DOI: 10.1111/desc.12093] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2012] [Accepted: 05/09/2013] [Indexed: 11/28/2022]
Affiliation(s)
- Carlijn van den Boomen
- Department of Experimental Psychology; Helmholtz Institute; Utrecht The Netherlands
- Department of Developmental Psychology; Utrecht University; The Netherlands
| | - Victor A.F. Lamme
- Brain and Cognition; Department of Psychology; Faculty of Behavioral and Societal Sciences; University of Amsterdam; The Netherlands
| | - Chantal Kemner
- Department of Experimental Psychology; Helmholtz Institute; Utrecht The Netherlands
- Department of Developmental Psychology; Utrecht University; The Netherlands
- Rudolf Magnus Institute of Neuroscience; Department of Child and Adolescent Psychiatry; University Medical Centre; Utrecht The Netherlands
| |
Collapse
|
50
|
Huang LT, Wong AMK, Chen CPC, Chang WH, Cheng JW, Lin YR, Pei YC. Global motion percept mediated through integration of barber poles presented in bilateral visual hemifields. PLoS One 2013; 8:e74032. [PMID: 24009764 PMCID: PMC3756956 DOI: 10.1371/journal.pone.0074032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2012] [Accepted: 08/01/2013] [Indexed: 11/21/2022] Open
Abstract
How is motion information that has been obtained through multiple viewing apertures integrated to form a global motion percept? We investigated the mechanisms of motion integration across apertures in two hemifields by presenting gratings through two rectangles (that form the dual barber poles) and recording the perceived direction of motion by human observers. To this end, we presented dual barber poles in conditions with various inter-component distances between the apertures and evaluated the degree to which the hemifield information was integrated by measuring the magnitude of the perceived barber pole illusion. Surprisingly, when the inter-component distance between the two apertures was short, the perceived direction of motion of the dual barber poles was similar to that of a single barber pole formed by the concatenation of the two component barber poles, indicating motion integration is achieved through a simple concatenation mechanism. We then presented dual barber poles in which the motion and contour properties of the two component barber poles differed to characterize the constraints underlying cross-hemifield integration. We found that integration is achieved only when phase, speed, wavelength, temporal frequency, and duty cycle are identical in the two barber poles, but can remain robust when the contrast of the two component barber poles differs substantially. We concluded that a motion stimulus presented in bilateral hemifields tends to be integrated to yield a global percept with a substantial tolerance for spatial distance and contrast difference.
Collapse
Affiliation(s)
- Li-Ting Huang
- Department of Physical Medicine and Rehabilitation, Chang Gung Memorial Hospital at Linkou, Taoyuan, Taiwan
| | - Alice M. K. Wong
- Department of Physical Medicine and Rehabilitation, Chang Gung Memorial Hospital at Linkou, Taoyuan, Taiwan
| | - Carl P. C. Chen
- Department of Physical Medicine and Rehabilitation, Chang Gung Memorial Hospital at Linkou, Taoyuan, Taiwan
- School of Medicine, Chang Gung University, Taoyuan, Taiwan
| | - Wei-Han Chang
- Department of Physical Medicine and Rehabilitation, Chang Gung Memorial Hospital at Linkou, Taoyuan, Taiwan
| | - Ju-Wen Cheng
- Department of Physical Medicine and Rehabilitation, Chang Gung Memorial Hospital at Linkou, Taoyuan, Taiwan
| | - Yu-Ru Lin
- Department of Physical Medicine and Rehabilitation, Chang Gung Memorial Hospital at Linkou, Taoyuan, Taiwan
| | - Yu-Cheng Pei
- Department of Physical Medicine and Rehabilitation, Chang Gung Memorial Hospital at Linkou, Taoyuan, Taiwan
- School of Medicine, Chang Gung University, Taoyuan, Taiwan
- Healthy Aging Research Center, Chang Gung University, Taoyuan, Taiwan
- * E-mail:
| |
Collapse
|