Incremental grouping of image elements in vision

The attentive reconstruction of objects facilitates robust object recognition

Humans are extremely robust in our ability to perceive and recognize objects-we see faces in tea stains and can recognize friends on dark streets. Yet, neurocomputational models of primate object recognition have focused on the initial feed-forward pass of processing through the ventral stream and less on the top-down feedback that likely underlies robust object perception and recognition. Aligned with the generative approach, we propose that the visual system actively facilitates recognition by reconstructing the object hypothesized to be in the image. Top-down attention then uses this reconstruction as a template to bias feedforward processing to align with the most plausible object hypothesis. Building on auto-encoder neural networks, our model makes detailed hypotheses about the appearance and location of the candidate objects in the image by reconstructing a complete object representation from potentially incomplete visual input due to noise and occlusion. The model then leverages the best object reconstruction, measured by reconstruction error, to direct the bottom-up process of selectively routing low-level features, a top-down biasing that captures a core function of attention. We evaluated our model using the MNIST-C (handwritten digits under corruptions) and ImageNet-C (real-world objects under corruptions) datasets. Not only did our model achieve superior performance on these challenging tasks designed to approximate real-world noise and occlusion viewing conditions, but also better accounted for human behavioral reaction times and error patterns than a standard feedforward Convolutional Neural Network. Our model suggests that a complete understanding of object perception and recognition requires integrating top-down and attention feedback, which we propose is an object reconstruction.

Serial processing of proximity groups and similarity groups

Proximity and feature similarity are two important determinants of perceptual grouping in vision. When viewing visual scenes conveying both grouping options simultaneously, people most usually detect proximity groups faster than similarity groups. This article demonstrates that perceptual judgments of grouping orientation guided by either proximity or contrast similarity are indicative of a sequential organization of grouping operations in the visual pathway, which lends a temporal processing advantage to proximity grouping (Experiment 1). Invoking the double-factorial paradigm, latent cognitive architecture for perceptual grouping is also investigated in a task with redundant signals (Experiment 2). Reaction time data from this task is assessed in terms of the race model inequality, workload capacity analysis, and interaction contrasts of means and survivor functions. Again, empirical benchmarks indicate serial processing of proximity groups and similarity groups, with a self-terminating stopping rule for processing. A subset of participants exhibit atypical performance metrics, hinting at possible individual differences in configural visual processing.

Recurrent neural networks that learn multi-step visual routines with reinforcement learning

Many cognitive problems can be decomposed into series of subproblems that are solved sequentially by the brain. When subproblems are solved, relevant intermediate results need to be stored by neurons and propagated to the next subproblem, until the overarching goal has been completed. We will here consider visual tasks, which can be decomposed into sequences of elemental visual operations. Experimental evidence suggests that intermediate results of the elemental operations are stored in working memory as an enhancement of neural activity in the visual cortex. The focus of enhanced activity is then available for subsequent operations to act upon. The main question at stake is how the elemental operations and their sequencing can emerge in neural networks that are trained with only rewards, in a reinforcement learning setting. We here propose a new recurrent neural network architecture that can learn composite visual tasks that require the application of successive elemental operations. Specifically, we selected three tasks for which electrophysiological recordings of monkeys' visual cortex are available. To train the networks, we used RELEARNN, a biologically plausible four-factor Hebbian learning rule, which is local both in time and space. We report that networks learn elemental operations, such as contour grouping and visual search, and execute sequences of operations, solely based on the characteristics of the visual stimuli and the reward structure of a task. After training was completed, the activity of the units of the neural network elicited by behaviorally relevant image items was stronger than that elicited by irrelevant ones, just as has been observed in the visual cortex of monkeys solving the same tasks. Relevant information that needed to be exchanged between subroutines was maintained as a focus of enhanced activity and passed on to the subsequent subroutines. Our results demonstrate how a biologically plausible learning rule can train a recurrent neural network on multistep visual tasks.

Fundamental units of numerosity estimation

Humans can approximately enumerate a large number of objects at a single glance. While several mechanisms have been proposed to account for this ability, the fundamental units over which they operate remain unclear. Previous studies have argued that estimation mechanisms act only on topologically distinct units or on units formed by spatial grouping cues such as proximity and connectivity, but not on units grouped by similarity. Over four experiments, we tested this claim by systematically assessing and demonstrating that similarity grouping leads to underestimation, just as spatial grouping does. Ungrouped objects with the same low-level properties as grouped objects did not cause underestimation. Further, the underestimation caused by spatial and similarity grouping was additive, suggesting that these grouping processes operate independently. These findings argue against the proposal that estimation mechanisms operate solely on topological units. Instead, we conclude that estimation processes act on representations constructed after Gestalt grouping principles, whether similarity based or spatial, have organised incoming visual input.

The representation of occluded image regions in area V1 of monkeys and humans

Neuronal activity in the primary visual cortex (V1) is driven by feedforward input from within the neurons' receptive fields (RFs) and modulated by contextual information in regions surrounding the RF. The effect of contextual information on spiking activity occurs rapidly and is therefore challenging to dissociate from feedforward input. To address this challenge, we recorded the spiking activity of V1 neurons in monkeys viewing either natural scenes or scenes where the information in the RF was occluded, effectively removing the feedforward input. We found that V1 neurons responded rapidly and selectively to occluded scenes. V1 responses elicited by occluded stimuli could be used to decode individual scenes and could be predicted from those elicited by non-occluded images, indicating that there is an overlap between visually driven and contextual responses. We used representational similarity analysis to show that the structure of V1 representations of occluded scenes measured with electrophysiology in monkeys correlates strongly with the representations of the same scenes in humans measured with functional magnetic resonance imaging (fMRI). Our results reveal that contextual influences rapidly alter V1 spiking activity in monkeys over distances of several degrees in the visual field, carry information about individual scenes, and resemble those in human V1. VIDEO ABSTRACT.

The symmetry-induced numerosity illusion depends on visual attention

Symmetry is an important and strong cue we rely on to organize the visual world. Although it is at the basis of objects segmentation in a visual scene, it can sometimes bias our perception. When asked to discriminate numerical quantities between symmetric and asymmetric arrays, individuals tend to underestimate the number of items in the symmetric stimuli. The reason for this underestimation is currently unknown. In this study we investigated whether the symmetry-induced numerosity underestimation depends on perceptual grouping mechanisms by depriving attentional resources. Twenty-six adults judged the numerosity of dot arrays arranged symmetrically or randomly, while ignoring a visual distractor (single task) or while simultaneously judging its color and orientation (dual-task). Diverting attention to the concurrent color-orientation conjunction task halved the symmetry-induced numerosity underestimation. Taken together these results showed that the bias in numerosity perception of symmetric arrays depends-at least partially-on attentional resources and suggested that it might originate from the recruitment of attentional dependent incremental grouping mechanisms.

Solving the binding problem: Assemblies form when neurons enhance their firing rate-they don't need to oscillate or synchronize

When we look at an image, its features are represented in our visual system in a highly distributed manner, calling for a mechanism that binds them into coherent object representations. There have been different proposals for the neuronal mechanisms that can mediate binding. One hypothesis is that binding is achieved by oscillations that synchronize neurons representing features of the same perceptual object. This view allows separate communication channels between different brain areas. Another hypothesis is that binding of features that are represented in different brain regions occurs when the neurons in these areas that respond to the same object simultaneously enhance their firing rate, which would correspond to directing object-based attention to these features. This review summarizes evidence in favor of and against these two hypotheses, examining the neuronal correlates of binding and assessing the time course of perceptual grouping. I conclude that enhanced neuronal firing rates bind features into coherent object representations, whereas oscillations and synchrony are unrelated to binding.

Perceptual bias contextualized in visually ambiguous stimuli

The visual appearance of an object is a function of stimulus properties as well as perceptual biases imposed by the observer. The context-specific trade-off between both can be measured accurately in a perceptual judgment task, involving grouping by proximity in ambiguous dot lattices. Such grouping depends lawfully on a stimulus parameter of the dot lattices known as their aspect ratio (AR), whose effect is modulated by a perceptual bias representing the preference for a cardinal orientation. In two experiments, we investigated how preceding context can lead to bias modulation, either in a top-down fashion via visual working memory (VWM) or bottom-up via sensory priming. In Experiment 1, we embedded the perceptual judgment task in a change detection paradigm and studied how the factors of VWM load (complexity of the memory array) and content (congruency in orientation to the ensuing dot lattice) affect the prominence of perceptual bias. A robust vertical orientation bias was observed, which was increased by VWM load and modulated by congruent VWM content. In Experiment 2, dot lattices were preceded by oriented primes. Here, primes regardless of orientation elicited a vertical orientation bias in dot lattices compared to a neutral baseline. Taken together, the two experiments demonstrate that top-down context (VWM load and content) effectively controls orientation bias modulation, while bottom-up context (i.e., priming) merely acts as an undifferentiated trigger to perceptual bias. These findings characterize the temporal context sensitivity of Gestalt perception, shed light on the processes responsible for different perceptual outcomes of ambiguous stimuli, and identify some of the mechanisms controlling perceptual bias.

The flickering spotlight of visual attention: Characterizing abnormal object-based attention in schizophrenia

Schizophrenia is associated with deficits in both object perception and visual attention. However, few studies in schizophrenia have investigated object-based attention, which is dissociable from other forms of visuospatial attention. Recent research in healthy populations has shown that the 'spotlight' of sustained visual attention flickers in a rhythmic, oscillatory fashion at specific frequencies in the 4-12 Hz range. In healthy samples, this oscillatory signature has been used to investigate spatiotemporal dynamics of object-based attention, showing that the attentional spotlight spreads to uncued locations within cued objects, and also periodically alternates focus between cued and uncued objects. In this study, we adapted a performance-based visual object cueing task to investigate object-based attention in individuals with a schizophrenia diagnosis and healthy controls. In controls, spatiotemporal patterns of object-based attention closely resembled those reported in previous studies of healthy individuals. In the schizophrenia group, the oscillatory signature of attention also appeared in the location of the cue and on uncued objects, similar to the effects in controls. Indeed, the oscillatory signature of attention at the spatial location of the cue was stronger in the schizophrenia group than in controls. However, attention did not spread across the cued object in schizophrenia; rather, attention appeared to remain hyperfocused at the spatial location of the cue. These findings provide the first evidence that visual attention has oscillatory characteristics in schizophrenia, as in the general population. The results also show that the fundamental process of attentional spreading which underlies object-based attention is abnormal in schizophrenia.

A multi-scale neurodynamic implementation of incremental grouping

Incremental grouping is a process entailing serial binding of distal image elements into a unified object representation. At the neural level, incremental grouping involves propagation of the enhanced firing rate among feature-tuned neurons in the early visual cortex. Here, we developed a multi-resolution neural model of incremental grouping. In the model, propagation of the enhanced firing rate is achieved by computing the activity difference between two sets of units: attentional or A-units, whose firing rate is modulated by their horizontal collaterals, and non-attentional or N-units that receive only feedforward input. The activity difference is computed on dendrites that act as independent computational subunits. The proposed model employs multiple spatial scales to account for a variable speed of incremental grouping. In addition, the model incorporates the L-junction detection network that enables incremental grouping over L-junctions. Computer simulations show that the timing of attentional modulations in the model is comparable with neurophysiological measurements in monkey primary visual cortex.

Nonsymbolic numerosity in sets with illusory-contours exploits a context-sensitive, but contrast-insensitive, visual boundary formation process

The visual mechanisms underlying approximate numerical representation are still intensely debated because numerosity information is often confounded with continuous sensory cues (e.g., texture density, area, convex hull). However, numerosity is underestimated when a few items are connected by illusory contours (ICs) lines without changing other physical cues, suggesting in turn that numerosity processing may rely on discrete visual input. Yet, in these previous works, ICs were generated by black-on-gray inducers producing an illusory brightness enhancement, which could represent a further continuous sensory confound. To rule out this possibility, we tested participants in a numerical discrimination task in which we manipulated the alignment of 0, 2, or 4 pairs of open/closed inducers and their contrast polarity. In Experiment 1, aligned open inducers had only one polarity (all black or all white) generating ICs lines brighter or darker than the gray background. In Experiment 2, open inducers had always opposite contrast polarity (one black and one white inducer) generating ICs without strong brightness enhancement. In Experiment 3, reverse-contrast inducers were aligned but closed with a line preventing ICs completion. Results showed that underestimation triggered by ICs lines was independent of inducer contrast polarity in both Experiment 1 and Experiment 2, whereas no underestimation was found in Experiment 3. Taken together, these results suggest that mere brightness enhancement is not the primary cause of the numerosity underestimation induced by ICs lines. Rather, a boundary formation mechanism insensitive to contrast polarity may drive the effect, providing further support to the idea that numerosity processing exploits discrete inputs.

Preserved sensory processing but hampered conflict detection when stimulus input is task-irrelevant

Conflict detection in sensory input is central to adaptive human behavior. Perhaps unsurprisingly, past research has shown that conflict may even be detected in the absence of conflict awareness, suggesting that conflict detection is an automatic process that does not require attention. To test the possibility of conflict processing in the absence of attention, we manipulated task relevance and response overlap of potentially conflicting stimulus features across six behavioral tasks. Multivariate analyses on human electroencephalographic data revealed neural signatures of conflict only when at least one feature of a conflicting stimulus was attended, regardless of whether that feature was part of the conflict, or overlaps with the response. In contrast, neural signatures of basic sensory processes were present even when a stimulus was completely unattended. These data reveal an attentional bottleneck at the level of objects, suggesting that object-based attention is a prerequisite for cognitive control operations involved in conflict detection.

Focusing your attention on one thing can leave you surprisingly unaware of what goes on around you. A classic experiment known as ‘the invisible gorilla’ highlights this phenomenon. Volunteers were asked to watch a clip featuring basketball players, and count how often those wearing white shirts passed the ball: around half of participants failed to spot that someone wearing a gorilla costume wandered into the game and spent nine seconds on screen.

Yet, things that you are not focusing on can sometimes grab your attention anyway. Take for example, the ‘cocktail party effect’, the ability to hear your name among the murmur of a crowded room. So why can we react to our own names, but fail to spot the gorilla? To help answer this question, Nuiten et al. examined how paying attention affects the way the brain processes input.

Healthy volunteers were asked to perform various tasks while the words ‘left’ or ‘right’ played through speakers. The content of the word was sometimes consistent with its location (‘left’ being played on the left speaker), and sometimes opposite (‘left’ being played on the right speaker). Processing either the content or the location of the word is relatively simple for the brain; however detecting a discrepancy between these two properties is challenging, requiring the information to be processed in a brain region that monitors conflict in sensory input.

To manipulate whether the volunteers needed to pay attention to the words, Nuiten et al. made their content or location either relevant or irrelevant for a task. By analyzing brain activity and task performance, they were able to study the effects of attention on how the word properties were processed.

The results showed that the volunteers’ brains were capable of dealing with basic information, such as location or content, even when their attention was directed elsewhere. But discrepancies between content and location could only be detected when the volunteers were focusing on the words, or when their content or location was directly relevant to the task.

The findings by Nuiten et al. suggest that while performing a difficult task, our brains continue to react to basic input but often fail to process more complex information. This, in turn, has implications for a range of human activities such as driving. New technology could potentially help to counteract this phenomenon, aiming to direct attention towards complex information that might otherwise be missed.

The essential role of recurrent processing for figure-ground perception in mice

The segregation of figures from the background is an important step in visual perception. In primary visual cortex, figures evoke stronger activity than backgrounds during a delayed phase of the neuronal responses, but it is unknown how this figure-ground modulation (FGM) arises and whether it is necessary for perception. Here, we show, using optogenetic silencing in mice, that the delayed V1 response phase is necessary for figure-ground segregation. Neurons in higher visual areas also exhibit FGM and optogenetic silencing of higher areas reduced FGM in V1. In V1, figures elicited higher activity of vasoactive intestinal peptide-expressing (VIP) interneurons than the background, whereas figures suppressed somatostatin-positive interneurons, resulting in an increased activation of pyramidal cells. Optogenetic silencing of VIP neurons reduced FGM in V1, indicating that disinhibitory circuits contribute to FGM. Our results provide insight into how lower and higher areas of the visual cortex interact to shape visual perception.

Attention capture by salient object groupings in the neglected visual field

The integration of fragmentary parts into coherent whole objects has been proposed either to rely on the availability of attentional resources or to arise automatically, that is, from preattentive processing (prior to the engagement of selective attention). In the present study, these two alternative accounts were tested in a group of neglect patients with right-hemisphere parietal brain damage and associated deficits of selective attention in the left (visual) hemispace. The reported experiment employed a search task that required detection of targets in the left and/or right hemifields, which were embedded in configurations that consisted of variants of Kanizsa figures. The results showed that a salient, grouped Kanizsa triangle presented within the unattended, left hemifield can substantially improve contralesional target detection, though the very same triangle configuration does not facilitate target detection in the impaired hemifield when presented together with an ipsilesional, but non-salient (i.e., structurally non-integrated, isolated) target. That is, attention is captured by the grouped object in the impaired hemispace only when it is not engaged in the processing of an (isolated) object in the attended hemispace. This demonstrates that both part-to-whole-object integration and search guidance by salient, integrated objects crucially require attentional resources.

Object Selection by Automatic Spreading of Top-Down Attentional Signals in V1

What is selected when attention is directed to a specific location of the visual field? Theories of object-based attention have suggested that when spatial attention is directed to part of an object, attention does not simply enhance the attended location but automatically spreads to enhance all locations that comprise the object. Here, we tested this hypothesis by reconstructing the distribution of attention from primary visual cortex (V1) population neuronal activity patterns in 24 human adults (17 female) using functional magnetic resonance imaging (fMRI) and population-based receptive field (prf) mapping. We find that attention spreads from a spatially cued location to the underlying object, and enhances all spatial locations that comprise the object. Importantly, this spreading was also evident when the object was not task relevant. These data suggest that attentional selection automatically operates at an object level, facilitating the reconstruction of coherent objects from fragmented representations in early visual cortex.SIGNIFICANCE STATEMENT Object perception is an astonishing feat of the visual system. When visual information about orientation, shape, and color enters through our eyes, it has yet to be integrated into a coherent representation of an object. But which visual features constitute a single object and which features belong to the background? The brain mechanisms underpinning object perception are yet to be understood. We now demonstrate that one candidate mechanism, the successive activation of all parts of an object, occurs in early visual cortex and results in a detailed representation of the object following Gestalt principles. Furthermore, our results suggest that object selection occurs automatically, without involving voluntary control.

Topographic signatures of global object perception in human visual cortex

Our visual system readily groups dynamic fragmented input into global objects. How the brain represents global object perception remains however unclear. To address this question, we recorded brain responses using functional magnetic resonance imaging whilst observers viewed a dynamic bistable stimulus that could either be perceived globally (i.e., as a grouped and coherently moving shape) or locally (i.e., as ungrouped and incoherently moving elements). We further estimated population receptive fields and used these to back-project the brain activity measured during stimulus perception into visual space via a searchlight procedure. Global perception resulted in universal suppression of responses in lower visual cortex accompanied by wide-spread enhancement in higher object-sensitive cortex. However, follow-up experiments indicated that higher object-sensitive cortex is suppressed if global perception lacks shape grouping, and that grouping-related suppression can be diffusely confined to stimulated sites and accompanied by background enhancement once stimulus size is reduced. These results speak to a non-generic involvement of higher object-sensitive cortex in perceptual grouping and point to an enhancement-suppression mechanism mediating the perception of figure and ground.

•

Lower visual cortex activity to grouped vs ungrouped dynamic stimuli is suppressed.

•

When grouping a shape, activity in higher object-sensitive cortex is enhanced.

•

Without shape grouping, activity in higher object-sensitive cortex is suppressed.

•

Grouping-related suppression can be diffusely confined to stimulated cortical sites.

Attention in Psychology, Neuroscience, and Machine Learning

Attention is the important ability to flexibly control limited computational resources. It has been studied in conjunction with many other topics in neuroscience and psychology including awareness, vigilance, saliency, executive control, and learning. It has also recently been applied in several domains in machine learning. The relationship between the study of biological attention and its use as a tool to enhance artificial neural networks is not always clear. This review starts by providing an overview of how attention is conceptualized in the neuroscience and psychology literature. It then covers several use cases of attention in machine learning, indicating their biological counterparts where they exist. Finally, the ways in which artificial attention can be further inspired by biology for the production of complex and integrative systems is explored.

Conscious Processing and the Global Neuronal Workspace Hypothesis

We review the central tenets and neuroanatomical basis of the global neuronal workspace (GNW) hypothesis, which attempts to account for the main scientific observations regarding the elementary mechanisms of conscious processing in the human brain. The GNW hypothesis proposes that, in the conscious state, a non-linear network ignition associated with recurrent processing amplifies and sustains a neural representation, allowing the corresponding information to be globally accessed by local processors. We examine this hypothesis in light of recent data that contrast brain activity evoked by either conscious or non-conscious contents, as well as during conscious or non-conscious states, particularly general anesthesia. We also discuss the relationship between the intertwined concepts of conscious processing, attention, and working memory.

Response selection modulates crowding: a cautionary tale for invoking top-down explanations

Object recognition in the periphery is limited by clutter. This phenomenon of visual crowding is ameliorated when the objects are dissimilar. This effect of inter-object similarity has been extensively studied for low-level features and is thought to reflect bottom-up processes. Recently, crowding was also found to be reduced when objects belonged to explicitly distinct groups; that is, crowding was weak when they had low group membership similarity. It has been claimed that top-down knowledge is necessary to explain this effect of group membership, implying that the effect of similarity on crowding cannot be a purely bottom-up process. We tested the claim that the effect of group membership relies on knowledge in two experiments and found that neither explicit knowledge about differences in group membership nor the possibility of acquiring knowledge about target identities is necessary to produce the effects. These results suggest that top-down processes need not be invoked to explain the effect of group membership. Instead, we suggest that differences in flanker reportability that emerge from the differences in group membership are the source of the effect. That is, when targets and flankers are sampled from distinct groups, flankers cannot be inadvertently reported, leading to fewer errors and hence weaker crowding. Further, we argue that this effect arises at the stage of response selection. This conclusion is well supported by an analytical model based on these principles. We conclude that previously observed effects in crowding attributed to top-down or higher level processes might instead be due to post-perceptual response selection strategies.

Neural dynamics of spreading attentional labels in mental contour tracing

Behavioral and neural data suggest that visual attention spreads along contour segments to bind them into a unified object representation. Such attentional labeling segregates the target contour from distractors in a process known as mental contour tracing. A recurrent competitive map is developed to simulate the dynamics of mental contour tracing. In the model, local excitation opposes global inhibition and enables enhanced activity to propagate on the path offered by the contour. The extent of local excitatory interactions is modulated by the output of the multi-scale contour detection network, which constrains the speed of activity spreading in a scale-dependent manner. Furthermore, an L-junction detection network enables tracing to switch direction at the L-junctions, but not at the X- or T-junctions, thereby preventing spillover to a distractor contour. Computer simulations reveal that the model exhibits a monotonic increase in tracing time as a function of the distance to be traced. Also, the speed of tracing increases with decreasing proximity to the distractor contour and with the reduced curvature of the contours. The proposed model demonstrated how an elaborated version of the winner-takes-all network can implement a complex cognitive operation such as contour tracing.

Recurrent Processing of Contour Integration in the Human Visual Cortex as Revealed By fMRI-Guided TMS

Contour integration is a critical step in visual perception because it groups discretely local elements into perceptually global contours. Previous investigations have suggested that striate and extrastriate visual areas are involved in this mid-level processing of visual perception. However, the temporal dynamics of these areas in the human brain during contour integration is less understood. The present study used functional magnetic resonance imaging-guided transcranial magnetic stimulation (TMS) to briefly disrupt 1 of 2 visual areas (V1/V2 and V3B) and examined the causal contributions of these areas to contour detection. The results demonstrated that the earliest critical time window at which behavioral detection performance was impaired by TMS pluses differed between V1/V2 and V3B. The first critical window of V3B (90-110 ms after stimulus onset) was earlier than that of V1/V2 (120-140 ms after stimulus onset), thus indicating that feedback connection from higher to lower area was necessary for complete contour integration. These results suggested that the fine processing of contour-related information in V1/V2 follows the generation of a coarse template in the higher visual areas, such as V3B. Our findings provide direct causal evidence that a recurrent mechanism is necessary for the integration of contours from cluttered background in the human brain.

Two's company, three's a crowd: Individuation is necessary for object recognition

Object recognition is essential for navigating the real world. Despite decades of research on this topic, the processing steps necessary for recognition remain unclear. In this study, we examined the necessity and role of individuation, the ability to select a small number of spatially distinct objects irrespective of their identity, in the recognition process. More specifically, we tested if the ability to rapidly individuate and enumerate a small number of objects (subitizing) can be impaired by crowding. Crowding is flanker-induced interference that specifically impedes the recognition process. We found that subitizing is impaired when objects are close to each other (Experiment 1), and if the target objects are surrounded by irrelevant but perceptually similar flankers (Experiments 2-4). This impairment cannot be attributed to confusion between targets and flankers, wherein flankers are inadvertently included in or targets are excluded from enumeration (Experiments 3-4). Importantly, the flanker induced interference was comparable in both subitizing and crowding tasks (Experiment 4), suggesting that individuation and identification share a common processing pathway. We conclude that individuation is an essential stage in the object recognition pipeline and argue for a cohesive proposal that both crowding and subitizing are due to limitations of selective attention.

Remediation of perceptual organisation in schizophrenia

Introduction Impaired perceptual organisation in schizophrenia has been repeatedly described in clinical and research literatures. It has also been associated with problems in more complex aspects of visual function, including visuospatial and visual cognitive test performance. Two therapeutic interventions were developed here that target perceptual organisation: (1) Computer-based training, which emphasized stimulus-driven processing (bottom-up approach), and (2) Instrumental Enrichment therapy, which is a therapist-guided interactive learning method (top-down approach). Methods Twenty-eight patients diagnosed with schizophrenia or schizoaffective disorder participated in a 12-week programme. For both forms of interventions, task difficulty increased progressively, based upon successful performance. The third group of patients, which served as controls, received a similar therapeutic intervention that did not include a perceptual organisation component. Before and after intervention, participants received tests of perceptual organisation, as well as a battery of neuropsychological tests. Results Results indicate that both forms of intervention improved perceptual organisation ability relative to the control condition. In addition, the improvement was found for select neuropsychological tests, although the pattern of improvement did not favour capacities more closely associated with visual organisational or visuospatial function. Conclusions Together, results demonstrate the effectiveness of new remediation protocols that target mid-level visual processing, which generalized to select visual cognitive functions.

The global precedence effect in English and Korean native speakers with Roman, Korean Hangul and Thai compound letters

The aim was to investigate whether native English speakers (Experiment 1) and native Korean speakers (Experiment 2) processed familiar letters in an analytic manner in comparison to unfamiliar letters or symbols. Participants performed a two-alternative-forced-choice identification task with Roman, Korean Hangul and Thai Navon compound letters (large letters made up of small letters). The English speakers were familiar with Roman script but not Korean or Thai, whereas the Korean speakers were familiar with Korean and Roman script but not Thai. The global precedence effect (GPE), an indication of holistic processing, is characterised by a global advantage (global processing is faster than local processing) and asymmetric congruence (global processing interferes with the processing of local features). Based on previous research, it was predicted that there would be a global precedence effect for unfamiliar but not familiar letters. Results from the English speakers did not support this prediction as we found a GPE for familiar Roman as well as unfamiliar Thai letters but not for unfamiliar Korean letters. In contrast, for the Korean speakers, we found support for the prediction as we found a GPE for Thai letters but not for familiar Korean and Roman letters. Based on this evidence, we propose that the Koreans are processing Korean and Roman letters in a more analytic manner than the English native speakers for Roman script. Due to the characteristics of Korean Hangul, Korean readers may be processing letters in a more analytic manner than the English readers.

Effectiveness of a neuropsychological treatment for confabulations after brain injury: A clinical trial with theoretical implications

Confabulators consistently generate false memories without intention to deceive and with great feelings of rightness. However, to our knowledge, there is currently no known effective treatment for them. In order to fill this gap, our aim was to design a neuropsychological treatment based on current theoretical models and test it experimentally in 20 confabulators sequentially allocated to two groups: an experimental and a control group. The experimental group received nine sessions of treatment for three weeks (three sessions per week). The sessions consisted of some brief material that participants had to learn and recall at both immediate and delayed time points. After this, patients were given feedback about their performance (errors and correct responses). Pre- and post-treatment measurements were recorded. Confabulators in the control group were included in a waiting list for three weeks, performed the pre- and post- measurements without treatment, and only then received the treatment, after which a post-treatment measurement was recorded. This applied to only half of the participants; the other half quit the study prematurely. Results showed a significant decrease in confabulations and a significant increase in correct responses in the experimental group; by contrast, patients in the control group did not improve during the waiting list period. Only control group patients who subsequently received the treatment after serving as controls improved. The effects of the treatment were generalized to patients' everyday lives, as reported by relatives, and persisted over time. This treatment seems to be effective and easy to implement and consequently of clinical interest. Moreover, it also has theoretical implications regarding the processes related to the genesis and/or maintenance of confabulations. In particular, results point to a deficit in early stages of memory retrieval with the preservation of later strategic monitoring processes. Specifically, some of the processes involved may include selective attention or early conflict detection deficits. Future research should test these hypotheses.

Human visual perceptual organization beats thinking on speed

What is the degree to which knowledge influences visual perceptual processes? This question, which is central to the seeing-versus-thinking debate in cognitive science, is often discussed using examples claimed to be proof of one stance or another. It has, however, also been muddled by the usage of different and unclear definitions of perception. Here, for the well-defined process of perceptual organization, I argue that including speed (or efficiency) into the equation opens a new perspective on the limits of top-down influences of thinking on seeing. While the input of the perceptual organization process may be modifiable and its output enrichable, the process itself seems so fast (or efficient) that thinking hardly has time to intrude and is effective mostly after the fact.

Perceptual learning in a non-human primate model of artificial vision

Visual perceptual grouping, the process of forming global percepts from discrete elements, is experience-dependent. Here we show that the learning time course in an animal model of artificial vision is predicted primarily from the density of visual elements. Three naïve adult non-human primates were tasked with recognizing the letters of the Roman alphabet presented at variable size and visualized through patterns of discrete visual elements, specifically, simulated phosphenes mimicking a thalamic visual prosthesis. The animals viewed a spatially static letter using a gaze-contingent pattern and then chose, by gaze fixation, between a matching letter and a non-matching distractor. Months of learning were required for the animals to recognize letters using simulated phosphene vision. Learning rates increased in proportion to the mean density of the phosphenes in each pattern. Furthermore, skill acquisition transferred from trained to untrained patterns, not depending on the precise retinal layout of the simulated phosphenes. Taken together, the findings suggest that learning of perceptual grouping in a gaze-contingent visual prosthesis can be described simply by the density of visual activation.

Early Visual Cortex as a Multiscale Cognitive Blackboard

Neurons in early visual cortical areas not only represent incoming visual information but are also engaged by higher level cognitive processes, including attention, working memory, imagery, and decision-making. Are these cognitive effects an epiphenomenon or are they functionally relevant for these mental operations? We review evidence supporting the hypothesis that the modulation of activity in early visual areas has a causal role in cognition. The modulatory influences allow the early visual cortex to act as a multiscale cognitive blackboard for read and write operations by higher visual areas, which can thereby efficiently exchange information. This blackboard architecture explains how the activity of neurons in the early visual cortex contributes to scene segmentation and working memory, and relates to the subject's inferences about the visual world. The architecture also has distinct advantages for the processing of visual routines that rely on a number of sequentially executed processing steps.

Serial grouping of 2D-image regions with object-based attention in humans

After an initial stage of local analysis within the retina and early visual pathways, the human visual system creates a structured representation of the visual scene by co-selecting image elements that are part of behaviorally relevant objects. The mechanisms underlying this perceptual organization process are only partially understood. We here investigate the time-course of perceptual grouping of two-dimensional image-regions by measuring the reaction times of human participants and report that it is associated with the gradual spread of object-based attention. Attention spreads fastest over large and homogeneous areas and is slowed down at locations that require small-scale processing. We find that the time-course of the object-based selection process is well explained by a 'growth-cone' model, which selects surface elements in an incremental, scale-dependent manner. We discuss how the visual cortical hierarchy can implement this scale-dependent spread of object-based attention, leveraging the different receptive field sizes in distinct cortical areas.

DOI:http://dx.doi.org/10.7554/eLife.14320.001

When we look at an object, we perceive it as a whole. However, this is not how the brain processes objects. Instead, cells at early stages of the visual system respond selectively to single features of the object, such as edges. Moreover, each cell responds to its target feature in only a small region of space known as its receptive field. At higher levels of the visual system, cells respond to more complex features: angles rather than edges, for example. The receptive fields of the cells are also larger. For us to see an object, the brain must therefore 'stitch' together diverse features into a unified impression.

This process is termed perceptual grouping. But how does it work? Jeurissen et al. hypothesized that this process depends on the visual system’s attention spreading over a region in the image occupied by an object, and that the speed of the process will depend on the size of the receptive fields involved. If an image region is narrow, the visual system must recruit cells with small receptive fields to process the individual features. Grouping will therefore be slow. By contrast, if the object consists of large uniform areas lacking in detail, grouping should be fast. These assumptions give rise to a model called the “growth-conemodel”, which makes a number of specific predictions about reaction times during perceptual grouping.

Jeurissen et al. tested the growth-cone model’s predictions by measuring the speed of perceptual grouping in 160 human volunteers. These volunteers looked at an image made up of two simple shapes, and reported whether two dots fell on the same or different shapes. The results supported the growth-cone model. People were able to group large and uniform areas quickly, but were slower for narrow areas. Grouping also took more time when the distance between the dots increased. Hence, perceptual grouping of everyday objects calls on a step-by-step process that resembles solving a small maze.

The results also revealed that perceptual grouping of simple shapes relies on the spreading of visual attention over the relevant object. Furthermore, the data support the hypothesis that perceptual grouping makes use of the different sizes of receptive fields at various levels of the visual system. Further research will be needed to translate these findings to the more complex natural scenes we encounter in our daily lives.

DOI:http://dx.doi.org/10.7554/eLife.14320.002

Assessment of perceptibility and acceptability of color variations between matched teeth among trainee dentist and lay person

Aim:

The aim of this study was to find the difference in perceptibility and acceptability of changes done to various color coordinates of matched teeth, between trainee dental surgeons, and lay person.

Materials and Methods:

A photograph with a set of matched central incisor teeth was selected. In one of the central incisors, the color coordinates (hue, value, and chroma) were altered to a preset value. These pictures were presented to trainee dental surgeons and lay person and their level of perception of color change and acceptance of color change was registered and compared.

Results:

It was found that trainee dental surgeons fared better in perceiving the color change and accepted less of the color changed specimens. The dimension of color that was more discerned both by lay person and trainee dental surgeons was value, hue, and last chroma.

Conclusion:

When compared to a lay person, dental surgeons are more acute in perceiving color changes and do not accept the color difference between teeth to a higher degree.

How visual perceptual grouping influences foot placement

Everybody would agree that vision guides locomotion; but how does vision influence choice when there are different solutions for possible foot placement? We addressed this question by investigating the impact of perceptual grouping on foot placement in humans. Participants performed a stepping stone task in which pathways consisted of target stones in a spatially regular path of foot falls and visual distractor stones in their proximity. Target and distractor stones differed in shape and colour so that each subset of stones could be easily grouped perceptually. In half of the trials, one target stone swapped shape and colour with a distractor in its close proximity. We show that in these 'swapped' conditions, participants chose the perceptually groupable, instead of the spatially regular, stepping location in over 40% of trials, even if the distance between perceptually groupable steps was substantially larger than normal step width/length. This reveals that the existence of a pathway that could be traversed without spatial disruption to periodic stepping is not sufficient to guarantee participants will select it and suggests competition between different types of visual input when choosing foot placement. We propose that a bias in foot placement choice in favour of visual grouping exists as, in nature, sudden changes in visual characteristics of the ground increase the uncertainty for stability.

Activity in human visual and parietal cortex reveals object-based attention in working memory

Visual attention enables observers to select behaviorally relevant information based on spatial locations, features, or objects. Attentional selection is not limited to physically present visual information, but can also operate on internal representations maintained in working memory (WM) in service of higher-order cognition. However, only little is known about whether attention to WM contents follows the same principles as attention to sensory stimuli. To address this question, we investigated in humans whether the typically observed effects of object-based attention in perception are also evident for object-based attentional selection of internal object representations in WM. In full accordance with effects in visual perception, the key behavioral and neuronal characteristics of object-based attention were observed in WM. Specifically, we found that reaction times were shorter when shifting attention to memory positions located on the currently attended object compared with equidistant positions on a different object. Furthermore, functional magnetic resonance imaging and multivariate pattern analysis of visuotopic activity in visual (areas V1-V4) and parietal cortex revealed that directing attention to one position of an object held in WM also enhanced brain activation for other positions on the same object, suggesting that attentional selection in WM activates the entire object. This study demonstrated that all characteristic features of object-based attention are present in WM and thus follows the same principles as in perception.

A salient and task-irrelevant collinear structure hurts visual search

Salient distractors draw our attention spontaneously, even when we intentionally want to ignore them. When this occurs, the real targets close to or overlapping with the distractors benefit from attention capture and thus are detected and discriminated more quickly. However, a puzzling opposite effect was observed in a search display with a column of vertical collinear bars presented as a task-irrelevant distractor [6]. In this case, it was harder to discriminate the targets overlapping with the salient distractor. Here we examined whether this effect originated from factors known to modulate attentional capture: (a) low probability—the probability occurrence of target location at the collinear column was much less (14%) than the rest of the display (86%), and observers might strategically direct their attention away from the collinear distractor; (b) attentional control setting—the distractor and target task interfered with each other because they shared the same continuity set in attentional task; and/or (c) lack of time to establish the optional strategy. We tested these hypotheses by (a) increasing to 60% the trials in which targets overlapped with the same collinear distractor columns, (b) replacing the target task to be connectivity-irrelevant (i.e., luminance discrimination), and (c) having our observers practice the same search task for 10 days. Our results speak against all these hypotheses and lead us to conclude that a collinear distractor impairs search at a level that is unaffected by probabilistic information, attentional setting, and learning.

Response priming evidence for feedforward processing of snake contours but not of ladder contours and textures

In contour integration, increased difficulty in detection and shape discrimination of a chain of parallel elements (a ladder contour) compared to collinear elements (a snake contour) suggests more extensive processing of ladders than of snakes. In addition, conceptual similarities between ladders and textures - which also involve grouping of parallel elements - raises the question whether ladder and texture processing requires feedback from higher visual areas while snakes are processed in a fast feedforward sweep. We tested this in a response priming paradigm, where participants responded as quickly and accurately as possible to the orientation of a diagonal contour in a Gabor array (target). The diagonal was defined either by a snake, ladder, texture, or a continuous line. The target was preceded with varying stimulus onset asynchrony (SOA) by a prime that was either a snake, ladder, or texture, and was consistent or inconsistent to the response demands of the target. Resulting priming effects clearly distinguished between processing of snakes, ladders, and textures. Effects generally increased with SOA but were stronger for snakes and textures compared to ladders. Importantly, only priming effects for snakes were fully present already in the fastest response times, in accordance with a simple feedforward processing model. We conclude that snakes, ladders, and textures do not share similar processing characteristics, with snakes exhibiting a pronounced processing advantage.

The predictive mind and the experience of visual art work

Among the main challenges of the predictive brain/mind concept is how to link prediction at the neural level to prediction at the cognitive-psychological level and finding conceptually robust and empirically verifiable ways to harness this theoretical framework toward explaining higher-order mental and cognitive phenomena, including the subjective experience of aesthetic and symbolic forms. Building on the tentative prediction error account of visual art, this article extends the application of the predictive coding framework to the visual arts. It does so by linking this theoretical discussion to a subjective, phenomenological account of how a work of art is experienced. In order to engage more deeply with a work of art, viewers must be able to tune or adapt their prediction mechanism to recognize art as a specific class of objects whose ontological nature defies predictability, and they must be able to sustain a productive flow of predictions from low-level sensory, recognitional to abstract semantic, conceptual, and affective inferences. The affective component of the process of predictive error optimization that occurs when a viewer enters into dialog with a painting is constituted both by activating the affective affordances within the image and by the affective consequences of prediction error minimization itself. The predictive coding framework also has implications for the problem of the culturality of vision. A person's mindset, which determines what top-down expectations and predictions are generated, is co-constituted by culture-relative skills and knowledge, which form hyperpriors that operate in the perception of art.

Distributed processing of color and form in the visual cortex

To what extent does the visual system process color and form separately? Proponents of the segregation view claim that distinct regions of the cortex are dedicated to each of these two dimensions separately. However, evidence is accumulating that color and form processing may, at least to some extent, be intertwined in the brain. In this perspective, we review psychophysical and neurophysiological studies on color and form perception and evaluate their results in light of recent developments in population coding.

Priming effects on the perceived grouping of ambiguous dot patterns

For ambiguous stimuli, complex dynamics guide processes of perceptual grouping. Previous studies have suggested two opposing effects on grouping that are produced by the preliminary stimulus state: one that enhances grouping towards the existing structure, and another that opposes this structure. To examine effects of the preliminary state on grouping directly, measurements were made of perceived grouping of dot patterns that followed a visual prime. Three stimuli were presented in sequence: prime, target, and mask. Targets were composed of an evenly spaced dot grid in which grouping was established by similarity in luminance. Subjects indicated the dominant perceived grouping. The prime either corresponded to or opposed the prevailing organization of the target. Contrary to the hypothesis, solid-line primes biased grouping away from the structure of the prime, even when the prevailing organization of dot patterns strongly favored the primes' structure. This effect occurred, although to a lesser extent, when primes did not occupy the same location of targets, but were presented in a marginal area surrounding the grid. Priming effects did not occur for primes constructed of dot patterns. Effects found here may be attributed to a forward masking effect by primes, which more effectively disrupts grouping of patterns matched to the prime. Effects may also be attributed to a type of pattern contrast, in which a grouped pattern dissimilar to primes gains salience. For the pattern contrast model, the partial activation of multiple grouped configurations is compared to the pattern of the solid-line primes.

Reading as functional coordination: not recycling but a novel synthesis

The Functional Coordination approach describes the processes involved in learning to read as a form of procedural learning in which pre-existing skills, mainly from the visual, and auditory domain, are (1) recruited, (2) modified, and (3) coordinated to create the procedures for reading text, which form the basis of subsequent (4) automatization. In this context, we discuss evidence relating to the emerging prevalence of analytic processing in letter perception. We argue that the process of learning to read does not have to lead to a loss of perceptual skill as consequence of a "cultural recycling"; learning to read just leads to a novel synthesis of functions, which are coordinated for reading and then automatized as a package over several years. Developmental dyslexia is explained within this framework as a Functional Coordination Deficit (Lachmann, 2002), since the coordination level is assumed to be most liable to manifest deficiencies. This is because, at this level, the greatest degree of fine tuning of complex functions is required. Thus, developmental dyslexia is not seen as a consequence of a deficient automatization per se, but of automatization of abnormally developed functional coordination.

Invisibility and interpretation

Invisibility is often thought to occur because of the low-level limitations of the visual system. For example, it is often assumed that backward masking renders a target invisible because the visual system is simply too slow to resolve the target and the mask separately. Here, we propose an alternative explanation in which invisibility is a goal rather than a limitation and occurs naturally when making sense out of the plethora of incoming information. For example, we present evidence that (in)visibility of an element can strongly depend on how it groups with other elements. Changing grouping changes visibility. In addition, we will show that features often just appear to be invisible but are in fact visible in a way the experimenter is not aware of.

Perceptual grouping across eccentricity

Across the visual field, progressive differences exist in neural processing as well as perceptual abilities. Expansion of stimulus scale across eccentricity compensates for some basic visual capacities, but not for high-order functions. It was hypothesized that as with many higher-order functions, perceptual grouping ability should decline across eccentricity. To test this prediction, psychophysical measurements of grouping were made across eccentricity. Participants indicated the dominant grouping of dot grids in which grouping was based upon luminance, motion, orientation, or proximity. Across trials, the organization of stimuli was systematically decreased until perceived grouping became ambiguous. For all stimulus features, grouping ability remained relatively stable until 40°, beyond which thresholds significantly elevated. The pattern of change across eccentricity varied across stimulus feature, in which stimulus scale, dot size, or stimulus size interacted with eccentricity effects. These results demonstrate that perceptual grouping of such stimuli is not reliant upon foveal viewing, and suggest that selection of dominant grouping patterns from ambiguous displays operates similarly across much of the visual field.

Letters in the forest: global precedence effect disappears for letters but not for non-letters under reading-like conditions

Normally skilled reading involves special processing strategies for letters, which are habitually funneled into an abstract letter code. On the basis of previous studies we argue that this habit leads to the preferred usage of an analytic strategy for the processing of letters, while non-letters are preferably processed via a holistic strategy. The well-known global precedence effect (GPE) seems to contradict to this assumption, since, with compound, hierarchical figures, including letter items, faster responses are observed to the global than to the local level of the figure, as well as an asymmetric interference effect from global to local level. We argue that with letters these effects depend on presentation conditions; only when they elicit the processing strategies automatized for reading, an analytic strategy for letters in contrast to non-letters is to be expected. We compared the GPE for letters and non-letters in central viewing, with the global stimulus size close to the functional visual field in whole word reading (6.5° of visual angle) and local stimuli close to the critical size for fluent reading of individual letters (0.5° of visual angle). Under these conditions, the GPE remained robust for non-letters. For letters, however, it disappeared: letters showed no overall response time advantage for the global level and symmetric congruence effects (local-to-global as well as global-to-local interference). We interpret these results as according to the view that reading is based on resident analytic visual processing strategies for letters.

Object grouping based on real-world regularities facilitates perception by reducing competitive interactions in visual cortex

In virtually every real-life situation humans are confronted with complex and cluttered visual environments that contain a multitude of objects. Because of the limited capacity of the visual system, objects compete for neural representation and cognitive processing resources. Previous work has shown that such attentional competition is partly object based, such that competition among elements is reduced when these elements perceptually group into an object based on low-level cues. Here, using functional MRI (fMRI) and behavioral measures, we show that the attentional benefit of grouping extends to higher-level grouping based on the relative position of objects as experienced in the real world. An fMRI study designed to measure competitive interactions among objects in human visual cortex revealed reduced neural competition between objects when these were presented in commonly experienced configurations, such as a lamp above a table, relative to the same objects presented in other configurations. In behavioral visual search studies, we then related this reduced neural competition to improved target detection when distracter objects were shown in regular configurations. Control studies showed that low-level grouping could not account for these results. We interpret these findings as reflecting the grouping of objects based on higher-level spatial-relational knowledge acquired through a lifetime of seeing objects in specific configurations. This interobject grouping effectively reduces the number of objects that compete for representation and thereby contributes to the efficiency of real-world perception.

Sustained perceptual invisibility of solid shapes following contour adaptation to partial outlines

Contour adaptation (CA) is a recently described paradigm that renders otherwise salient visual stimuli temporarily perceptually invisible. Here we investigate whether this illusion can be exploited to study visual awareness. We found that CA can induce seconds of sustained invisibility following similarly long periods of uninterrupted adaptation. Furthermore, even fragmented adaptors are capable of producing CA, with the strength of CA increasing monotonically as the adaptors encompass a greater fraction of the stimulus outline. However, different types of adaptor patterns, such as distinctive shapes or illusory contours, produce equivalent levels of CA suggesting that the main determinants of CA are low-level stimulus characteristics, with minimal modulation by higher-order visual processes. Taken together, our results indicate that CA has desirable properties for studying visual awareness, including the production of prolonged periods of perceptual dissociation from stimulation as well as parametric dependencies of that dissociation on a host of stimulus parameters.

An integrated face-body representation in the fusiform gyrus but not the lateral occipital cortex

Faces and bodies are processed by distinct category-selective brain areas. Neuroimaging studies have so far presented isolated faces and headless bodies, and therefore little is known on whether and where faces and headless bodies are grouped together to one object, as they appear in the real world. The current study examined whether a face presented above a body are represented as two separate images or as an integrated face-body representation in face and body-selective brain areas by employing a fMRI competition paradigm. This paradigm has been shown to reveal higher fMRI response to sequential than simultaneous presentation of multiple stimuli (i.e., the competition effect), indicating competitive interactions among simultaneously presented multiple stimuli. We therefore hypothesized that if a face above a body is integrated to an image of a person whereas a body above a face is represented as two separate objects, the competition effect will be larger for the latter than the former. Consistent with our hypothesis, our findings reveal a competition effect when a body is presented above a face, but not when a face is presented above a body, suggesting that a body above a face is represented as two separate objects whereas a face above a body is represented as an integrated image of a person. Interestingly, this integration of a face and a body to an image of a person was found in the fusiform, but not the lateral-occipital face and body areas. We conclude that faces and bodies are processed separately at early stages and are integrated to a unified image of a person at mid-level stages of object processing.