The role of reentrant processes in feature binding: Evidence from neuropsychology and TMS on late onset illusory conjunctions

Functional characterization of correct and incorrect feature integration

Our sensory system constantly receives information from the environment and our own body. Despite our impression to the contrary, we remain largely unaware of this information and often cannot report it correctly. Although perceptual processing does not require conscious effort on the part of the observer, it is often complex, giving rise to errors such as incorrect integration of features (illusory conjunctions). In the present study, we use functional magnetic resonance imaging to study the neural bases of feature integration in a dual task that produced ~30% illusions. A distributed set of regions demonstrated increased activity for correct compared to incorrect (illusory) feature integration, with increased functional coupling between occipital and parietal regions. In contrast, incorrect feature integration (illusions) was associated with increased occipital (V1-V2) responses at early stages, reduced functional connectivity between right occipital regions and the frontal eye field at later stages, and an overall decrease in coactivation between occipital and parietal regions. These results underscore the role of parietal regions in feature integration and highlight the relevance of functional occipito-frontal interactions in perceptual processing.

The Birthday Party Test (BPT): A new picture description test to support the assessment of simultanagnosia in patients with acquired brain injury

There is broad consensus on the utility of complex pictures in the assessment of simultanagnosia in patients with acquired brain injury (ABI). To overcome various shortcomings of current instruments, we have developed the Birthday Party Test (BPT); a picture description test that contains a neutral scene, a balanced representation of events, and provides clear instructions and a scoring-aid. We have applied the BPT in a large group of patients with ABI (n = 502) and in an age-matched healthy control group (n = 194). Our results show that performance on the BPT was associated with a range of descriptive, neuropsychological and clinical characteristics and that poor test performance appeared to be more common in patients with etiologies that have an increased risk of bilateral damage. Furthermore, we assume a high correspondence between test performance on the BPT and the assessor's clinical judgment of likely having simultanagnosia in preliminary analyses. This study shows the potential usefulness of the BPT to support diagnostic decision making in simultanagnosia. The BPT is made freely available to facilitate its broad application in the clinical assessment of patients with visual impairment and to enable a further evaluation of its utility and validity in future studies.

EXPRESS: Attention does not always help: the role of expectancy, divided, and spatial attention on illusory conjunctions

Humans have the subjective impression of a rich perceptual experience, but this perception is riddled with errors that might be produced by top-down expectancies or failures in feature integration. The role of attention in feature integration is still unclear. Some studies support the importance of attention in feature integration (Paul & Schyns, 2003), whereas others suggest that feature integration does not require attention (Humphreys, 2016). Understanding attention as a heterogeneous system, in this study we explored the role of divided (as opposed to focused - Experiment 1) attention, and endogenous-exogenous spatial orienting (Experiments 2 and 3) in feature integration. We also explored the role of feature expectancy, by presenting stimulus features that were completely unexpected to the participants. Results demonstrated that both endogenous and exogenous orienting improved feature integration while divided attention did not. Moreover, a strong and consistent feature expectancy effect was observed, demonstrating perceptual completion when an unexpected perceptual feature was presented in the scene. These results support the feature confirmation account (Humphreys, 2016), which proposes that attention is important for top-down matching of stable representations.

Stimulus variability and task relevance modulate binding-learning

Classical theories of attention posit that integration of features into object representation (or feature binding) requires engagement of focused attention. Studies challenging this idea have demonstrated that feature binding can happen outside of the focus of attention for familiar objects, as well as for arbitrary color-orientation conjunctions. Detection performance for arbitrary feature conjunction improves with training, suggesting a potential role of perceptual learning mechanisms in the integration of features, a process called "binding-learning". In the present study, we investigate whether stimulus variability and task relevance, two critical determinants of visual perceptual learning, also modulate binding-learning. Transfer of learning in a visual search task to a pre-exposed color-orientation conjunction was assessed under conditions of varying stimulus variability and task relevance. We found transfer of learning for the pre-exposed feature conjunctions that were trained with high variability (Experiment 1). Transfer of learning was not observed when the conjunction was rendered task-irrelevant during training due to pop-out targets (Experiment 2). Our findings show that feature binding is determined by principles of perceptual learning, and they support the idea that functions traditionally attributed to goal-driven attention can be grounded in the learning of the statistical structure of the environment.

A neural correlate of visual feature binding in primate lateral prefrontal cortex

We effortlessly perceive visual objects as unified entities, despite the preferential encoding of their various visual features in separate cortical areas. A 'binding' process is assumed to be required for creating this unified percept, but the underlying neural mechanism and specific brain areas are poorly understood. We investigated 'feature-binding' across two feature dimensions, using a novel stimulus configuration, designed to disambiguate whether a given combination of color and motion direction is perceived as bound or unbound. In the "bound" condition, two behaviorally relevant features (color and motion) belong to the same object, while in the "unbound" condition they belong to different objects. We recorded local field potentials from the lateral prefrontal cortex (lPFC) in macaque monkeys that actively monitored the different stimulus configurations. Our data show a neural representation of visual feature binding especially in the 4-12 Hz frequency band and a transmission of binding information between different lPFC neural subpopulations. This information is linked to the animal's reaction time, suggesting a behavioral relevance of the binding information. Together, our results document the involvement of the prefrontal cortex, targeted by the dorsal and ventral visual streams, in binding visual features from different dimensions, in a process that includes a dynamic modulation of low frequency inter-regional communication.

The causal role of α-oscillations in feature binding

The binding problem-how to integrate features into objects-poses a fundamental challenge for the brain. Neural oscillations, especially γ-oscillations, have been proposed as a potential mechanism to solve this problem. However, since γ-oscillations usually reflect local neural activity, how to implement feature binding involving a large-scale brain network remains largely unknown. Here, combining electroencephalogram (EEG) and transcranial alternating current stimulation (tACS), we employed a bistable color-motion binding stimulus to probe the role of neural oscillations in feature binding. Subjects' perception of the stimulus switched between its physical binding and its illusory (active) binding. The active binding has been shown to involve a large-scale network consisting of spatially distant brain areas. α-Oscillations presumably reflect the dynamics of such large-scale networks, especially due to volume conduction effects in EEG. We found that, relative to the physical binding, the α-power decreased during the active binding. Additionally, individual α-power was negatively correlated with the time proportion of the active binding. Subjects' perceptual switch rate between the 2 bindings was positively correlated with their individual α-frequency. Furthermore, applying tACS at individual α-frequency decreased the time proportion of the active binding. Moreover, delivering tACS at different temporal frequencies in the α-band changed subjects' perceptual switch rate through affecting the active binding process. Our findings provide converging evidence for the causal role of α-oscillations in feature binding, especially in active feature binding, thereby uncovering a function of α-oscillations in human cognition.

Model of Multiple Identity Tracking (MOMIT) 2.0: Resolving the serial vs. parallel controversy in tracking

The present study investigated whether during tracking of multiple moving objects with distinct identities only one identity is tracked at each moment (serial tracking) or whether multiple identities can be tracked simultaneously (parallel tracking). By adopting the gaze-contingent display change technique, we manipulated in real time the presence/absence of object identities during tracking. The data on performance accuracy revealed a serial tracking pattern for facial images and a parallel pattern for color discs: when tracking faces, the presence/absence of only the currently foveated identity impacted the performance, whereas when tracking colors, the presence of multiple identities across the visual field led to improved tracking performance. This pattern is consistent with the identifiability of the different types of objects in the visual field. The eye movements during MIT showed a bias towards visiting and dwelling on individual targets when facial identities were present and towards visiting the blank areas between targets when color identities were present. Nevertheless, the eye visits were predominately on individual targets regardless of the type of objects and the presence of object identities. The eye visits to targets were beneficial for target tracking, particularly in face tracking. We propose the Model of Multiple Identity Tracking (MOMIT) 2.0 which accounts for the results and reconcile the serial vs. parallel controversy. The model suggests that observers cooperatively use attention, eye movements, perception, and working memory for dynamic tracking. Tracking appears more serial when high-resolution information needs to be sampled and maintained for discriminating the targets, whereas it appears more parallel when low-resolution information is sufficient.

Attention is a sterile concept; iterative reentry is a fertile substitute

Attention has been defined as a filter, a limited resource, a spotlight, a zoom lens, and even as a glue that binds disconnected visual features into a coherent object. Here, I claim that all of these metaphor-based explanations are circular. As such, they fail to provide adequate accounts of the phenomena they are purported to explain. In contrast, those very phenomena can be explained on the idea that perceptions emerge from iterative exchanges between cortical regions linked by two-way pathways. Processing can occur in one of two modes: feed-forward and reentrant. In feed-forward mode, the system is configured optimally for the expected input, and perception occurs on the feed-forward sweep. This form of processing corresponds to what is commonly referred to as "preattentive". If the system cannot be configured appropriately, perceptions emerge from iterative reentrant processing, which is slower, and corresponds to what is commonly referred to as "attentive".

Feature Confirmation in Object Perception: Feature Integration Theory 26 Years on from the Treisman Bartlett Lecture

The Treisman Bartlett lecture, reported in the Quarterly Journal of Experimental Psychology in 1988, provided a major overview of the feature integration theory of attention. This has continued to be a dominant account of human visual attention to this day. The current paper provides a summary of the work reported in the lecture and an update on critical aspects of the theory as applied to visual object perception. The paper highlights the emergence of findings that pose significant challenges to the theory and which suggest that revisions are required that allow for (a) several rather than a single form of feature integration, (b) some forms of feature integration to operate preattentively, (c) stored knowledge about single objects and interactions between objects to modulate perceptual integration, (d) the application of feature-based inhibition to object files where visual features are specified, which generates feature-based spreading suppression and scene segmentation, and (e) a role for attention in feature confirmation rather than feature integration in visual selection. A feature confirmation account of attention in object perception is outlined.

To bind or not to bind, that's the wrong question: Features and objects coexist in visual short-term memory

In three experiments, we investigated whether features and whole-objects can be represented simultaneously in visual short-term memory (VSTM). Participants were presented with a memory set of colored shapes; we probed either for the constituent features or for the whole object, and analyzed retrieval dynamics (cumulative response time distributions). In our first experiment, we used whole-object probes that recombined features from the memory display; we found that subjects' data conformed to a kitchen-line model, showing that they used whole-object representations for the matching process. In the second experiment, we encouraged independent-feature representations by using probes that used features not present in the memory display; subjects' data conformed to the race-model inequality, showing that they used independent-feature representations for the matching process. In a final experiment, we used both types of probes; subjects now used both types of representations, depending on the nature of the probe. Combined, our three experiments suggest that both feature and whole-object representations can coexist in VSTM.

Hand proximity differentially affects visual working memory for color and orientation in a binding task

Observers determined whether two sequentially presented arrays of six lines were the same or different. Differences, when present, involved either a swap in the color of two lines or a swap in the orientation of two lines. Thus, accurate change detection required the binding of color and orientation information for each line within visual working memory. Holding viewing distance constant, the proximity of the arrays to the hands was manipulated. Placing the hands near the to-be-remembered array decreased participants’ ability to remember color information, but increased their ability to remember orientation information. This pair of results indicates that hand proximity differentially affects the processing of various types of visual information, a conclusion broadly consistent with functional and anatomical differences in the magnocellular and parvocellular pathways. It further indicates that hand proximity affects the likelihood that various object features will be encoded into integrated object files.

Perception of successive brief objects as a function of stimulus onset asynchrony: model experiments based on two-stage synchronization of neuronal oscillators

Recently we introduced a new version of the perceptual retouch model incorporating two interactive binding operations-binding features for objects and binding the bound feature-objects with a large scale oscillatory system that acts as a mediary for the perceptual information to reach consciousness-level representation. The relative level of synchronized firing of the neurons representing the features of an object obtained after the second-stage synchronizing modulation is used as the equivalent of conscious perception of the corresponding object. Here, this model is used for simulating interaction of two successive featured objects as a function of stimulus onset asynchrony (SOA). Model output reproduces typical results of mutual masking-with shortest and longest SOAs first and second object correct perception rate is comparable while with intermediate SOAs second object dominates over the first one. Additionally, with shortest SOAs misbinding of features to form illusory objects is simulated by the model.

The feature-binding problem is an ill-posed problem

The binding problem arises when visual features (colour, orientation), said to be coded in independent brain modules, are to be integrated into unitary percepts. I argue that binding is an ill-posed problem, because those modules are now known to code jointly for multiple features, rendering the feature-binding issue moot. A hierarchical reentrant system explains the emergence of coherent visual objects from primitive features. An initial feed-forward sweep activates many high-level perceptual hypotheses, which descend to lower levels, where they correlate themselves with the ongoing activity. Low correlations are discarded, whereas the hypothesis that yields the highest correlation is confirmed and leads to conscious awareness. In this system, there is no separate binding process that actively assigns features to objects.

Relationship between visual binding, reentry and awareness

Visual feature binding has been suggested to depend on reentrant processing. We addressed the relationship between binding, reentry, and visual awareness by asking the participants to discriminate the color and orientation of a colored bar (presented either alone or simultaneously with a white distractor bar) and to report their phenomenal awareness of the target features. The success of reentry was manipulated with object substitution masking and backward masking. The results showed that late reentrant processes are necessary for successful binding but not for phenomenal awareness of the bound features. Binding errors were accompanied by phenomenal awareness of the misbound feature conjunctions, demonstrating that they were experienced as real properties of the stimuli (i.e., illusory conjunctions). Our results suggest that early preattentive binding and local recurrent processing enable features to reach phenomenal awareness, while later attention-related reentrant iterations modulate the way in which the features are bound and experienced in awareness.

Shape and color conjunction stimuli are represented as bound objects in visual working memory

The integrated object view of visual working memory (WM) argues that objects (rather than features) are the building block of visual WM, so that adding an extra feature to an object does not result in any extra cost to WM capacity. Alternative views have shown that complex objects consume additional WM storage capacity so that it may not be represented as bound objects. Additionally, it was argued that two features from the same dimension (i.e., color-color) do not form an integrated object in visual WM. This led some to argue for a "weak" object view of visual WM. We used the contralateral delay activity (the CDA) as an electrophysiological marker of WM capacity, to test those alternative hypotheses to the integrated object account. In two experiments we presented complex stimuli and color-color conjunction stimuli, and compared performance in displays that had one object but varying degrees of feature complexity. The results supported the integrated object account by showing that the CDA amplitude corresponded to the number of objects regardless of the number of features within each object, even for complex objects or color-color conjunction stimuli.

Deficits in visual search for conjunctions of motion and form after parietal damage but with spared hMT+/V5

It has been argued that area hMT+/V5 in humans acts as a motion filter, enabling targets defined by a conjunction of motion and form to be efficiently selected. We present data indicating that (a) damage to parietal cortex leads to a selective problem in processing motion-form conjunctions, and (b) that the presence of a structurally and functional intact hMT+/V5 is not sufficient for efficient search for motion-form conjunctions. We suggest that, in addition to motion-processing areas (e.g., hMT+/V5), the posterior parietal cortex is necessary for efficient search with motion-form conjunctions, so that damage to either brain region may bring about deficits in search. We discuss the results in terms of the involvement of the posterior parietal cortex in the top-down guidance of search or in the binding of motion and form information.

Visual feature binding requires reentry

Reentrant processing has been proposed as a critical mechanism in feature binding. To test this claim, participants were shown arrays of six pairs of crossed vertical and horizontal bars. In each pair, one bar was white; one was red, green, or blue. Identifying the orientation, but not the color, of the nonwhite bar in the target item required correct binding. Four dots appeared around one of the items (the target) and either disappeared with it or persisted for 300 ms after the array disappeared. This type of trailing mask is thought to interfere with target processing by disrupting reentry. Consistent with the hypothesis that binding requires reentrant processing, the trailing mask significantly reduced the accuracy of orientation but not color judgments. In a control condition, when the white bar was omitted, binding was no longer required, and both color and orientation were accurately reported.