Independent, synchronous access to color and motion features

Revisiting the color-motion asynchrony

Color-motion asynchrony (CMA) refers to an illusion in which we perceive a change in color earlier than a change in motion direction when the two changes occur simultaneously. This phenomenon may indicate that color is processed earlier than motion in the visual system. However, the very existence of CMA is under question owing to contradictory findings and methodological deficits in previous studies. Here, we used both the motion and color correspondence tasks (experiment 1) and the temporal order judgment (TOJ) task (experiment 2) to re-examine CMA. Colored dots moved in one direction and changed their color/direction at some time, whereas the relative timing between color and direction changes varied across trials. In the correspondence task, participants reported which direction/color of dots with a particular color/direction lasted longer, the one before or after the change? In the TOJ task, participants reported whether the change in color or the change in motion direction occurred earlier. Results indicated that participants perceived the change in color earlier than the change in motion direction in either the motion or color correspondence task, with a stronger asynchrony in the former. In the TOJ task, although participants showed no difference in psychophysical measures, they responded faster when the change in color occurred before (versus after) the change in direction. Drift-diffusion modeling (DDM) revealed a lower decision threshold when the change in color occurred before (versus after) the change in direction, indicating less cautiousness was excised in judgment when the color changed earlier. These results confirmed the veracity of CMA in different tasks and point to the viability of analyzing response times in traditional psychophysical studies.

"Multiplexing" cells of the visual cortex and the timing enigma of the binding problem

In this opinion essay, I address the perennial binding problem, that is to say of how independently processed visual attributes such as form, colour and motion are brought together to give us a unified and holistic picture of the visual world. A solution to this central issue in neurobiology remains as elusive as ever. No one knows today how it is implemented. The issue is not a new one and, though discussed most commonly in the context of the visual brain, it is not unique to it either. Karl Lashley summarized it well years ago when he wrote that a critical problem for brain studies is to understand how "the specialized areas of the cerebral cortex interact to provide the integration evident in thought and behaviour" (Lashley, 1931).

A delay in sampling information from temporally autocorrelated visual stimuli

Much of our world changes smoothly in time, yet the allocation of attention is typically studied with sudden changes - transients. A sizeable lag in selecting feature information is seen when stimuli change smoothly. Yet this lag is not seen with temporally uncorrelated rapid serial visual presentation (RSVP) stimuli. This suggests that temporal autocorrelation of a feature paradoxically increases the latency at which information is sampled. To test this, participants are asked to report the color of a disk when a cue was presented. There is an increase in selection latency when the disk's color changed smoothly compared to randomly. This increase is due to the smooth color change presented after the cue rather than extrapolated predictions based on the color changes presented before. These results support an attentional drag theory, whereby attentional engagement is prolonged when features change smoothly. A computational model provides insights into the potential underlying neural mechanisms.

Probing the Characteristics of Colour-Motion Binding and Its Dependence on Persistent Surface Segregation

Identifying the spatial and temporal characteristics of visual feature binding is a remaining challenge in the science of perception. Within the feature-binding literature, disparate findings have suggested the existence of more than one feature-binding mechanism with differing temporal resolutions. For example, one surprising result is that temporal alternations between two different feature pairings of colour and motion (e.g., orange dots moving left with blue dots moving right) support accurate conjunction discrimination at alternation frequencies of around 10 Hz and greater. However, at lower alternation frequencies around 5 Hz, conjunction discrimination falls to chance. To further investigate this effect, we present two experiments that probe the stimulus characteristics that facilitate or impede feature binding. Using novel manipulations of random dot kinematograms, we identify that facilitating surface representations through temporal integration can enable accurate conjunction discrimination at both intermediate and high alternation frequencies. We also offer a neurally plausible evidence accumulator model to describe these results, removing the need to suggest multiple binding mechanisms acting at different timescales. In effect, we propose a single, flexible binding process, whereby the relatively low temporal resolution for binding features can be circumvented by extracting them from rapidly formed and persistent surface representations.

Differences in perceptual latency estimated from judgments of temporal order, simultaneity and duration are inconsistent

Differences in perceptual latency (ΔL) for two stimuli, such as an auditory and a visual stimulus, can be estimated from temporal order judgments (TOJ) and simultaneity judgments (SJ), but previous research has found evidence that ΔL estimated from these tasks do not coincide. Here, using an auditory and a visual stimulus we confirmed this and further show that ΔL as estimated from duration judgments also does not coincide with ΔL estimated from TOJ or SJ. These inconsistencies suggest that each judgment is subject to different processes that bias ΔL in different ways: TOJ might be affected by sensory interactions, a bias associated with the method of single stimuli and an order difficulty bias; SJ by sensory interactions and an asymmetrical criterion bias; duration judgments by an order duration bias.

Non-binding relationship between visual features

The answer as to how visual attributes processed in different brain loci at different speeds are bound together to give us our unitary experience of the visual world remains unknown. In this study we investigated whether bound representations arise, as commonly assumed, through physiological interactions between cells in the visual areas. In a focal attentional task in which correct responses from either bound or unbound representations were possible, participants discriminated the color or orientation of briefly presented single bars. On the assumption that representations of the two attributes are bound, the accuracy of reporting the color and orientation should co-vary. By contrast, if the attributes are not mandatorily bound, the accuracy of reporting the two attributes should be independent. The results of our psychophysical studies reported here supported the latter, non-binding, relationship between visual features, suggesting that binding does not necessarily occur even under focal attention. We propose a task-contingent binding mechanism, postulating that binding occurs at late, post-perceptual (PP), stages through the intervention of memory.

Contributions of magno- and parvocellular channels to conscious and non-conscious vision

The dorsal and ventral cortical pathways, driven predominantly by magnocellular (M) and parvocellular (P) inputs, respectively, assume leading roles in models of visual information processing. Although in prior proposals, the dorsal and ventral pathways support non-conscious and conscious vision, respectively, recent modelling and empirical developments indicate that each pathway plays important roles in both non-conscious and conscious vision. In these models, the ventral P-pathway consists of one subpathway processing an object's contour features, e.g. curvature, the other processing its surface attributes, e.g. colour. Masked priming studies have shown that feed-forward activity in the ventral P-pathway on its own supports non-conscious processing of contour and surface features. The dorsal M-pathway activity contributes directly to conscious vision of motion and indirectly to object vision by projecting to prefrontal cortex, which in turn injects top-down neural activity into the ventral P-pathway and there 'ignites' feed-forward-re-entrant loops deemed necessary for conscious vision. Moreover, an object's shape or contour remains invisible without the prior conscious registration of its surface properties, which for that reason are taken to comprise fundamental visual qualia. Besides suggesting avenues for future research, these developments bear on several recent and past philosophical issues.

Perceptual asynchrony for motion

Psychophysical experiments show that two different visual attributes, color and motion, processed in different areas of the visual brain, are perceived at different times relative to each other (Moutoussis and Zeki, 1997a). Here we demonstrate psychophysically that two variants of the same attribute, motion, which have the same temporal structure and are processed in the same visual areas, are also processed asynchronously. When subjects were asked to pair up–down motion of dots in one half of their hemifield with up-right motion in the other, they perceived the two directions of motion asynchronously, with the advantage in favor of up-right motion; when they were asked to pair the motion of white dots moving against a black background with that of red dots moving against an equiluminant green background, they perceived the luminant motion first, thus demonstrating a perceptual advantage of luminant over equiluminant motion. These results were not affected by motion speed or perceived motion “streaks.” We thus interpret these results to reflect the different processing times produced by luminant and equiluminant motion stimuli or by different degrees of motion direction change, thus adding to the evidence that processing time within the visual system is a major determinant of perceptual time.

Asynchrony in visual consciousness and the possible involvement of attention

When subjects are asked to perceptually bind rapidly alternating color and motion stimuli, the pairings they report are different from the ones actually occurring in physical reality. A possible explanation for this misbinding is that the time necessary for perception is different for different visual attributes. Such an explanation is in logical harmony with the fact that the visual brain is characterized by different, functionally specialized systems, with different processing times for each; this type of organization naturally leads to different perceptual times for the corresponding attributes. In the present review, the experimental findings supporting perceptual asynchrony are presented, together with the original theoretical explanation behind the phenomenon and its implication for visual consciousness. Alternative theoretical views and additional experimental facts concerning perceptual misbinding are also reviewed, with a particular emphasis given to the role of attention. With few exceptions, most theories converge on the idea that the observed misbinding reflects a difference in perception times, which is in turn due to differences in neuronal processing times for different attributes within the brain. These processing time differences have been attributed to several different factors, attention included, with the possibility of co-existence between them.

Feature binding of a continuously changing object

Consider a feature of a stimulus (such as color, luminance, or spatial frequency) that changes over time along a continuum. When a second stimulus is briefly pulsed with the same feature value as the first stimulus, the two stimuli are not perceived to match. Instead, the continuously changing stimulus is perceived to be further ahead on the feature continuum than the pulsed stimulus [Nat. Neurosci. 3, 489 (2000)]. This shift is quantified by the amount of time ahead on the changing continuum, which is different for various types of features. A basic question is how our percepts are affected when an object has two continuously changing features (such as color and orientation) with different magnitudes of time ahead. This was addressed using a bar continuously changing in both color and orientation. Even though the two features were part of the same object, each feature maintained a distinctly different time ahead. This implies that observers perceived at each moment a combination of color and orientation that never was presented to the eye.

Short Article: Similarity and Binding in Memory: Bound to be Detrimental

The process of binding information from different modalities and sources into an object is ubiquitous in cognition and has been a problem for research and modelling efforts in psychology. This process has been considered by most researchers as necessarily always beneficial to memory. In the present study we provide evidence that binding can be detrimental through the propagation of vulnerabilities to interference. Phonologically similar and dissimilar letters were presented sequentially at different locations on a computer monitor. Participants had to recall either the letters in their order of presentation or the spatial locations at which the letters had appeared. Whether binding was encouraged or not—providing prior knowledge of which dimension to remember—phonological similarity had a detrimental effect on recall of locations. Such a finding poses a challenge to the view that binding is the panacea in enhancing memory capacity.

Mobile computation: spatiotemporal integration of the properties of objects in motion

We demonstrate that, as an object moves, color and motion signals from successive, widely spaced locations are integrated, but letter and digit shapes are not. The features that integrate as an object moves match those that integrate when the eyes move but the object is stationary (spatiotopic integration). We suggest that this integration is mediated by large receptive fields gated by attention and that it occurs for surface features (motion and color) that can be summed without precise alignment but not shape features (letters or digits) that require such alignment. Rapidly alternating pairs of colors and motions were presented at several locations around a circle centered at fixation. The same two stimuli alternated at each location with the phase of the alternation reversing from one location to the next. When observers attended to only one location, the stimuli alternated in both retinal coordinates and in the attended stream: feature identification was poor. When the observer's attention shifted around the circle in synchrony with the alternation, the stimuli still alternated at each location in retinal coordinates, but now attention always selected the same color and motion, with the stimulus appearing as a single unchanging object stepping across the locations. The maximum presentation rate at which the color and motion could be reported was twice that for stationary attention, suggesting (as control experiments confirmed) object-based integration of these features. In contrast, the identification of a letter or digit alternating with a mask showed no advantage for moving attention despite the fact that moving attention accessed (within the limits of precision for attentional selection) only the target and never the mask. The masking apparently leaves partial information that cannot be integrated across locations, and we speculate that for spatially defined patterns like letters, integration across large shifts in location may be limited by problems in aligning successive samples. Our results also suggest that as attention moves, the selection of any given location (dwell time) can be as short as 50 ms, far shorter than the typical dwell time for stationary attention. Moving attention can therefore sample a brief instant of a rapidly changing stream if it passes quickly through, giving access to events that are otherwise not seen.

The 'when' parietal pathway explored by lesion studies

The perception of events in space and time is at the root of our interactions with the environment. The precision with which we perceive visual events in time enables us to act upon objects with great accuracy and the loss of such functions due to brain lesions can be catastrophic. We outline a visual timing mechanism that deals with the trajectory of an object's existence across time, a crucial function when keeping track of multiple objects that temporally overlap or occur sequentially. Recent evidence suggests these functions are served by an extended network of areas, which we call the 'when' pathway. Here we show that the when pathway is distinct from and interacts with the well-established 'where' and 'what' pathways.