The role of attention in motion extrapolation: are moving objects 'corrected' or flashed objects attentionally delayed?

Motion Extrapolation in Visual Processing: Lessons from 25 Years of Flash-Lag Debate

Because of the delays inherent in neural transmission, the brain needs time to process incoming visual information. If these delays were not somehow compensated, we would consistently mislocalize moving objects behind their physical positions. Twenty-five years ago, Nijhawan used a perceptual illusion he called the flash-lag effect (FLE) to argue that the brain's visual system solves this computational challenge by extrapolating the position of moving objects (Nijhawan, 1994). Although motion extrapolation had been proposed a decade earlier (e.g., Finke et al., 1986), the proposal that it caused the FLE and functioned to compensate for computational delays was hotly debated in the years that followed, with several alternative interpretations put forth to explain the effect. Here, I argue, 25 years later, that evidence from behavioral, computational, and particularly recent functional neuroimaging studies converges to support the existence of motion extrapolation mechanisms in the visual system, as well as their causal involvement in the FLE. First, findings that were initially argued to challenge the motion extrapolation model of the FLE have since been explained, and those explanations have been tested and corroborated by more recent findings. Second, motion extrapolation explains the spatial shifts observed in several FLE conditions that cannot be explained by alternative (temporal) models of the FLE. Finally, neural mechanisms that actually perform motion extrapolation have been identified at multiple levels of the visual system, in multiple species, and with multiple different methods. I outline key questions that remain, and discuss possible directions for future research.

When predictions fail: Correction for extrapolation in the flash-grab effect

Motion-induced position shifts constitute a broad class of visual illusions in which motion and position signals interact in the human visual pathway. In such illusions, the presence of visual motion distorts the perceived positions of objects in nearby space. Predictive mechanisms, which could contribute to compensating for processing delays due to neural transmission, have been given as an explanation. However, such mechanisms have struggled to explain why we do not usually perceive objects extrapolated beyond the end of their trajectory. Advocates of this interpretation have proposed a "correction-for-extrapolation" mechanism to explain this: When the object motion ends abruptly, this mechanism corrects the overextrapolation by shifting the perceived object location backwards to its actual location. However, such a mechanism has so far not been empirically demonstrated. Here, we use a novel version of the flash-grab illusion to demonstrate this mechanism. In the flash-grab effect, a target is flashed on a moving background that abruptly changes direction, leading to the mislocalization of the target. Here, we manipulate the angle of the direction change to dissociate the contributions of the background motion before and after the flash. Consistent with previous reports, we observe that perceptual mislocalization in the flash-grab illusion is mainly driven by motion after the flash. Importantly, however, we reveal a small but consistent mislocalization component in the direction opposite to the direction of the first motion sequence. This provides empirical support for the proposed correction-for-extrapolation mechanism, and therefore corroborates the interpretation that motion-induced position shifts might result from predictive interactions between motion and position signals.

Inhibition of return modulates the flash-lag effect

Transient events are known to draw exogenous attention, and visual processing at the attended location is transiently facilitated, but after several hundred milliseconds, attentional processing at the cued location becomes poorer than processing elsewhere, resulting in a slower reaction to a target stimulus that subsequently appears at the cued location. Despite a number of previous studies on this effect, termed inhibition of return (IOR), it is still unclear whether a perceptual process related to the subjective onset time of the target stimulus is disrupted when IOR occurs. In the present study, we used a distinct visual phenomenon termed the flash-lag effect (FLE) as a tool to quantify IOR. The FLE is an illusion in which a flashed stimulus appears to lag behind a moving stimulus, despite being physically aligned. We used an identical stimulus configuration and asked observers to conduct two independent tasks in separate sessions. The first was a simple reaction task to measure the onset reaction time (RT) to an abruptly appearing target. The second was an orientation judgment task to measure the degree of the FLE. Both the RT and the FLE were found to be altered in accordance with IOR, and a significant correlation was demonstrated between the changes in the RT and those in the FLE. These results demonstrate that the perceptual process related to the stimulus onset can be compromised by IOR.

The Flash-Lag, Fröhlich and Related Motion Illusions Are Natural Consequences of Discrete Sampling in the Visual System

The Fröhlich effect and flash-lag effect, in which moving objects appear advanced along their trajectories compared to their actual positions, have defied a simple and consistent explanation. Here, I show that these illusions can be understood as a natural consequence of temporal compression in the human visual system. Discrete sampling at some stage of sensory perception has long been considered, and if it were true, it would necessarily lead to these illusions of motion. I show that the discrete perception hypothesis, with a single free parameter, the perceptual moment or sampling rate, can quantitatively explain all of the scenarios of the Fröhlich and flash-lag effect. I interpret discrete perception as the implementation of data compression in the brain, and our conscious perception as the reconstruction of the compressed input.

The haptic and the visual flash-lag effect and the role of flash characteristics

When a short flash occurs in spatial alignment with a moving object, the moving object is seen ahead the stationary one. Similar to this visual "flash-lag effect" (FLE) it has been recently observed for the haptic sense that participants judge a moving hand to be ahead a stationary hand when judged at the moment of a short vibration ("haptic flash") that is applied when the two hands are spatially aligned. We further investigated the haptic FLE. First, we compared participants' performance in two isosensory visual or haptic conditions, in which moving object and flash were presented only in a single modality (visual: sphere and short color change, haptic: hand and vibration), and two bisensory conditions, in which the moving object was presented in both modalities (hand aligned with visible sphere), but the flash was presented only visually or only haptically. The experiment aimed to disentangle contributions of the flash's and the objects' modalities to the FLEs in haptics versus vision. We observed a FLE when the flash was visually displayed, both when the moving object was visual and visuo-haptic. Because the position of a visual flash, but not of an analogue haptic flash, is misjudged relative to a same visuo-haptic moving object, the difference between visual and haptic conditions can be fully attributed to characteristics of the flash. The second experiment confirmed that a haptic FLE can be observed depending on flash characteristics: the FLE increases with decreasing intensity of the flash (slightly modulated by flash duration), which had been previously observed for vision. These findings underline the high relevance of flash characteristics in different senses, and thus fit well with the temporal-sampling framework, where the flash triggers a high-level, supra-modal process of position judgement, the time point of which further depends on the processing time of the flash.

Visual Transients Reveal the Veridical Position of a Moving Object

The position of a moving object is often mislocalised in the direction of movement. At the input stage of visual processing, the position of a moving object should still be represented veridically, whereas it should become closer to the mislocalised position at a later processing stage responsible for positional judgment. Here, we show that visual transients expose the veridical position of a moving object represented in early visual areas. For example, when a ring is flashed on a moving bar, the part of the bar within the ring is perceived at the veridical position, whereas the part outside the ring is perceived to be ahead of the ring as in the flash-lag effect. Our observations suggest that a filling-in process is triggered at the edges of the flash. This indicates that, in early cortical areas, moving objects are still represented at their veridical positions, and the perceived location is determined by the higher visual areas.

The Modulation of the Flash-Lag Effect by Voluntary Attention

In the flash-lag effect (FLE), a flashing object appears to lag behind a moving object when both happen to be physically aligned to each other. According to an earlier account of the FLE (Baldo and Klein 1995 Nature378 565–566), this perceptual phenomenon would result from differential delays in the perceptual processing of moving and flashing stimuli, presumably involving attentional mechanisms. Here, we have attempted to demonstrate in a more convincing way the participation of voluntary attention as a major component of the FLE. In experiment 1 the observer's attentional set was induced by the spatial probability structure of the visual stimulus. A flashing dot (relative to which the location of a moving dot should be judged) was presented, in separate blocks, at fixed, alternating, or randomly chosen locations. The two former conditions, providing a higher spatial predictability, yielded a smaller FLE than the latter condition, which provided a lower spatial predictability of the flashing dot. In experiment 2 we employed a standard cueing procedure, in which a participant was instructed to shift covertly his/her attentional focus according to a symbolic cue. The cue indicated, with a validity of 80%, the visual hemifield at which the flashing dot would be presented. As predicted by our conceptual model, the mean magnitude of the FLE in the valid trials was significantly smaller than that found in the invalid ones. Therefore, both experiments provided strong evidence supporting the participation of voluntary attention in the FLE. Attentional mechanisms should be seen not as the primary cause of the FLE, but rather as an important modulatory component of a broader process whose spatiotemporal dynamics engenders the FLE and possibly other related phenomena. Even though we elected an account based on the influence of attention on perceptual latencies, our empirical findings are compatible with other theoretical models embraced by the current flash-lag controversy and should be accommodated by every attempt to explain this perceptual phenomenon.

Illusory Causal Crescents: Misperceived Spatial Relations Due to Perceived Causality

When an object A moves toward an object B until they are adjacent, at which point A stops and B starts moving, we often see a collision—ie we see A as the cause of B's motion. The spatiotemporal parameters which mediate the perception of causality have been explored in many studies, but this work is seldom related to other aspects of perception. Here we report a novel illusion, wherein the perception of causality affects the perceived spatial relations among two objects involved in a collision event: observers systematically underestimate the amount of overlap between two items in an event which is seen as a causal collision. This occurs even when the causal nature of the event is induced by a surrounding context, such that estimates of the amount of overlap in the very same event are much improved when the event is displayed in isolation, without a ‘causal’ interpretation. This illusion implies that the perception of causality does not proceed completely independently of other visual processes, but can affect the perception of other spatial properties.

Keeping postdiction simple

Postdiction effects are phenomena in which a stimulus influences the appearance of events taking place before it. In metacontrast masking, for instance, a masking stimulus can render a target stimulus shown before the mask invisible. This and other postdiction effects have been considered incompatible with a simple explanation according to which (i) our perceptual experiences are delayed for only the time it takes for a distal stimulus to reach our sensory receptors and for our neural mechanisms to process it, and (ii) the order in which the processing of stimuli is completed corresponds with the apparent temporal order of stimuli. As a result, the theories that account for more than a single postdiction effect reject at least one of these theses. This paper presents a new framework for the timing of experiences-the non-linear latency difference view-in which the three most discussed postdiction effects-apparent motion, the flash-lag effect, and metacontrast masking-can be accounted for while simultaneously holding theses (i) and (ii). This view is grounded in the local reentrant processes, which are known to have a crucial role in perception. Accordingly, the non-linear latency difference view is both more parsimonious and more empirically plausible than the competing theories, all of which remain largely silent about the neural implementation of the mechanisms they postulate.

Truncation of the flash-lag effect by a fixed spatial landmark

The flash-lag effect is a visual illusion where a moving image is perceived to be advanced in its spatial location relative to a flashed image. Multiple studies have shown that the flash-lag effect can be enhanced by increasing the uncertainty of the moving and/or flashed images. However, little is known about the effect of task-irrelevant visual objects on the flash-lag effect. We were interested to see whether a task-irrelevant spatial landmark might reduce uncertainty and hence reduce the flash-lag effect. We placed a fixed bar between moving and flashed bars while measuring the flash-lag effect in six participants. For most participants, the fixed bar substantially truncated the flash-lag effect. The effect was maximal when the fixed bar was aligned with the flashed bar and decreased when the fixed bar was positioned more peripherally. A second experiment with two participants used a smaller fixed bar; the smaller bar had less truncation effect in one participant, while the other participant showed similar truncation regardless of the fixed bar size. Our results support models that place the locus of the flash-lag effect in higher-order brain areas, e.g., the parietal lobe.

Do the flash-lag effect and representational momentum involve similar extrapolations?

In the flash-lag effect (FLE) and in representational momentum (RM), the represented position of a moving target is displaced in the direction of motion. Effects of numerous variables on the FLE and on RM are briefly considered. In many cases, variables appear to have the same effect on the FLE and on RM, and this is consistent with a hypothesis that displacements in the FLE and in RM result from overlapping or similar mechanisms. In other cases, variables initially appear to have different effects on the FLE and on RM, but accounts reconciling those apparent differences with a hypothesis of overlapping or similar mechanisms are suggested. Given that RM is simpler and accounts for a wider range of findings (i.e., RM involves a single stimulus rather than the relationship between two stimuli, RM accounts for displacement in absolute position of a single stimulus and for differences in relative position of two stimuli), it is suggested that (at least some cases of) the FLE might be a special case of RM in which the position of the target is assessed relative to the position of another stimulus (i.e., the flashed object) rather than relative to the actual position of the target.

Effects of spatial cuing on the onset repulsion effect

Effects of cuing the onset (initial) location of a moving target on memory for the onset location of that target were examined. If a cue presented prior to target onset indicated the location where that target would appear, the onset repulsion effect (in which the judged initial location of the target was displaced in the direction opposite to target motion) was decreased, and the onset repulsion effect was smaller if the cue was valid than if the cue was invalid. If a cue presented during target motion or after the target vanished indicated the location where that target had appeared, the onset repulsion effect was eliminated. The data (1) suggest that positional uncertainty might contribute to the onset repulsion effect, (2) provide the first evidence of an effect of expectancy regarding target trajectory on the onset repulsion effect, and (3) are partially consistent with previous data involving effects of attention and spatial cuing on the Fröhlich effect and on representational momentum.

An empirical explanation of the flash-lag effect

When a flash of light is presented in physical alignment with a moving object, the flash is perceived to lag behind the position of the object. This phenomenon, known as the flash-lag effect, has been of particular interest to vision scientists because of the challenge it presents to understanding how the visual system generates perceptions of objects in motion. Although various explanations have been offered, the significance of this effect remains a matter of debate. Here, we show that: (i) contrary to previous reports based on limited data, the flash-lag effect is an increasing nonlinear function of image speed; and (ii) this function is accurately predicted by the frequency of occurrence of image speeds generated by the perspective transformation of moving objects. These results support the conclusion that perceptions of the relative position of a moving object are determined by accumulated experience with image speeds, in this way allowing for visual behavior in response to real-world sources whose speeds and positions cannot be perceived directly.

Visual prediction: psychophysics and neurophysiology of compensation for time delays

A necessary consequence of the nature of neural transmission systems is that as change in the physical state of a time-varying event takes place, delays produce error between the instantaneous registered state and the external state. Another source of delay is the transmission of internal motor commands to muscles and the inertia of the musculoskeletal system. How does the central nervous system compensate for these pervasive delays? Although it has been argued that delay compensation occurs late in the motor planning stages, even the earliest visual processes, such as phototransduction, contribute significantly to delays. I argue that compensation is not an exclusive property of the motor system, but rather, is a pervasive feature of the central nervous system (CNS) organization. Although the motor planning system may contain a highly flexible compensation mechanism, accounting not just for delays but also variability in delays (e.g., those resulting from variations in luminance contrast, internal body temperature, muscle fatigue, etc.), visual mechanisms also contribute to compensation. Previous suggestions of this notion of "visual prediction" led to a lively debate producing re-examination of previous arguments, new analyses, and review of the experiments presented here. Understanding visual prediction will inform our theories of sensory processes and visual perception, and will impact our notion of visual awareness.

Predictive perceptions, predictive actions, and beyond

Challenges to visual prediction as an organizing concept come from three main sources: (1) from observations arising from the results of experiments employing unpredictable motion, (2) from the assertions that motor processes compensate for all neural delays, and (3) from multiple interpretations specific to the flash-lag effect. One clarification that has emerged is that visual prediction is a process that either complements or reflects non-visual (e.g., motor) prediction.

Tracking the changing features of multiple objects: progressively poorer perceptual precision and progressively greater perceptual lag

To measure the limits on attentive tracking of continuously changing features, in our task objects constantly changed smoothly and unpredictably in orientation, spatial period or position. Observers reported the last state of one of the objects. We observed a gradual decline in performance as the number of tracked objects increased, implicating a graded processing resource. Additionally, responses were more similar to previous states of the tracked object than its final state, especially in the case of spatial frequency. Indeed for spatial frequency, this perceptual lag reached 250ms when tracking four objects. The pattern of the perceptual lags, the graded effect of set size, and the double-report performance suggest the presence of both serial and parallel processing elements.

The flash-lag effect during illusory chopstick rotation

In the 'flash-lag' effect, a static object that is briefly flashed next to a moving object appears to lag behind the moving object. A flash was put up next to an intersection that appeared to be moving clockwise along a circular path but was actually moving counterclockwise [the chopstick illusion; Anstis, 1990, in AI and the Eye Eds A Blake, T Troscianko (London: John Wiley) pp 105 117; 2003, in Levels of Perception Eds L Harris, M Jenkin (New York: Springer) pp 90 93]. As a result, the flash appeared displaced clockwise. This was appropriate to the physical, not the subjective, direction of rotation, and it suggests that the flash-lag illusion occurs early in the visual system, before motion signals are parsed into moving objects.

Dividing attention in the flash-lag illusion

A dual-task paradigm was used to examine the effect of withdrawing attentional and/or cognitive resources from the flash-lag judgment. The flash-lag illusion was larger, and performance in a detection task was generally poorer, under dual-task conditions than in single-task control conditions. These effects were particularly pronounced when decisions in the two tasks were required simultaneously, as compared to when they could be made sequentially. The results suggest that a time-consuming process is involved in the flash-lag decision, of such a nature that prolonging the process increases the magnitude of the illusion.

Flash-lag chimeras: The role of perceived alignment in the composite face effect

Spatial alignment of different face halves results in a configuration that mars the recognition of the identity of either face half (). What would happen to the recognition performance for face halves that were aligned on the retina but were perceived as misaligned, or were misaligned on the retina but were perceived as aligned? We used the 'flash-lag' effect () to address these questions. We created chimeras consisting of a stationary top half-face initially aligned with a moving bottom half-face. Flash-lag chimeras were better recognized than their stationary counterparts. However when flashed face halves were presented physically ahead of moving halves thereby nulling the flash-lag effect, recognition was impaired. This counters the notion that relative movement between the two face halves per se is sufficient to explain better recognition of flash-lag chimeras. Thus, the perceived spatial alignment of face halves (despite retinal misalignment) impairs recognition, while perceived misalignment (despite retinal alignment) does not.

Attention 'capture' by the flash-lag flash

We report data from eight participants who made alignment judgements between a moving object and a stationary, continuously visible 'landmark'. A reversing object had to overshoot the landmark by a significant amount in order to appear to reverse aligned with it. In addition, an adjacent flash irrelevant to the judgment task reliably increased this illusory 'foreshortening'. This and other results are most simply explained by a model in which the flash causes attentional capture, complemented by processes of temporal integration, or backward inhibition, and object representation. A flash used to probe the perception of a moving object's position disrupts that very perception.

Manual control of the visual stimulus reduces the flash-lag effect

We investigated how observers' control of the stimulus change affects temporal aspects of visual perception. We compared the flash-lag effects for motion (Experiment 1) and for luminance (Experiment 2) under several conditions that differed in the degree of the observers' control of change in a stimulus. The flash-lag effect was salient if the observers passively viewed the automatic change in the stimulus. However, if the observers controlled the stimulus change by a computer-mouse, the flash-lag effect was significantly reduced. In Experiment 3, we examined how observers' control of the stimulus movement by a mouse affects the reaction time for the shape change in the moving stimulus and flash. Results showed that the control reduced the reaction time for both moving stimulus and flash. These results suggest that observers' manual control of the stimulus reduces the flash-lag effect in terms of facilitation in visual processing.

The flash-lag effect is reduced when the flash is perceived as a sensory consequence of our action

The flash-lag effect (FLE) is defined as an error in localization that consists of perceiving a flashed object to lag behind a moving one when both are presented in physical alignment. Previous studies have addressed the question if it is the predictability of the flash, or the moving object, that modulates the amount of the error. However, the case when the flash is self-generated, and hence can be internally predicted, has not yet been addressed. In Experiment 1, we compare four conditions: flash unpredictable, flash externally predicted by a beep, flash internally generated (and predicted) by pressing a key, and flash triggered by a key press but temporally unpredictable. The FLE was significantly reduced only when the flash was internally predictable. In Experiment 2, we rule out the possibility that the reduction of the FLE was due to the use of the key press as a temporal marker. We conclude that when the flash is perceived as a sensory consequence of our own action, its detection can be speeded up, thereby resulting in a reduction of the FLE. A third experiment supports this interpretation. The mechanism by virtue of which the detection is accelerated could be related to efferent signals from motor areas predicting the sensory consequences of our actions.

Differential latencies and the dynamics of the position computation process for moving targets, assessed with the flash-lag effect

To investigate the dynamics of the position computation process for a moving object in human vision, we measured the response to a continuous change in position at a constant velocity (ramp-response) using the flash-lag illusion. In this illusion, flashed and moving objects appear spatially offset when their retinal images are physically aligned. The steady-state phase of the ramp-response was probed using the "continuous-motion" (CM) paradigm, in which the motion of the moving object starts long before the occurrence of the flash. To probe the transient phase of the ramp-response, we used the "flash-initiated cycle" (FIC) paradigm, in which the motion of the moving object starts within a short time window around the presentation of the flash. The sampling instant of the ramp-response was varied systematically by changing the luminance or the presentation time of the flashed stimulus. We found that the perceived flash misalignments in the FIC and CM paradigms were approximately equal when sampling of the ramp-response occurred after a relatively long delay from the onset of motion and, were significantly different when sampling of the ramp-response occurred at a relatively short delay. The systematic variations in the perceived misalignment between the moving and flashed stimuli as a function of stimulus parameters are compared to the predictions of our differential latency and to alternative models of position computation.

Perceived onset time and position of a moving stimulus

During the few past years, there has been a growing interest in the timing and locating of moving stimuli. The most popular spatio-temporal phenomena that have been studied are the flash-lag effect (FLE) [Nature 370 (1994) 256] and the Fröhlich effect (FE) [Z. Sinnesphysiol. 54 (1923) 58]. Most often these phenomena are examined by some spatial task (e.g., judging whether moving and flashed stimuli are spatially aligned or not; explicitly pointing or adjusting the moving stimulus position). Usually, from the measured spatial offset temporal differences in processing of moving and stationary stimuli are inferred. Our experiments show that this practice may not be justified because the spatial and temporal properties were clearly disassociated for the movement onset perception. The disassociation demonstrates that the FLE and FE are most probably based on different internal representations.

Perceptual acceleration of objects in stream: evidence from flash-lag displays

An object in continuous motion is perceived ahead of the briefly flashed object, although the two images are physically aligned (Nijhawan, 1994), the phenomenon called flash-lag effect. Flash-lag effects have been found also with other continuously changing features such as color, pattern entropy, and brightness (Sheth, Nijhawan, & Shimojo, 2000) as well as with streamed pre- and post-target input without any change of the feature values of streaming items in feature space (Bachmann & Põder, 2001a. 2001b). We interpret all instances of the flash-lag as a consequence of a more fundamental property of conscious perception in general: acceleration of the speed with which samples of perceptual information become represented in explicit format immediately after the stimulation onset. Decreased visual latency of the samples of stimulus information from the streamed input leads to the relative perceptual lag for the separately flashed stimulus because it is not preceded by adjacent sensory input that would have accelerated its perception. Experimental support for the notion of perceptual acceleration is reviewed.

Attention maintains mental extrapolation of target position: irrelevant distractors eliminate forward displacement after implied motion

Observers' judgments of the final position of a moving target are typically shifted in the direction of implied motion ("representational momentum"). The role of attention is unclear: visual attention may be necessary to maintain or halt target displacement. When attention was captured by irrelevant distractors presented during the retention interval, forward displacement after implied target motion disappeared, suggesting that attention may be necessary to maintain mental extrapolation of target motion. In a further corroborative experiment, the deployment of attention was measured after a sequence of implied motion, and faster responses were observed to stimuli appearing in the direction of motion. Thus, attention may guide the mental extrapolation of target motion. Additionally, eye movements were measured during stimulus presentation and retention interval. The results showed that forward displacement with implied motion does not depend on eye movements. Differences between implied and smooth motion are discussed with respect to recent neurophysiological findings.

Flag errors in soccer games: the flash-lag effect brought to real life

In soccer games, an attacking player is said to be in an offside position if he or she is closer to the opponents' goal line than both the ball and the second-to-last defender. It is an offence for the attacker to be in an offside position and in active play at the moment a fellow team member plays the ball. Assistant referees often make mistakes when judging an offside offence, probably because of optical errors arising from the viewing angle adopted by them (Oudejans, Verheijen, Bakker, Gerrits, Steinbrückner, Beek, 2000 Nature 404 33). Looking more closely at Oudejans et al's data, we show evidence that the flash-lag effect may contribute significantly to these mistakes. Participation of the flash-lag effect in assistant referees' misjudgments would take this perceptual phenomenon from laboratory setups to a real-life situation for the first time.

Latency correction explains the classical geometrical illusions

There is a significant delay between the time when light hits the retina and the time of the consequent percept. It has been hypothesized that the visual system attempts to correct for this latency by generating a percept representative of the way the world probably is at the time the percept is elicited, rather than a percept of the recent past. Here we show that such a 'perceiving the present' hypothesis explains a number of classical geometrical illusions: the Hering, Orbison, Müller-Lyer, Double Judd, Poggendorff, Corner, and Upside-down-T illusions. Each stimulus is perceived as it would project in the next moment were the observer moving through the scene the stimulus probably represents. We also examine one general class of predictions made by the hypothesis, and report psychophysical experiments confirming the predictions.

Smooth shifts of visual attention

To examine whether visual attention shifts continuously across the visual field, we measured sensitivity to a small flash presented at various locations while the observer was tracking a moving target in an ambiguous apparent motion display. The sensitivity peaked near the target and the peak shifted smoothly along the apparent motion path. Since the peak-shift speed varied with the speed of the tracked target, we conclude that the attention mechanism selects the location to facilitate processing by tracking the target disk continuously. Attention does not simply select a location for enhanced processing, but rather predicts the future location of the object of interest based on its velocity.

Neural delays, visual motion and the flash-lag effect

In the primate visual system, there is a significant delay in the arrival of photoreceptor signals in visual cortical areas. Since Helmholtz, scientists have pondered over the implications of these delays for human perception. Do visual delays cause the ' position of a moving object to lag its 'real' position? This question has recently been re-evaluated in the context of the flash-lag phenomenon, in which a flashed object appears to lag behind a moving object, when physically the two objects are co-localized at the instant of the flash. This article critically examines recent accounts of this phenomenon, assesses its biological significance, and offers new hypotheses.

The attentional modulation of the flash-lag effect

If a dot is flashed in perfect alignment with a pair of dots rotating around the visual fixation point, most observers perceive the rotating dots as being ahead of the flashing dot (flash-lag effect). This perceptual effect has been interpreted to result from the perceptual extrapolation of the moving dots, the differential visual latencies between flashing and moving stimuli, as well as the modulation of attentional mechanisms. Here we attempted to uncouple the attentional effects brought about by the spatial predictability of the flashing dot from the sensory effects dependent on its visual eccentricity. The stimulus was a pair of dots rotating clockwise around the fixation point. Another dot was flashed at either the upper right or the lower left of the visual field according to three separate blocked situations: fixed, alternate and random positions. Twenty-four participants had to judge, in all three situations, the location of the rotating dots in relation to the imaginary line connecting the flashing dot and the fixation point at the moment the dot was flashed. The flash-lag effect was observed in all three situations, and a clear influence of the spatial predictability of the flashing dot on the magnitude of the perceptual phenomenon was revealed, independently of sensory effects related to the eccentricity of the stimulus in the visual field. These findings are consistent with our proposal that, in addition to sensory factors, the attentional set modulates the magnitude of the differential latencies that give rise to the flash-lag phenomenon.

Physical, neural, and mental timing

The conclusions drawn by Benjamin Libet from his work with colleagues on the timing of somatosensorial conscious experiences has met with a lot of praise and criticism. In this issue we find three examples of the latter. Here I attempt to place the divide between the two opponent camps in a broader perspective by analyzing the question of the relation between physical timing, neural timing, and experiential (mental) timing. The nervous system does a sophisticated job of recombining and recoding messages from the sensorial surfaces and if these processes are slighted in a theory, it might become necessary to postulate weird operations, including subjective back-referral. Neuroscientifically inspired theories are of necessity still based on guesses, extrapolations, and philosophically dubious manners of speech. They often assume some neural correlate of consciousness (NCC) as a part of the nervous system that transforms neural activity in reportable experiences. The majority of neuroscientists appear to assume that the NCC can compare and bind activity patterns only if they arrive simultaneously at the NCC. This leads to a search for synchrony or to theories in terms of the compensation of differences in neural delays (latencies). This is the main dimension of the Libet discussion. Examples from vision research, such as "temporal-binding-by-synchrony" and the "flash-lag" effect, are then used to illustrate these reasoning patterns in more detail. Alternatively one could assume symbolic representations of time and space (symbolic "tags") that are not coded in their own dimension (not time in time and space in space). Unless such tags are multiplexed with the quality message (tickle, color, or motion), one gets a binding problem for tags. One of the hidden aspects of the discussion between Libet and opponents appears to be the following. Is the NCC smarter than the rest of the nervous system, so that it can solve the problems of local sign (e.g., "where is the event"?) and timing (e.g., "when did it occur?" and "how long did it last?") on its own, or are these pieces of information coded symbolically early on in the system? A supersmart NCC appears to be the assumption of Libet's camp (which includes Descartes, but also mystics). The wish to distribute the smartness evenly across all stages of processing in the nervous system (smart recodings) appears to motivate the opponents. I argue that there are reasons to side with the latter group.

Evidence for an attentional component of the perceptual misalignment between moving and flashing stimuli

If a pair of dots, diametrically opposed to each other, is flashed in perfect alignment with another pair of dots rotating about the visual fixation point, most observers perceive the rotating dots as being ahead of the flashing dots (flash-lag effect). This psychophysical effect was first interpreted as the result of a perceptual extrapolation of the position of the moving dots. Also, it has been conceived as the result of differential visual latencies between flashing and moving stimuli, arising from purely sensory factors and/or expressing the contribution of attentional mechanisms as well. In a series of two experiments, we had observers judge the relative position between rotating and static dots at the moment a temporal marker was presented in the visual field. In experiment 1 we manipulated the nature of the temporal marker used to prompt the alignment judgment. This resulted in three main findings: (i) the flash-lag effect was observed to depend on the visual eccentricity of the flashing dots; (ii) the magnitude of the flash-lag effect was not dependent on the offset of the flashing dot; and (iii) the moving stimulus, when suddenly turned off, was perceived as lagging behind its disappearance location. Taken altogether, these results suggest that neither visible persistence nor motion extrapolation can account for the perceptual flash-lag phenomenon. The participation of attentional mechanisms was investigated in experiment 2, where the magnitude of the flash-lag effect was measured under both higher and lower predictability of the location of the flashing dot. Since the magnitude of the flash-lag effect significantly increased with decreasing predictability, we conclude that the observer's attentional set can modulate the differential latencies determining this perceptual effect. The flash-lag phenomenon can thus be conceived as arising from differential visual latencies which are determined not only by the physical attributes of the stimulus, such as its luminance or eccentricity, but also by attentional mechanisms influencing the delays involved in the perceptual processing.

The influence of visual motion on perceived position

The ability of the visual system to localize objects is one of its most important functions and yet remains one of the least understood, especially when either the object or the surrounding scene is in motion. The specific process that assigns positions under these circumstances is unknown, but two major classes of mechanism have emerged: spatial mechanisms that directly influence the coded locations of objects, and temporal mechanisms that influence the speed of perception. Disentangling these mechanisms is one of the first steps towards understanding how the visual system assigns locations to objects when there are motion signals present in the scene.

The impact of spatiotemporal sampling on time-to-contact judgments

When motion in the frontoparallel plane is temporally sampled, it is often perceived to be slower than its continuous counterpart. This finding stands in contrast to humans' ability to extrapolate and anticipate constant-velocity motion. We investigated whether this sampling bias generalizes to motion in the sagittal plane (i.e., objects approaching the observer). We employed a paradigm in which observers judged the arrival time of an oncoming object. We found detrimental effects of time sampling on both perceived time to contact and time to passage. Observers systematically overestimated the time it would take a frontally approaching object to intersect their eye plane. To rule out artifacts inherent in computer simulation, we replicated the experiment, using real objects. The bias persisted and proved to be robust across a large range of temporal and spatial variations. Energy and pooling mechanisms are discussed in an attempt to understand the effect.

Attention shifts and memory averaging

When observers are asked to localize the final position of a moving stimulus, judgements may be influenced by additional elements that are presented in the visual scene. Typically, judgements arc biased toward a salient non-target element. It has been assumed that the non-target element acts as a landmark and attracts the remembered final target position. The present study investigated the effects of briefly flashed non-target elements on localization performance. Similar to landmark attraction, localization was biased toward these elements. However, an influence was only noted if the distractor was presented at the time of target disappearance or briefly thereafter. It is suggested that memory traces of distracting elements are only averaged with the final target position if they are highly activated at the time the target vanishes.

A flash-lag effect in random motion

The flash-lag effect refers to the phenomenon in which a flash adjacent to a continuously moving object is perceived to lag behind it. To test three previously proposed hypotheses (motion extrapolation, positional averaging, and differential latency), a new stimulus configuration, to which the three hypotheses give different predictions, was introduced. Instead of continuous motion, a randomly jumping bar was used as the moving stimulus, relative to which the position of the flash was judged. The results were visualized as a spatiotemporal correlogram, in which the response to a flash was plotted at the space-time relative to the position and onset of the jumping bar. The actual human performance was not consistent with any of the original hypotheses. However, all the results were explained well if the differential latency was assumed to fluctuate considerably, its probability density function being approximated by Gaussian. Also, the model fit well with previously published data on the flash-lag effect.

Perceptual organization of moving stimuli modulates the flash-lag effect

When a visual stimulus is flashed at a given location the moment a second moving stimulus arrives at the same location, observers report the flashed stimulus as spatially lagging behind the moving stimulus (the flash-lag effect). The authors investigated whether the global configuration (perceptual organization) of the moving stimulus influences the magnitude of the flash-lag effect. The results indicate that a flash presented near the leading portion of a moving stimulus lags significantly more than a flash presented near the trailing portion. This result also holds for objects consisting of several elements that group to form a unitary percept of an object in motion. The present study demonstrates a novel interaction between the global configuration of moving objects and the representation of their spatial position and may provide a new and useful tool for the study of perceptual organization.

Neuronal latencies and the position of moving objects

Neuronal latencies delay the registration of the visual signal from a moving object. By the time the visual input reaches brain structures that encode its position, the object has already moved on. Do we perceive the position of a moving object with a delay because of neuronal latencies? Or is there a brain mechanism that compensates for latencies such that we perceive the true position of a moving object in real time? This question has been intensely debated in the context of the flash-lag illusion: a moving object and an object flashed in alignment with it appear to occupy different positions. The moving object is seen ahead of the flash. Does this show that the visual system extrapolates the position of moving objects into the future to compensate for neuronal latencies? Alternative accounts propose that it simply shows that moving and flashed objects are processed with different delays, or that it reflects temporal integration in brain areas that encode position and motion. The flash-lag illusion and the hypotheses put forward to explain it lead to interesting questions about the encoding of position in the brain. Where is the 'where' pathway and how does it work?

'Perceiving the present' as a framework for ecological explanations of the misperception of projected angle and angular size

An implicit, underlying assumption of most Helmholtzian/Bayesian approaches to perception is the hypothesis that the scene an observer perceives is the probable source of the proximal stimulus. There is, however, a nontrivial latency (on the order of 100 ms) between the time of a proximal stimulus and the time a visual percept is elicited. It seems plausible that it would be advantageous for an observer to have, at any time t, a percept representative of what is out there at that very time t, not a percept of the recent past. If this is so, it implies a modification to the implicit hypothesis underlying most existing probabilistic approaches to perception: the new hypothesis is that, given the proximal stimulus, the scene an observer perceives is the probable scene present at the time of the percept. That is, the hypothesis is that what an observer perceives is not the probable source of the proximal stimulus, but the probable way the probable source will be when the percept actually occurs. A model of an observer's typical movements in the world is developed, and it is shown that projected angles are perceived in a way consistent with the way the probable source will project to the eye after a small time period of forward movement by the observer. The predicted and actual direction of projected-angle misperception is sometimes toward 90 degrees and sometimes away from 90 degrees, depending on whether the probable source angle is lying in a plane parallel or perpendicular to the probable direction of motion, respectively. The perception of angular size for lines in a figure with cues they are lying in a plane perpendicular to the direction of motion is also shown to fit the predictions of the model.