Mental extrapolation of motion modulates responsiveness to visual stimuli

Seeing and extrapolating motion trajectories share common informative activation patterns in primary visual cortex

The natural environment is dynamic and moving objects become constantly occluded, engaging the brain in a challenging completion process to estimate where and when the object might reappear. Although motion extrapolation is critical in daily life-imagine crossing the street while an approaching car is occluded by a larger standing vehicle-its neural underpinnings are still not well understood. While the engagement of low-level visual cortex during dynamic occlusion has been postulated, most of the previous group-level fMRI-studies failed to find evidence for an involvement of low-level visual areas during occlusion. In this fMRI-study, we therefore used individually defined retinotopic maps and multivariate pattern analysis to characterize the neural basis of visible and occluded changes in motion direction in humans. To this end, participants learned velocity-direction change pairings (slow motion-upwards; fast motion-downwards or vice versa) during a training phase without occlusion and judged the change in stimulus direction, based on its velocity, during a following test phase with occlusion. We find that occluded motion direction can be predicted from the activity patterns during visible motion within low-level visual areas, supporting the notion of a mental representation of motion trajectory in these regions during occlusion.

Topical Review: Visual Performance Assessments for Sport

SIGNIFICANCE

Eye care professionals seek to provide effective vision care for a variety of patient needs, including performance in sports and recreational pursuits. This review provides an analysis of common visual performance assessments, including summaries of recent clinical research from a diverse array of professional literature.Vision is recognized as an important element of sports performance. Elite athletes frequently demonstrate exceptional abilities to see and respond effectively in sports competition. Which visual factors are important and how to most effectively assess visual performance are the sources of much debate. This topical review presents an evidence-based review of the common visual performance factors assessed in athletes, beginning with guidance for conducting a visual task analysis for the variety of sports that athlete patients may compete in. An information processing model is used to provide a framework for understanding the contributions of the many visual performance factors used during sports.

Spatial and temporal adaptation of predictive saccades based on motion inference

Moving objects are often occluded behind larger, stationary objects, but we can easily predict when and where they reappear. Here, we show that the prediction of object reappearance is subject to adaptive learning. When monkeys generated predictive saccades to the location of target reappearance, systematic changes in the location or timing of target reappearance independently altered the endpoint or latency of the saccades. Furthermore, spatial adaptation of predictive saccades did not alter visually triggered reactive saccades, whereas adaptation of reactive saccades altered the metrics of predictive saccades. Our results suggest that the extrapolation of motion trajectory may be subject to spatial and temporal recalibration mechanisms located upstream from the site of reactive saccade adaptation. Repetitive exposure of visual error for saccades induces qualitatively different adaptation, which might be attributable to different regions in the cerebellum that regulate learning of trajectory prediction and saccades.

Intuitive physics of gravitational motion as shown by perceptual judgment and prediction-motion tasks

In Experiment 1, we explored participants' perceptual knowledge of vertical fall by presenting them with virtually simulated polystyrene or wooden spheres falling to the ground from about two meters high. Participants rated the perceived naturalness of the motion. Besides the implied mass of the sphere, we manipulated the motion pattern (i.e., uniform acceleration vs. uniform velocity), and the magnitude of acceleration or velocity. Results show that relatively low values of acceleration or velocity were judged as natural for the polystyrene sphere, whereas relatively high values of acceleration or velocity were judged as natural for the wooden sphere. In Experiment 2, the same stimuli of Experiment 1 were used, but the sphere disappeared behind an invisible occluder at some point of its trajectory. Participants were asked to predict the time-to-contact (TTC) of the sphere with the ground by pressing a key at the exact time of impact of the lower edge of the sphere with the floor of the room. Results show that the estimated TTC for the simulated wooden sphere was slightly but consistently smaller than the estimated TTC for the simulated polystyrene sphere. The influence of the implied mass on participants' responses might be the manifestation of two processes, namely an explicit 'heavy-fast, light-slow' heuristic, and/or an implicit, automatic association between mass and falling speed.

Motion prediction at low contrast

Accurate motion prediction is fundamental for survival. How does this reconcile with the well-known speed underestimation of low-contrast stimuli? Here we asked whether this contrast-dependent perceptual bias is retained in motion prediction under two different saccadic planning conditions: making a saccade to an occluded moving target, and real-time gaze interaction with multiple moving targets. In a first experiment, observers made a saccade to the mentally extrapolated position of a moving target (imagery condition). In a second experiment, observers had to prevent collisions among multiple moving targets by glancing at them through a gaze-contingent display or by hitting them with the touchpad cursor (interaction condition). In both experiments, target contrast was manipulated. We found that, whereas saccades to the imagined moving target were systematically biased by contrast, the gaze interaction performance, as measured by missed collisions, was generally unaffected - even though low-contrast targets looked slower. Interceptive actions increased at low contrast, but only when the gaze was used for interaction. Thus, perceptual speed underestimation transfers to saccades made to imagined low-contrast targets, without however necessarily being detrimental to effective performance when real-time interaction with multiple targets is required. This differential effect of stimulus contrast suggests that in complex dynamic conditions saccades are rather tolerant to visual speed biases.

Contribution of Visuospatial and Motion-Tracking to Invisible Motion

People experience an object's motion even when it is occluded. We investigate the processing of invisible motion in three experiments. Observers saw a moving circle passing behind an invisible, irregular hendecagonal polygon and had to respond as quickly as possible when the target had “just reappeared” from behind the occluder. Without explicit cues allowing the end of each of the eight hidden trajectories to be predicted (length ranging between 4.7 and 5 deg), we found as expected, if visuospatial attention was involved, anticipation errors, providing that information on pre-occluder motion was available. This indicates that the observers, rather than simply responding when they saw the target, tended to anticipate its reappearance (Experiment 1). The new finding is that, with a fixation mark indicating the center of the invisible trajectory, a linear relationship between the physical and judged occlusion duration is found, but not without it (Experiment 2) or with a fixation mark varying in position from trial to trial (Experiment 3). We interpret the role of central fixation in the differences in distinguishing trajectories smaller than 0.3 deg, by suggesting that it reflects spatiotemporal computation and motion-tracking. These two mechanisms allow visual imagery to form of the point symmetrical to that of the disappearance, with respect to fixation, and then for the occluded moving target to be tracked up to this point.

Covert enaction at work: Recording the continuous movements of visuospatial attention to visible or imagined targets by means of Steady-State Visual Evoked Potentials (SSVEPs)

Whereas overt visuospatial attention is customarily measured with eye tracking, covert attention is assessed by various methods. Here we exploited Steady-State Visual Evoked Potentials (SSVEPs) - the oscillatory responses of the visual cortex to incoming flickering stimuli - to record the movements of covert visuospatial attention in a way operatively similar to eye tracking (attention tracking), which allowed us to compare motion observation and motion extrapolation with and without eye movements. Observers fixated a central dot and covertly tracked a target oscillating horizontally and sinusoidally. In the background, the left and the right halves of the screen flickered at two different frequencies, generating two SSVEPs in occipital regions whose size varied reciprocally as observers attended to the moving target. The two signals were combined into a single quantity that was modulated at the target frequency in a quasi-sinusoidal way, often clearly visible in single trials. The modulation continued almost unchanged when the target was switched off and observers mentally extrapolated its motion in imagery, and also when observers pointed their finger at the moving target during covert tracking, or imagined doing so. The amplitude of modulation during covert tracking was ∼25-30% of that measured when observers followed the target with their eyes. We used 4 electrodes in parieto-occipital areas, but similar results were achieved with a single electrode in Oz. In a second experiment we tested ramp and step motion. During overt tracking, SSVEPs were remarkably accurate, showing both saccadic-like and smooth pursuit-like modulations of cortical responsiveness, although during covert tracking the modulation deteriorated. Covert tracking was better with sinusoidal motion than ramp motion, and better with moving targets than stationary ones. The clear modulation of cortical responsiveness recorded during both overt and covert tracking, identical for motion observation and motion extrapolation, suggests to include covert attention movements in enactive theories of mental imagery.

Probing the involvement of the earliest levels of cortical processing in motion extrapolation with rapid forms of visual motion priming and adaptation

In this study, we investigated the effect of brief motion priming and adaptation, occurring at the earliest levels of the cortical visual stream, on time-to-contact (TTC) estimation of a target passing behind an occluder. By using different exposure times of directional motion presented in the occluder area prior to the target's disappearance behind it, our aim was to modulate (prime or adapt) the extrapolated motion of the invisible target, thus producing different TTC estimates. Our results showed that longer (yet subsecond) exposures to motion in the same direction as the target produced late TTC estimates, whereas shorter exposures produced shorter TTC estimates, indicating that rapid forms of motion adaptation and motion priming affect extrapolated motion. Our findings suggest that motion extrapolation might occur at the earliest levels of cortical processing of motion, at which these rapid mechanisms of priming and adaptation take place.

Asymmetric saccade reaction times to smooth pursuit

Before initiating a saccade to a moving target, the brain must take into account the target’s eccentricity as well as its movement direction and speed. We tested how the kinematic characteristics of the target influence the time course of this oculomotor response. Participants performed a step-ramp task in which the target object stepped from a central to an eccentric position and moved at constant velocity either to the fixation position (foveopetal) or further to the periphery (foveofugal). The step size and target speed were varied. Of particular interest were trials that exhibited an initial saccade prior to a smooth pursuit eye movement. Measured saccade reaction times were longer in the foveopetal than in the foveofugal condition. In the foveopetal (but not the foveofugal) condition, the occurrence of an initial saccade, its reaction time as well as the strength of the pre-saccadic pursuit response depended on both the target’s speed and the step size. A common explanation for these results may be found in the neural mechanisms that select between oculomotor response alternatives, i.e., a saccadic or smooth response.

Filling gaps in visual motion for target capture

A remarkable challenge our brain must face constantly when interacting with the environment is represented by ambiguous and, at times, even missing sensory information. This is particularly compelling for visual information, being the main sensory system we rely upon to gather cues about the external world. It is not uncommon, for example, that objects catching our attention may disappear temporarily from view, occluded by visual obstacles in the foreground. Nevertheless, we are often able to keep our gaze on them throughout the occlusion or even catch them on the fly in the face of the transient lack of visual motion information. This implies that the brain can fill the gaps of missing sensory information by extrapolating the object motion through the occlusion. In recent years, much experimental evidence has been accumulated that both perceptual and motor processes exploit visual motion extrapolation mechanisms. Moreover, neurophysiological and neuroimaging studies have identified brain regions potentially involved in the predictive representation of the occluded target motion. Within this framework, ocular pursuit and manual interceptive behavior have proven to be useful experimental models for investigating visual extrapolation mechanisms. Studies in these fields have pointed out that visual motion extrapolation processes depend on manifold information related to short-term memory representations of the target motion before the occlusion, as well as to longer term representations derived from previous experience with the environment. We will review recent oculomotor and manual interception literature to provide up-to-date views on the neurophysiological underpinnings of visual motion extrapolation.

Noise distraction and mental practice in closed and open motor skills

Two experiments were conducted to investigate whether mental practice can serve as an effective intervention strategy for decreasing interference effects of distracting noise at the time of executing open (badminton service return) and closed (badminton serve) skills. In Experiment 1, 44 male students participated and in Experiment 2, 36 students participated. In both experiments, students were randomly assigned to one of three groups: (a) mental practice, (b) mental practice plus noise distraction (MPD), and (c) control. All participants performed 40 trials in the acquisition phase and 20 trials in the retention phase. Absolute error and adjusted variable error were recorded to measure performance. Experiment 1 showed that for a closed skill, the MPD group served with greater accuracy than did the control group in both acquisition and retention phases. Experiment 2 showed that for the open skill, the MP group performed with less error than did the control group, but only in the acquisition trials. Results indicate that mental practice may function to block out noise disturbances and that mental practice may work differently with different skill sets.

Illusory speed is retained in memory during invisible motion

The brain can retain speed information in early visual short-term memory in an astonishingly precise manner. We investigated whether this (early) visual memory system is active during the extrapolation of occluded motion and whether it reflects speed misperception due to contrast and size. Experiments 1A and 2A showed that reducing target contrast or increasing its size led to an illusory speed underestimation. Experiments 1B, 2B, and 3 showed that this illusory phenomenon is reflected in the memory of speed during occluded motion, independent of the range of visible speeds, of the length of the visible trajectory or the invisible trajectory, and of the type of task. These results suggest that illusory speed is retained in memory during invisible motion.

Covert tracking: a combined ERP and fixational eye movement study

Attention can be directed to particular spatial locations, or to objects that appear at anticipated points in time. While most work has focused on spatial or temporal attention in isolation, we investigated covert tracking of smoothly moving objects, which requires continuous coordination of both. We tested two propositions about the neural and cognitive basis of this operation: first that covert tracking is a right hemisphere function, and second that pre-motor components of the oculomotor system are responsible for driving covert spatial attention during tracking. We simultaneously recorded event related potentials (ERPs) and eye position while participants covertly tracked dots that moved leftward or rightward at 12 or 20°/s. ERPs were sensitive to the direction of target motion. Topographic development in the leftward motion was a mirror image of the rightward motion, suggesting that both hemispheres contribute equally to covert tracking. Small shifts in eye position were also lateralized according to the direction of target motion, implying covert activation of the oculomotor system. The data addresses two outstanding questions about the nature of visuospatial tracking. First, covert tracking is reliant upon a symmetrical frontoparietal attentional system, rather than being right lateralized. Second, this same system controls both pursuit eye movements and covert tracking.

Do common systems control eye movements and motion extrapolation?

People are able to judge the current position of occluded moving objects. This operation is known as motion extrapolation. It has previously been suggested that motion extrapolation is independent of the oculomotor system. Here we revisited this question by measuring eye position while participants completed two types of motion extrapolation task. In one task, a moving visual target travelled rightwards, disappeared, then reappeared further along its trajectory. Participants discriminated correct reappearance times from incorrect (too early or too late) with a two-alternative forced-choice button press. In the second task, the target travelled rightwards behind a visible, rectangular occluder, and participants pressed a button at the time when they judged it should reappear. In both tasks, performance was significantly different under fixation as compared to free eye movement conditions. When eye movements were permitted, eye movements during occlusion were related to participants' judgements. Finally, even when participants were required to fixate, small changes in eye position around fixation (<2°) were influenced by occluded target motion. These results all indicate that overlapping systems control eye movements and judgements on motion extrapolation tasks. This has implications for understanding the mechanism underlying motion extrapolation.

The neural correlates of visuospatial perceptual and oculomotor extrapolation

The human visual system must perform complex visuospatial extrapolations (VSE) across space and time in order to extract shape and form from the retinal projection of a cluttered visual environment characterized by occluded surfaces and moving objects. Even if we exclude the temporal dimension, for instance when judging whether an extended finger is pointing towards one object or another, the mechanisms of VSE remain opaque. Here we investigated the neural correlates of VSE using functional magnetic resonance imaging in sixteen human observers while they judged the relative position of, or saccaded to, a (virtual) target defined by the extrapolated path of a pointer. Using whole brain and region of interest (ROI) analyses, we compared the brain activity evoked by these VSE tasks to similar control judgements or eye movements made to explicit (dot) targets that did not require extrapolation. The data show that activity in an occipitotemporal region that included the lateral occipital cortex (LOC) was significantly greater during VSE than during control tasks. A similar, though less pronounced, pattern was also evident in regions of the fronto-parietal cortex that included the frontal eye fields. However, none of the ROIs examined exhibited a significant interaction between target type (extrapolated/explicit) and response type (oculomotor/perceptual). These findings are consistent with a close association between visuoperceptual and oculomotor responses, and highlight a critical role for the LOC in the process of VSE.

Observer's control of the moving stimulus increases the flash-lag effect

The flash-lag effect (FLE) consists in perceiving a briefly presented stationary stimulus to lag behind an aligned moving stimulus. This study investigates the effects of actively controlling the moving stimulus. By means of a robotic arm, observers continuously moved a dot along a circular trajectory, and a flash was displayed closely at unpredictable times. In two experiments, we found that the FLE was larger when participants controlled the moving stimulus, compared to a computer-controlled condition. Two control conditions tested the possibility that the observed modulation of the FLE was due to visuo-spatial attention or dual-task factors. This study provides evidence that the motor system interacts with and possibly speeds up the processing of a moving visual stimulus when the observer controls its movement.

Motion-induced positional biases in the flash-lag configuration

When both stationary and moving objects are present in the visual field, localizing objects in space may become difficult, as shown by illusory phenomena such as the Fröhlich effect and the flash-lag effect. Despite the efforts to decipher how motion and position information are combined to form a coherent visual representation, a unitary picture is still lacking. In the flash-lag effect, a flash presented in alignment with a moving stimulus is perceived to lag behind it. We investigated whether this relative spatial localization (i.e., judging the position of the flash relative to that of the moving stimulus) is the result of a linear combination of two absolute localization mechanisms--that is, the coding of the flash position in space and the coding of the position of the moving stimulus in space. In three experiments we showed that (a) the flash is perceived to be shifted in the direction of motion; (b) the moving stimulus is perceived to be ahead of its physical position, the forward shift being larger than that of the flash; (c) the linear combination of these two shifts is quantitatively equivalent to the flash-lag effect, which was measured independently. The results are discussed in relation to perceptual and motor localization mechanisms.

Head movements modulate visual responsiveness in the absence of gaze shifts

Visuospatial attention is strongly associated with saccades. Given that gaze shifts are often accomplished by combined eye-head movements, attention may also be coupled to head movements. We showed that simply turning the head without shifting the gaze is sufficient to cause a transient unbalance in responding to a visual stimulus. Manual responses to a stimulus flashed shortly before the onset of a horizontal head movement were faster in congruent trials, when the head moved towards the stimulus, than in incongruent trials, when the head moved away from the stimulus. These effects are similar to those observed for saccades. We take this as evidence for a tight link between visuospatial attention and head movements, even when the gaze does not shift.

Blind saccades: an asynchrony between seeing and looking

Saccades may not always wait for the completion of the perceptual analysis. By taking advantage of a motion-induced illusion of position and of the spontaneous scatter of saccade latency, we showed that in normal observers, regular saccades (latency, approximately 200 ms) were accurately directed to the target, whereas at higher latencies, saccades were increasingly biased by visual motion until they reflected the perceptual illusion. We reconstructed the time course of saccadic direction coding and identified an early phase in which saccades are mostly predictive (latencies less than approximately 100 ms), followed by a phase in which saccades are guided by the target position signal (latencies approximately 100-250 ms), and a later phase associated with the buildup of mislocalization (approximately 250-450 ms). This transient dissociation between action and perception indicates that seeing and looking are based on asynchronous processes, possibly because of independent thresholds for saccades and perceptual localization. The metrics of a saccade would then reflect the evolution of cortical visual signals from a predictive state to a perceptual state, passing through an intermediate visuomotor state. If saccades occur during the visuomotor state, they escape the tricks of perception.