Enhanced Optic Flow Speed Discrimination While Walking: Contextual Tuning of Visual Coding

Walking humans and running mice: perception and neural encoding of optic flow during self-motion

Locomotion produces full-field optic flow that often dominates the visual motion inputs to an observer. The perception of optic flow is in turn important for animals to guide their heading and interact with moving objects. Understanding how locomotion influences optic flow processing and perception is therefore essential to understand how animals successfully interact with their environment. Here, we review research investigating how perception and neural encoding of optic flow are altered during self-motion, focusing on locomotion. Self-motion has been found to influence estimation and sensitivity for optic flow speed and direction. Nonvisual self-motion signals also increase compensation for self-driven optic flow when parsing the visual motion of moving objects. The integration of visual and nonvisual self-motion signals largely follows principles of Bayesian inference and can improve the precision and accuracy of self-motion perception. The calibration of visual and nonvisual self-motion signals is dynamic, reflecting the changing visuomotor contingencies across different environmental contexts. Throughout this review, we consider experimental research using humans, non-human primates and mice. We highlight experimental challenges and opportunities afforded by each of these species and draw parallels between experimental findings. These findings reveal a profound influence of locomotion on optic flow processing and perception across species. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Optic Flow Speed and Retinal Stimulation Influence Microsaccades

Microsaccades are linked with extraretinal mechanisms that significantly alter spatial perception before the onset of eye movements. We sought to investigate whether microsaccadic activity is modulated by the speed of radial optic flow stimuli. Experiments were performed in the dark on 19 subjects who stood in front of a screen covering 135 × 107° of the visual field. Subjects were instructed to fixate on a central fixation point while optic flow stimuli were presented in full field, in the foveal, and in the peripheral visual field at different dot speeds (8, 11, 14, 17, and 20°/s). Fixation in the dark was used as a control stimulus. For almost all tested speeds, the stimulation of the peripheral retina evoked the highest microsaccade rate. We also found combined effects of optic flow speed and the stimulated retinal region (foveal, peripheral, and full field) for microsaccade latency. These results show that optic flow speed modulates microsaccadic activity when presented in specific retinal portions, suggesting that eye movement generation is strictly dependent on the stimulated retinal regions.

A Dynamic Efficient Sensory Encoding Approach to Adaptive Tuning in Neural Models of Optic Flow Processing

This paper introduces a self-tuning mechanism for capturing rapid adaptation to changing visual stimuli by a population of neurons. Building upon the principles of efficient sensory encoding, we show how neural tuning curve parameters can be continually updated to optimally encode a time-varying distribution of recently detected stimulus values. We implemented this mechanism in a neural model that produces human-like estimates of self-motion direction (i.e., heading) based on optic flow. The parameters of speed-sensitive units were dynamically tuned in accordance with efficient sensory encoding such that the network remained sensitive as the distribution of optic flow speeds varied. In two simulation experiments, we found that model performance with dynamic tuning yielded more accurate, shorter latency heading estimates compared to the model with static tuning. We conclude that dynamic efficient sensory encoding offers a plausible approach for capturing adaptation to varying visual environments in biological visual systems and neural models alike.

Perceptual Awareness of Optic Flows Paced Optimally and Non-optimally to Walking Speed

Previous research suggests that visual processing depends strongly on locomotor activity and is tuned to optic flows consistent with self-motion speed. Here, we used a binocular rivalry paradigm to investigate whether perceptual access to optic flows depends on their optimality in relation to walking velocity. Participants walked at two different speeds on a treadmill while viewing discrepant visualizations of a virtual tunnel in each eye. We hypothesized that visualizations paced appropriately to the walking speeds will be perceived longer than non optimal (too fast/slow) ones. The presented optic flow speeds were predetermined individually in a task based on matching visual speed to both walking velocities. In addition, perceptual preference for optimal optic flows was expected to increase with proprioceptive ability to detect threshold-level changes in walking speed. Whereas faster (more familiar) optic flows showed enhanced access to awareness during faster compared with slower walking conditions, for slower visual flows, only a nonsignificant tendency for the analogous effect was observed. These effects were not dependent on individual proprioceptive sensitivity. Our findings concur with the emerging view that the velocity of one's locomotion is used to calibrate visual perception of self-motion and extend the scope of reported action effects on visual awareness.

Forward optic flow is prioritised in visual awareness independently of walking direction

When two different images are presented separately to each eye, one experiences smooth transitions between them-a phenomenon called binocular rivalry. Previous studies have shown that exposure to signals from other senses can enhance the access of stimulation-congruent images to conscious perception. However, despite our ability to infer perceptual consequences from bodily movements, evidence that action can have an analogous influence on visual awareness is scarce and mainly limited to hand movements. Here, we investigated whether one's direction of locomotion affects perceptual access to optic flow patterns during binocular rivalry. Participants walked forwards and backwards on a treadmill while viewing highly-realistic visualisations of self-motion in a virtual environment. We hypothesised that visualisations congruent with walking direction would predominate in visual awareness over incongruent ones, and that this effect would increase with the precision of one's active proprioception. These predictions were not confirmed: optic flow consistent with forward locomotion was prioritised in visual awareness independently of walking direction and proprioceptive abilities. Our findings suggest the limited role of kinaesthetic-proprioceptive information in disambiguating visually perceived direction of self-motion and indicate that vision might be tuned to the (expanding) optic flow patterns prevalent in everyday life.

Influence of the Size of the Field of View on Visual Perception While Running in a Treadmill-Mediated Virtual Environment

We investigated how the size of the horizontal field of view (FoV) affects visual speed perception with individuals running on a treadmill. Twelve moderately trained to trained participants ran on a treadmill at two different speeds (8 and 12 km/h) in front of a moving virtual scene. Different masks were used to manipulate the visible visual field, masking either the central or the peripheral area of the virtual scene or showing the full visual field. We asked participants to match the visual speed of the scene to their actual running speed. For each trial, participants indicated whether the scene was moving faster or slower than they were running. Visual speed was adjusted according to the responses using a staircase method until the Point of Subjective Equality was reached, that is until visual and running speed were perceived as matching. For both speeds and all FoV conditions, participants underestimated visual speed relative to the actual running speed. However, this underestimation was significant only when the peripheral FoV was masked. These results confirm that the size of the FoV should absolutely be taken into account for the design of treadmill-mediated virtual environments (VEs).

Distance perception during self-movement

The perception of distance in open fields was widely studied with static observers. However, it is a fact that we and the world around us are in continuous relative movement, and that our perceptual experience is shaped by the complex interactions between our senses and the perception of our self-motion. This poses interesting questions about how our nervous system integrates this multisensory information to resolve specific tasks of our daily life, for example, distance estimation. This study provides new evidence about how visual and motor self-motion information affects our perception of distance and a hypothesis about how these two sources of information can be integrated to calibrate the estimation of distance. This model accounts for the biases found when visual and proprioceptive information is inconsistent.

Quantitative Predictions Orchestrate Visual Signaling in Drosophila

Vision influences behavior, but ongoing behavior also modulates vision in animals ranging from insects to primates. The function and biophysical mechanisms of most such modulations remain unresolved. Here, we combine behavioral genetics, electrophysiology, and high-speed videography to advance a function for behavioral modulations of visual processing in Drosophila. We argue that a set of motion-sensitive visual neurons regulate gaze-stabilizing head movements. We describe how, during flight turns, Drosophila perform a set of head movements that require silencing their gaze-stability reflexes along the primary rotation axis of the turn. Consistent with this behavioral requirement, we find pervasive motor-related inputs to the visual neurons, which quantitatively silence their predicted visual responses to rotations around the relevant axis while preserving sensitivity around other axes. This work proposes a function for a behavioral modulation of visual processing and illustrates how the brain can remove one sensory signal from a circuit carrying multiple related signals.

Evidence against an ecological explanation of the jitter advantage for vection

Visual-vestibular conflicts have been traditionally used to explain both perceptions of self-motion and experiences of motion sickness. However, sensory conflict theories have been challenged by findings that adding simulated viewpoint jitter to inducing displays enhances (rather than reduces or destroys) visual illusions of self-motion experienced by stationary observers. One possible explanation of this jitter advantage for vection is that jittering optic flows are more ecological than smooth displays. Despite the intuitive appeal of this idea, it has proven difficult to test. Here we compared subjective experiences generated by jittering and smooth radial flows when observers were exposed to either visual-only or multisensory self-motion stimulations. The display jitter (if present) was generated in real-time by updating the virtual computer-graphics camera position to match the observer’s tracked head motions when treadmill walking or walking in place, or was a playback of these head motions when standing still. As expected, the (more naturalistic) treadmill walking and the (less naturalistic) walking in place were found to generate very different physical head jitters. However, contrary to the ecological account of the phenomenon, playbacks of treadmill walking and walking in place display jitter both enhanced visually induced illusions of self-motion to a similar degree (compared to smooth displays).

Establishing the range of perceptually natural visual walking speeds for virtual walking-in-place locomotion

Walking-In-Place (WIP) techniques make it possible to facilitate relatively natural locomotion within immersive virtual environments that are larger than the physical interaction space. However, in order to facilitate natural walking experiences one needs to know how to map steps in place to virtual motion. This paper describes two within-subjects studies performed with the intention of establishing the range of perceptually natural walking speeds for WIP locomotion. In both studies, subjects performed a series of virtual walks while exposed to visual gains (optic flow multipliers) ranging from 1.0 to 3.0. Thus, the slowest speed was equal to an estimate of the subjects normal walking speed, while the highest speed was three times greater. The perceived naturalness of the visual speed was assessed using self-reports. The first study compared four different types of movement, namely, no leg movement, walking on a treadmill, and two forms of gestural input for WIP locomotion. The results suggest that WIP locomotion is accompanied by a perceptual distortion of the speed of optic flow. The second study was performed using a 4×2 factorial design and compared four different display field-of-views (FOVs) and two types of movement, walking on a treadmill and WIP locomotion. The results revealed significant main effects of both movement type and field of view, but no significant interaction between the two variables. Particularly, they suggest that the size of the display FOV is inversely proportional to the degree of underestimation of the virtual speeds for both treadmill-mediated virtual walking and WIP locomotion. Combined, the results constitute a first attempt at establishing a set of guidelines specifying what virtual walking speeds WIP gestures should produce in order to facilitate a natural walking experience.

The role of perceived speed in vection: does perceived speed modulate the jitter and oscillation advantages?

Illusory self-motion (‘vection’) in depth is strongly enhanced when horizontal/vertical simulated viewpoint oscillation is added to optic flow inducing displays; a similar effect is found for simulated viewpoint jitter. The underlying cause of these oscillation and jitter advantages for vection is still unknown. Here we investigate the possibility that perceived speed of motion in depth (MID) plays a role. First, in a 2AFC procedure, we obtained MID speed PSEs for briefly presented (vertically oscillating and smooth) radial flow displays. Then we examined the strength, duration and onset latency of vection induced by oscillating and smooth radial flow displays matched either for simulated or perceived MID speed. The oscillation advantage was eliminated when displays were matched for perceived MID speed. However, when we tested the jitter advantage in the same manner, jittering displays were found to produce greater vection in depth than speed-matched controls. In summary, jitter and oscillation advantages were the same across experiments, but slower MID speed was required to match jittering than oscillating stimuli. Thus, to the extent that vection is driven by perceived speed of MID, this effect is greater for oscillating than for jittering stimuli, which suggests that the two effects may arise from separate mechanisms.

Vection in depth during treadmill walking

Vection has typically been induced in stationary observers (ie conditions providing visual-only information about self-motion). Two recent studies have examined vection during active treadmill walking--one reported that treadmill walking in the same direction as the visually simulated self-motion impaired vection (Onimaru et al, 2010 Journal of Vision 10(7):860), the other reported that it enhanced vection (Seno et al, 2011 Perception 40 747-750; Seno et al, 2011 Attention, Perception, & Psychophysics 73 1467-1476). Our study expands on these earlier investigations of vection during observer active movement. In experiment 1 we presented radially expanding optic flow and compared the vection produced in stationary observers with that produced during walking forward on a treadmill at a 'matched' speed. Experiment 2 compared the vection induced by forward treadmill walking while viewing expanding or contracting optic flow with that induced by viewing playbacks of these same displays while stationary. In both experiments subjects' tracked head movements were either incorporated into the self-motion displays (as simulated viewpoint jitter) or simply ignored. We found that treadmill walking always reduced vection (compared with stationary viewing conditions) and that simulated viewpoint jitter always increased vection (compared with constant velocity displays). These findings suggest that while consistent visual-vestibular information about self-acceleration increases vection, biomechanical self-motion information reduces this experience (irrespective of whether it is consistent or not with the visual input).

Direct perception of action-scaled affordances: the shrinking gap problem

The aim of this study was to investigate the perception of possibilities for action (i.e., affordances) that depend on one's movement capabilities, and more specifically, the passability of a shrinking gap between converging obstacles. We introduce a new optical invariant that specifies in intrinsic units the minimum locomotor speed needed to safely pass through a shrinking gap. Detecting this information during self-motion requires recovering the component of the obstacles' local optical expansion attributable to obstacle motion, independent of self-motion. In principle, recovering the obstacle motion component could involve either visual or non-visual self-motion information. We investigated the visual and non-visual contributions in two experiments in which subjects walked through a virtual environment and made judgments about whether it was possible to pass through a shrinking gap. On a small percentage of trials, visual and non-visual self-motion information were independently manipulated by varying the speed with which subjects moved through the virtual environment. Comparisons of judgments on such catch trials with judgments on normal trials revealed both visual and non-visual contributions to the detection of information about minimum walking speed.

Treadmill experience alters treadmill effects on perceived visual motion

Information on ongoing body movements can affect the perception of ambiguous visual motion. Previous studies on “treadmill capture” have shown that treadmill walking biases the perception of ambiguous apparent motion in backward direction in accordance with the optic flow during normal walking, and that long-term treadmill experience changes the effect of treadmill capture. To understand the underlying mechanisms for these phenomena, we conducted Experiment 1 with non-treadmill runners and Experiment 2 with treadmill runners. The participants judged the motion direction of the apparent motion stimuli of horizontal gratings in front of their feet under three conditions: walking on a treadmill, standing on a treadmill, and standing on the floor. The non-treadmill runners showed the presence of downward bias only under the walking condition, indicating that ongoing treadmill walking but not the awareness of being on a treadmill biased the visual directional discrimination. In contrast, the treadmill runners showed no downward bias under any of the conditions, indicating that neither ongoing activity nor the awareness of spatial context produced perception bias. This suggests that the long-term repetitive experience of treadmill walking without optic flow induced the formation of a treadmill-specific locomotor-visual linkage to perceive the complex relationship between self and the environment.

Humans do not have direct access to retinal flow during walking

Perceived visual speed has been reported to be reduced during walking. This reduction has been attributed to a partial subtraction of walking speed from visual speed (F. H. Durgin & K. Gigone, 2007; F. H. Durgin, K. Gigone, & R. Scott, 2005). We tested whether observers still have access to the retinal flow before subtraction takes place. Observers performed a 2IFC visual speed discrimination task while walking on a treadmill. In one condition, walking speed was identical in the two intervals, while in a second condition walking speed differed between intervals. If observers have access to the retinal flow before subtraction, any changes in walking speed across intervals should not affect their ability to discriminate retinal flow speed. Contrary to this "direct access hypothesis," we found that observers were worse at discrimination when walking speed differed between intervals. The results therefore suggest that observers do not have access to retinal flow before subtraction. We also found that the amount of subtraction depended on the visual speed presented, suggesting that the interaction between the processing of visual input and of self-motion is more complex than previously proposed.

Palm boards are not action measures: an alternative to the two-systems theory of geographical slant perception

Whereas most reports of the perception of outdoor hills demonstrate dramatic overestimation, estimates made by adjusting a palm board are much closer to the true hill orientation. We test the dominant hypothesis that palm board accuracy is related to the need for motor action to be accurately guided and conclude instead that the perceptual experience of palm-board orientation is biased and variable due to poorly calibrated proprioception of wrist flexion. Experiments 1 and 3 show that wrist-flexion palm boards grossly underestimate the orientations of near, reachable surfaces whereas gesturing with a free hand is fairly accurate. Experiment 2 shows that palm board estimates are much lower than free hand estimates for an outdoor hill as well. Experiments 4 shows that wrist flexion is biased and noisy compared to elbow flexion, while Experiment 5 shows that small changes in palm board height produce large changes in palm board estimates. Together, these studies suggest that palm boards are biased and insensitive measures. The existing literature arguing that there are two systems in the perception of geographical slant is re-evaluated, and a new theoretical framework is proposed in which a single exaggerated representation of ground-surface orientation guides both action and perception.

When Walking Makes Perception Better

When we move, the visual world moves toward us. That is, self-motion normally produces visual signals (flow) that tell us about our own motion. But these signals are distorted by our motion: Visual flow actually appears slower while we are moving than it does when we are stationary and our surroundings move past us. Although for many years these kinds of distortions have been interpreted as a suppression of flow to promote the perception of a stable world, current research has shown that these shifts in perceived visual speed may have an important function in measuring our own self-motion. Specifically, by slowing down the apparent rate of visual flow during self-motion, our visual system is able to perceive differences between actual and expected flow more precisely. This is useful in the control of action.