Visual control of action without retinal optic flow

The Role of Binocular Vision in Avoiding Virtual Obstacles While Walking

Advances in Virtual Reality technology have enabled physical walking in virtual environments. While most Virtual Reality systems render stereoscopic images to users, the implication of binocular viewing with respect to the performance of human walking in virtual environments remains largely unknown. In the present study, we conducted two walking experiments in virtual environments using a linear treadmill and a novel projected display known as the Wide Immersive Stereo Environment (WISE) to study the role of binocular viewing in virtual locomotion. The first experiment investigated the walking performance of people stepping over obstacles while the second experiment focused on a scenario on stepping over gaps. Both experiments were conducted under both stereoscopic viewing and non-stereoscopic viewing conditions. By analysing the gait parameters, we found that binocular viewing helped people to make more accurate movements to step over obstacles and gaps in virtual locomotion.

Visually directed action

When people throw or walk to targets in front of them without visual feedback, they often respond short. With feedback, responses rapidly become approximately accurate. To understand this, an experiment is performed with four stages. 1) The errors in blind walking and blind throwing are measured in a virtual environment in light and dark cue conditions. 2) Error feedback is introduced and the resulting learning measured. 3) Transfer to the other response is then measured. 4) Finally, responses to the perceived distances of the targets are measured. There is large initial under-responding. Feedback rapidly makes responses almost accurate. Throw training transfers completely to walking. Walk training produces a small effect on throwing. Under instructions to respond to perceived distances, under-responding recurs. The phenomena are well described by a model in which the relation between target distance and response distance is determined by a sequence of a perceptual, a cognitive, and a motor transform. Walk learning is primarily motor; throw learning is cognitive.

Visual control of steering in curve driving

This pair of studies investigated steering in the absence of continuous visual information. In a driving simulator, participants steered a curving path that was displayed either continuously or intermittently. Optic flow conditions were manipulated to alter the nature of the heading information with respect to the path being steered. Removing or biasing heading information had little effect on steering even during long and frequent path occlusions as long as turn rate was available. This demonstrates that participants can use intermittent views of the path to plan their steering actions and optic flow to accurately update vehicle turns with respect to that path.

Modeling the Human Visuo-Motor System to Support Remote-Control Operation

The working hypothesis in this project is that gaze interactions play a central role in structuring the joint control and guidance strategy of the human operator performing spatial tasks. Perceptual guidance and control is the idea that the visual and motor systems form a unified perceptuo-motor system where necessary information is naturally extracted by the visual system. As a consequence, the response of this system is constrained by the visual and motor mechanisms and these effects should manifest in the behavioral data. Modeling the perceptual processes of the human operator provides the foundation necessary for a systems-based approach to the design of control and display systems used by remotely operated vehicles. This paper investigates this hypothesis using flight tasks conducted with remotely controlled miniature rotorcraft, taking place in indoor settings that provide rich environments to investigate the key processes supporting spatial interactions. This work also applies to spatial control tasks in a range of application domains that include tele-operation, gaming, and virtual reality. The human-in-the-loop system combines the dynamics of the vehicle, environment, and human perception–action with the response of the overall system emerging from the interplay of perception and action. The main questions to be answered in this work are as follows: (i) what is the general control and guidance strategy of the human operator, and (ii) how is information about the vehicle and environment extracted visually by the operator. The general approach uses gaze as the primary sensory mechanism by decoding the gaze patterns of the pilot to provide information for estimation, control, and guidance. This work differs from existing research by taking what have largely been conceptual ideas on action–perception and structuring them to be implemented for a real-world problem. The paper proposes a system model that captures the human pilot’s perception–action loop; the loop that delineates the main components of the pilot’s perceptuo-motor system, including estimation of the vehicle state and task elements based on operator gaze patterns, trajectory planning, and tracking control. The identified human visuo-motor model is then exploited to demonstrate how the perceptual and control functions system can be augmented to reduce the operator workload.

For Efficient Navigational Search, Humans Require Full Physical Movement, but Not a Rich Visual Scene

During navigation, humans combine visual information from their surroundings with body-based information from the translational and rotational components of their movement. Theories of navigation focus on the role of visual and rotational body-based information, even though experimental evidence shows they are not sufficient for complex spatial tasks. To investigate the contribution of all three sources of information, we asked participants to search a computer-generated virtual room for targets. Participants were provided with only visual information or with visual information supplemented with body-based information for all movement (walk group) or rotational movement (rotate group). The walk group performed the task with near-perfect efficiency, irrespective of whether a rich or impoverished visual scene was provided. The visual-only and rotate groups were significantly less efficient and frequently searched parts of the room at least twice. These results suggest that full physical movement plays a critical role in navigational search, but only moderate visual detail is required.

The role of binocular vision in walking

Despite the extensive investigation of binocular and stereoscopic vision, relatively little is known about its importance in natural visually guided behavior. In this paper, we explored the role of binocular vision when walking over and around obstacles. We monitored eye position during the task as an indicator of the difference between monocular and binocular performances. We found that binocular vision clearly facilitates walking performance. Walkers were slowed by about 10% in monocular vision and raised their foot higher when stepping over obstacles. Although the location and sequence of the fixations did not change in monocular vision, the timing of the fixations relative to the actions was different. Subjects spent proportionately more time fixating the obstacles and fixated longer while guiding foot placement near an obstacle. The data are consistent with greater uncertainty in monocular vision, leading to a greater reliance on feedback in the control of the movements.

Multiple systems of spatial memory and action

Recent findings from spatial cognition and cognitive neuroscience suggest that different types of mental representations could mediate the off-line retrieval of spatial relations from memory and the on-line guidance of motor actions in space. As a result, a number of models proposing multiple systems of spatial memory have been recently formulated. In the present article we review these models and we evaluate their postulates based on available experimental evidence. Furthermore, we discuss how a multiple-system model can apply to situations in which people reason about their immediate surroundings or non-immediate environments by incorporating a model of sensorimotor facilitation/interference. This model draws heavily on previous accounts of sensorimotor interference and takes into account findings from the stimulus-response compatibility literature.