Differential effects of shared attention on perception of heading and 3-D object motion

A Bayesian inference model can predict the effects of attention on the serial dependence in heading estimation from optic flow

It has been demonstrated that observers can accurately estimate their self-motion direction (i.e., heading) from optic flow, which can be affected by attention. However, it remains unclear how attention affects the serial dependence in the estimation. In the current study, participants conducted two experiments. The results showed that the estimation accuracy decreased when attentional resources allocated to the heading estimation task were reduced. Additionally, the estimates of currently presented headings were biased toward the headings of previously seen headings, showing serial dependence. Especially, this effect decreased (increased) when the attentional resources allocated to the previously (currently) seen headings were reduced. Furthermore, importantly, we developed a Bayesian inference model, which incorporated attention-modulated likelihoods and qualitatively predicted changes in the estimation accuracy and serial dependence. In summary, the current study shows that attention affects the serial dependence in heading estimation from optic flow and reveals the Bayesian computational mechanism behind the heading estimation.

Attention affects the perception of self-motion direction from optic flow

Many studies have demonstrated that attention affects the perception of many visual features. However, previous studies show conflicting results regarding the effect of attention on the perception of self-motion direction (i.e., heading) from optic flow. To address this question, we conducted three behavioral experiments and found that estimation accuracies of large headings (>14°) decreased with attention load, discrimination thresholds of these headings increased with attention load, and heading estimates were systematically compressed toward the focus of attention. Therefore, the current study demonstrated that attention affected heading perception from optic flow, showing that the perception is both information-driven and cognitive.

Primate extrastriate cortical area MST: a gateway between sensation and cognition

Primate visual cortex consists of dozens of distinct brain areas, each providing a highly specialized component to the sophisticated task of encoding the incoming sensory information and creating a representation of our visual environment that underlies our perception and action. One such area is the medial superior temporal cortex (MST), a motion-sensitive, direction-selective part of the primate visual cortex. It receives most of its input from the middle temporal (MT) area, but MST cells have larger receptive fields and respond to more complex motion patterns. The finding that MST cells are tuned for optic flow patterns has led to the suggestion that the area plays an important role in the perception of self-motion. This hypothesis has received further support from studies showing that some MST cells also respond selectively to vestibular cues. Furthermore, the area is part of a network that controls the planning and execution of smooth pursuit eye movements and its activity is modulated by cognitive factors, such as attention and working memory. This review of more than 90 studies focuses on providing clarity of the heterogeneous findings on MST in the macaque cortex and its putative homolog in the human cortex. From this analysis of the unique anatomical and functional position in the hierarchy of areas and processing steps in primate visual cortex, MST emerges as a gateway between perception, cognition, and action planning. Given this pivotal role, this area represents an ideal model system for the transition from sensation to cognition.

Perceived shift of the centres of contracting and expanding optic flow fields: Different biases in the lower-right and upper-right visual quadrants

We studied differences in localizing the centres of flow in radially expanding and contracting patterns in different regions of the visual field. Our results suggest that the perceived centre of a peripherally viewed expanding pattern is shifted towards the fovea relative to that of a contracting pattern, but only in the lower right and upper right visual quadrants and when a single speed gradient with appropriate overall speeds of the trajectories of the moving dots was used. The biases were not systematically related to differences of sensitivity to optic flow in different quadrants. Further experiments demonstrated that the biases were likely due to a combination of two effects: an advantage of global processing in favor of the lower visual hemifield and a hemispheric asymmetry in attentional allocation in favor of motion-induced spatial displacement in the right visual hemifield. The bias in the lower right visual quadrant was speed gradient-sensitive and could be reduced to a non-significant level with the usage of multiple speed gradients, possibly due to a special role of the lower visual hemifield in extracting global information from the multiple speed gradients. A holistic processing on multiple speed gradients, rather than a predominant processing on a single speed gradient, was likely adopted. In contrast, the perceived bias in the upper right visual quadrant was overall speed-sensitive and could be reduced to a non-significant level with the reduction of the overall speeds of the trajectories. The implications of these results for understanding motion-induced spatial illusions are discussed.

Dissociation of Self-Motion and Object Motion by Linear Population Decoding That Approximates Marginalization

We use visual image motion to judge the movement of objects, as well as our own movements through the environment. Generally, image motion components caused by object motion and self-motion are confounded in the retinal image. Thus, to estimate heading, the brain would ideally marginalize out the effects of object motion (or vice versa), but little is known about how this is accomplished neurally. Behavioral studies suggest that vestibular signals play a role in dissociating object motion and self-motion, and recent computational work suggests that a linear decoder can approximate marginalization by taking advantage of diverse multisensory representations. By measuring responses of MSTd neurons in two male rhesus monkeys and by applying a recently-developed method to approximate marginalization by linear population decoding, we tested the hypothesis that vestibular signals help to dissociate self-motion and object motion. We show that vestibular signals stabilize tuning for heading in neurons with congruent visual and vestibular heading preferences, whereas they stabilize tuning for object motion in neurons with discrepant preferences. Thus, vestibular signals enhance the separability of joint tuning for object motion and self-motion. We further show that a linear decoder, designed to approximate marginalization, allows the population to represent either self-motion or object motion with good accuracy. Decoder weights are broadly consistent with a readout strategy, suggested by recent computational work, in which responses are decoded according to the vestibular preferences of multisensory neurons. These results demonstrate, at both single neuron and population levels, that vestibular signals help to dissociate self-motion and object motion.SIGNIFICANCE STATEMENT The brain often needs to estimate one property of a changing environment while ignoring others. This can be difficult because multiple properties of the environment may be confounded in sensory signals. The brain can solve this problem by marginalizing over irrelevant properties to estimate the property-of-interest. We explore this problem in the context of self-motion and object motion, which are inherently confounded in the retinal image. We examine how diversity in a population of multisensory neurons may be exploited to decode self-motion and object motion from the population activity of neurons in macaque area MSTd.

Static Scene Analysis for the Perception of Heading

In the current study, we explored observers' use of two distinct analyses for determining their direction of motion, or heading: a scene-based analysis and a motion-based analysis. In two experiments, subjects viewed sequentially presented, paired digitized images of real-world scenes and judged the direction of heading; the pairs were presented with various interstimulus intervals (ISIs). In Experiment 1, subjects could determine heading when the two frames were separated with a 1,000-ms ISI, long enough to eliminate apparent motion. In Experiment 2, subjects performed two tasks, a path-of-motion task and a memory-load task, under three different ISIs, 50 ms, 500 ms, and 1,000 ms. Heading accuracy decreased with an increase in ISI. Increasing memory load influenced heading judgments only for the longer ISI when motion-based information was not available. These results are consistent with the hypothesis that the scene-based analysis has a coarse spatial representation, is a sustained temporal process, and is capacity limited, whereas the motion-based analysis has a fine spatial resolution, is a transient temporal process, and is capacity unlimited.

Multisensory Integration of Visual and Vestibular Signals Improves Heading Discrimination in the Presence of a Moving Object

Humans and animals are fairly accurate in judging their direction of self-motion (i.e., heading) from optic flow when moving through a stationary environment. However, an object moving independently in the world alters the optic flow field and may bias heading perception if the visual system cannot dissociate object motion from self-motion. We investigated whether adding vestibular self-motion signals to optic flow enhances the accuracy of heading judgments in the presence of a moving object. Macaque monkeys were trained to report their heading (leftward or rightward relative to straight-forward) when self-motion was specified by vestibular, visual, or combined visual-vestibular signals, while viewing a display in which an object moved independently in the (virtual) world. The moving object induced significant biases in perceived heading when self-motion was signaled by either visual or vestibular cues alone. However, this bias was greatly reduced when visual and vestibular cues together signaled self-motion. In addition, multisensory heading discrimination thresholds measured in the presence of a moving object were largely consistent with the predictions of an optimal cue integration strategy. These findings demonstrate that multisensory cues facilitate the perceptual dissociation of self-motion and object motion, consistent with computational work that suggests that an appropriate decoding of multisensory visual-vestibular neurons can estimate heading while discounting the effects of object motion.

SIGNIFICANCE STATEMENT

Objects that move independently in the world alter the optic flow field and can induce errors in perceiving the direction of self-motion (heading). We show that adding vestibular (inertial) self-motion signals to optic flow almost completely eliminates the errors in perceived heading induced by an independently moving object. Furthermore, this increased accuracy occurs without a substantial loss in the precision. Our results thus demonstrate that vestibular signals play a critical role in dissociating self-motion from object motion.

Human cortical and behavioral sensitivity to patterns of complex motion at eccentricity

Complex patterns of image motion (contracting, expanding, rotating, and spiraling fields) are important in the coordination of visually guided behaviors. Whereas specialized detectors in monkey visual cortex show selectivity for particular patterns of complex motion, their representation in human visual cortex remains unclear. In the present study, functional magnetic resonance imaging (fMRI) was used to investigate the sensitivity of functionally defined regions of human visual cortex to parametrically modulated complex motion trajectories, coupled with complementary psychophysical testing. A unique stimulus design made it possible to disambiguate the neural responses and psychophysical sensitivity to complex motions per se from the distribution of local motions relative to the fovea, which are known to enhance cortical activity when presented radial to fixation. This involved presenting several small, separate motion fields in the periphery in a manner that distinguished them from global optic flow patterns. The patterns were morphed through complex motion space in a systematic time-locked fashion when presented in the scanner. Anisotropies were observed in the fMRI signal, marked by an enhanced response to expanding vs. contracting fields, even in early visual cortex. Anisotropies in the psychophysical sensitivity measures followed a similar pattern that was correlated with activity in areas hV4, V5/MT, and MST. This represents the first systematic examination of complex motion perception at both a behavioral and neural level in human observers. The characteristic processing anisotropy revealed in both data sets can inform models of complex motion processing, particularly with respect to computations performed in early visual cortex.

Approaching objects cause confusion in patients with Alzheimer's disease regarding their direction of self-movement

Navigation requires real-time heading estimation based-on self-movement cues from optic flow and object motion. We presented a simulated heading discrimination task to young, middle-aged and older adult, normal, control subjects and to patients with mild cognitive impairment or Alzheimer's disease. Age-related decline and neurodegenerative disease effects were evident on a battery of neuropsychological and visual motion psychophysical measures. All subject groups made more accurate heading judgements when using optic flow patterns than when using simulated movement past earth-fixed objects. When both optic flow and congruent object were presented together, heading judgements showed intermediate accuracy. In separate trials, we combined optic flow with non-congruent object motion, simulating an independently moving object. In the case of non-congruent objects, almost all of our subjects shifted their perceived self-movement to heading in the direction of the moving object. However, patients with Alzheimer's disease uniquely indicated that perceived self-movement was straight-ahead, in the direction of visual fixation. The tendency to be confused by objects that appear to move independently in the simulated visual scene corresponded to the difficulty patients with Alzheimer's disease encountered in real-world navigation through the hospital lobby (R(2) = 0.87). This was not the case in older normal controls (R(2) = 0.09). We conclude that perceptual factors limit safe, autonomous navigation in early Alzheimer's disease. In particular, the presence of independently moving objects in naturalistic environments limits the capacity of patients with Alzheimer's disease to judge their heading of self-movement.

Inattentional blindness with the same scene at different scales

People with severely restricted peripheral visual fields have difficulty walking confidently and safely in the physical environment. Augmented vision devices that we are developing for low-vision rehabilitation implement vision multiplexing, providing two views of the same scene at two different scales (sizes), with a cartooned minified wide view overlaying a natural see-through view. Inattentional blindness may partially limit the utility of these devices as low-vision aids. Inattentional blindness, the apparent inability to notice significant but unexpected events in an unattended scene when attention is fixed on another scene, has classically been demonstrated by overlaying two unrelated game scenes, with unexpected events occurring in one scene while attention is maintained on the other scene by a distractor task. We hypothesized that context like that provided by the related wide view in our devices might mitigate inattentional blindness in a study with two simultaneous views of the same scene shown at different scales. It did not, and unexpected event detection rates were remarkably consistent with our and other mixed-scene studies. Still, detecting about half of the unexpected events bodes well for our use of vision aids that employ vision multiplexing. Without the aids, is it likely that many more events would be missed.

Receptive field dynamics underlying MST neuronal optic flow selectivity

Optic flow informs moving observers about their heading direction. Neurons in monkey medial superior temporal (MST) cortex show heading selective responses to optic flow and planar direction selective responses to patches of local motion. We recorded MST neuronal responses to a 90 x 90 degrees optic flow display and to a 3 x 3 array of local motion patches covering the same area. Our goal was to test the hypothesis that the optic flow responses reflect the sum of the local motion responses. The local motion responses of each neuron were modeled as mixtures of Gaussians, combining the effects of two Gaussian response functions derived using a genetic algorithm, and then used to predict that neuron's optic flow responses. Some neurons showed good correspondence between local motion models and optic flow responses, others showed substantial differences. We used the genetic algorithm to modulate the relative strength of each local motion segment's responses to accommodate interactions between segments that might modulate their relative efficacy during co-activation by global patterns of optic flow. These gain modulated models showed uniformly better fits to the optic flow responses, suggesting that coactivation of receptive field segments alters neuronal response properties. We tested this hypothesis by simultaneously presenting local motion stimuli at two different sites. These two-segment stimuli revealed that interactions between response segments have direction and location specific effects that can account for aspects of optic flow selectivity. We conclude that MST's optic flow selectivity reflects dynamic interactions between spatially distributed local planar motion response mechanisms.

Optic-flow and egocentric-direction strategies in walking: Central vs peripheral visual field

The impact of a central or peripheral visual field loss on the vision strategy used to guide walking was determined by measuring walking paths of visually impaired participants. An immersive virtual environment was used to dissociate the expected paths of the optic-flow and egocentric-direction strategies by offsetting the walker's point of view from the actual direction of walking. Environments consisted of a goal within a forest, the goal alone, or the forest alone following a brief presentation of the goal. The first two environments allowed an evaluation of the visual information used in a goal-directed task whereas the third environment investigated the information used in a memory-guided task. Participants had either a central (CFL) or peripheral visual field loss (PFL) or were fully sighted (FS). Results showed that, for the goal-directed task, the CFL group was less influenced by optic flow than was an age-matched FS group. Optic flow decreased heading error by only 1.3 degrees (16%) in the CFL group compared to 3.6 degrees (42%) in the FS group. The PFL group showed an optic-flow influence (2.4 degrees or 26%) comparable to an older, age-matched FS group (2.9 degrees or 31%). For the memory-guided task, all but the PFL group had heading errors comparable to those obtained in the goal-alone scene, demonstrating the ability to use an egocentric-direction strategy with a stored representation of either the goal's position or an offset relative to a landmark instead of a visible goal. The paths of the PFL group veered significantly from the predicted paths of both the optic-flow and egocentric-direction strategies. The findings of this study suggest that central vision is important for using optic flow to guide walking, whereas peripheral vision is important for establishing and/or updating an accurate representation of spatial structure for navigation.

Perceiving curvilinear heading in the presence of moving objects

Four experiments were directed at understanding the influence of multiple moving objects on curvilinear (i.e., circular and elliptical) heading perception. Displays simulated observer movement over a ground plane in the presence of moving objects depicted as transparent, opaque, or black cubes. Objects either moved parallel to or intersected the observer's path and either retreated from or approached the moving observer. Heading judgments were accurate and consistent across all conditions. The significance of these results for computational models of heading perception and for information in the global optic flow field about observer and object motion is discussed.

Human brain regions involved in heading estimation

Observer motion in a stationary visual environment results in an optic flow pattern on the retina, which in simple situations can be used to determine the direction of self motion or heading. The present study, using positron emission tomography (PET) and functional magnetic resonance imaging (fMRI), investigated the human cerebral activation pattern, elicited when subjects viewing a ground plane optic flow pattern actively judged heading. Several successive experiments controlled for visual input, visuospatial attention, and motor response effects. Results indicate that the network specifically involved in heading consists of only two motion sensitive areas: human MT/V5+, including an inferior satellite, and dorsal intraparietal sulcus area (DIPSM/L), predominantly in the right hemisphere, plus a dorsal premotor region bilaterally. These results suggest possible homologies with the dorsal part of the medial superior temporal area and area 7a in the monkey.

Visual search asymmetries in motion and optic flow fields

In visual search, items defined by a unique feature are found easily and efficiently. Search for a moving target among stationary distractors is one such efficient search. Search for a stationary target among moving distractors is markedly more difficult. In the experiments reported here, we confirm this finding and further show that searches for a stationary target within a structured flow field are more efficient than searches for stationary targets among distractors moving in random directions. The structured motion fields tested included uniform direction of motion, a radial flow field simulating observer forward motion, and a deformation flow field inconsistent with observer motion. The results using optic flow stimuli were not significantly different from the results obtained with other structured fields of distractors. The results suggest that the local properties of the flow fields rather than global optic flow properties are important for determining the efficiency of search for a stationary target.

Heading judgement from second-order motion

We examined human heading judgement from second-order motion which was generated by random-dots with the contrast polarity determined randomly on each frame. It was found that human observers can judge heading fairly accurately from second-order motion when pure translation is simulated or when self-motion toward a ground plane with gaze rotation is simulated but they cannot when self-motion toward cloud-like random dots with gaze rotations is simulated. It is suggested that the human visual system cannot decompose the flow fields into rotational and translational components by using second-order motion information alone, but it can do in some ways from the flow field of the ground plane.

Heading perception and the allocation of attention

A central theme in previous studies of heading judgements has been whether the retinal flow field can be decomposed to recover the translation component of locomotion when flow also contains the effects of gaze rotation. We explored not just the effect of moving gaze, but also moving attention away from the locomotor path by presenting the case of fixating a road sign and completing different attentional tasks during locomotion. Heading errors increased significantly with attentional load, in the absence of extra-retinal gaze information. When we introduced extra-retinal gaze information with the same tasks this resulted in a significant improvement in heading judgements. These results lead us to question whether the decomposition argument translates to real-world judgements of locomotor heading. If observers need to closely attend to roadside information it seems that decomposition is ineffective, whereas if they have the latitude to alternate gaze it is unnecessary.

Attentional modulation of motion-in-depth processing

I used a novel effect of adaptation to a moving stimulus to investigate the role of attention in the processing of motion-in-depth (MID). Following adaptation to expansion, the time required to detect an expanding target in the same location as the adaptation target was significantly longer than following viewing of a constant-sized target. Conversely, following adaptation to contraction, detection times (DTs) for an expanding target were significantly shorter than for a constant-sized adaptation target. Changes in DTs following adaptation to contraction were substantially larger than those following adaptation to expansion. Attentional modulation of MID processing was examined by adding an alphanumeric discrimination task that distracted the observer's attention away from the location of the adaptation target. I compared the magnitude of DT effects while (a) observers passively fixated the alphanumeric sequence (single task condition) and (b) observers performed the discrimination task (dual task condition). DT effects were significantly smaller in the dual task condition than in the single task condition indicating that MID processing is modulated by attention.