Spatial updating of dynamic scenes: Tracking multiple invisible objects across viewpoint changes

What Is the Role of Sustained Visual Attention in the Maintenance of Postural Control in Young Adults?

Dual tasks requiring sustained visual attention and upright stance are common, yet their impact on standing balance is not well understood. We investigated the role of visual attention in the maintenance of postural control, using the multiple-object tracking (MOT) task. Healthy young adults (n = 12) performed the MOT task at three object movement speeds while seated or standing. MOT performance was assessed using tracking capacity (k). Metrics calculated to assess mediolateral (ML) and anterior-posterior (AP) postural control included: maximum difference between CoM and CoP position (CoM-CoP Max), root mean square distance for center of pressure and center of mass position (CoP and CoM RMS distance), and correlation between CoM and CoP time series signals (CoM/CoP correlation). As predicted, k decreased significantly as object movement speed increased for both standing and seated conditions. Object movement speed also significantly affected AP CoM-CoP Max in seated conditions (p = .021) and AP CoM/CoP correlation for standing conditions (p = .002). The results demonstrate utility of the MOT task in understanding the role of visual attention in postural control, even though healthy young adults were able to compensate for the addition of a sustained visual attention task, with minimal deficits to postural control.

Effect of depth information on multiple-object tracking in three dimensions: A probabilistic perspective

Many daily situations require us to track multiple objects and people. This ability has traditionally been investigated in observers tracking objects in a plane. This simplification of reality does not address how observers track objects when targets move in three dimensions. Here, we study how observers track multiple objects in 2D and 3D while manipulating the average speed of the objects and the average distance between them. We show that performance declines as speed increases and distance decreases and that overall tracking accuracy is always higher in 3D than in 2D. The effects of distance and dimensionality interact to produce a more than additive improvement in performance during tracking in 3D compared to 2D. We propose an ideal observer model that uses the object dynamics and noisy observations to track the objects. This model provides a good fit to the data and explains the key findings of our experiment as originating from improved inference of object identity by adding the depth dimension.

Many daily life situations require us to track objects that are in motion. In the laboratory, this multiple object tracking problem is classically studied with objects moving on a two-dimensional screen, but in the real world objects typically move in three dimensions. Here we show that, despite the complexity of seeing in depth, observers track multiple objects better when they move in 3D than 2D. A probabilistic inference model explains this by showing that the association of noisy visual signals to the objects that caused them is less ambiguous when depth cues are available. This highlights the role that depth cues play in our everyday ability to track objects.

Distractor Locations Influence Multiple Object Tracking Beyond Interobject Spacing: Evidence From Equidistant Distractor Displacements

Human observers are able to keep track of several independently moving objects among other objects. Within theories of multiple object tracking (MOT), distractors are assumed to influence tracking performance only by their distance toward the next target. In order to test this assumption, we designed a variant of the MOT paradigm that involved spatially arranged target-distractor pairs and sudden displacements of distractors during a brief flash. Critically, these displacements maintained target-distractor spacing. Our results show that displacing distractors hurts tracking performance (Experiment 1). Importantly, target-distractor confusions occur within target-distractor pairs with displaced distractors (Experiment 2). This displacement effect increases with an increasing displacement angle (Experiment 3) but is equal at different distances between target and distractor (Experiment 4). This finding illustrates that distractors influence tracking performance beyond pure interobject spacing. We discuss how inhibitory processes as well as relations between targets and distractors might interfere with target tracking.

The role of relational triggers in event perception

Research exploring visual attention has demonstrated that people are aware of only a small proportion of visual properties, and that people only track these properties over a subset of moments in time. This makes it critical to understand how our perceptual system leverages its limited capacity, such that properties are tracked across views only when they can support an understanding of meaningful events. In this paper, we propose that relational triggers induce between-view property comparisons when spatial relationships between objects appear inconsistent across views-moments that are particularly likely to mark the beginning of meaningful events. In these experiments, we activate relational triggers by violating heuristics that filmmakers use to create visuospatial continuity across views. We find that these violations increase change detection when they coincide with visual property changes, demonstrating that relational triggers induce a comparison of properties held in working memory. We also demonstrate that relational triggers increase the likelihood of event segmentation, and that change detection increases both in response to triggers and natural event boundaries. We propose that relational triggers are an effective heuristic cue that facilitates the comparison of properties when they are likely to be useful during event perception.

Introduction to the special issue on visual working memory

Visual working memory is a volatile, limited-capacity memory that appears to play an important role in our impression of a visual world that is continuous in time. It also mediates between the contents of the mind and the contents of that visual world. Research on visual working memory has become increasingly prominent in recent years. The articles in this special issue of Attention, Perception, & Psychophysics describe new empirical findings and theoretical understandings of the topic.

Behavioral dynamics and neural grounding of a dynamic field theory of multi-object tracking

The ability to dynamically track moving objects in the environment is crucial for efficient interaction with the local surrounds. Here, we examined this ability in the context of the multi-object tracking (MOT) task. Several theories have been proposed to explain how people track moving objects; however, only one of these previous theories is implemented in a real-time process model, and there has been no direct contact between theories of object tracking and the growing neural literature using ERPs and fMRI. Here, we present a neural process model of object tracking that builds from a Dynamic Field Theory of spatial cognition. Simulations reveal that our dynamic field model captures recent behavioral data examining the impact of speed and tracking duration on MOT performance. Moreover, we show that the same model with the same trajectories and parameters can shed light on recent ERP results probing how people distribute attentional resources to targets vs. distractors. We conclude by comparing this new theory of object tracking to other recent accounts, and discuss how the neural grounding of the theory might be effectively explored in future work.

Spatial Reference in Multiple Object Tracking

Spatial reference in multiple object tracking is available from configurations of dynamic objects and static reference objects. In three experiments, we studied the use of spatial reference in tracking and in relocating targets after abrupt scene rotations. Observers tracked 1, 2, 3, 4, and 6 targets in 3D scenes, in which white balls moved on a square floor plane. The floor plane was either visible thus providing static spatial reference or it was invisible. Without scene rotations, the configuration of dynamic objects provided sufficient spatial reference and static spatial reference was not advantageous. In contrast, with abrupt scene rotations of 20°, static spatial reference supported in relocating targets. A wireframe floor plane lacking local visual detail was as effective as a checkerboard. Individually colored geometric forms as static reference objects provided no additional benefit either, even if targets were centered on these forms at the abrupt scene rotation. Individualizing the dynamic objects themselves by color for a brief interval around the abrupt scene rotation, however, did improve performance. We conclude that attentional tracking of moving targets proceeds within dynamic configurations but detached from static local background.

Brain activation during spatial updating and attentive tracking of moving targets

Keeping aware of the locations of objects while one is moving requires the updating of spatial representations. As long as the objects are visible, attentional tracking is sufficient, but knowing where objects out of view went in relation to one's own body involves an updating of spatial working memory. Here, multiple object tracking was employed to study spatial updating and its neural correlates. In a dynamic 3D-scene, targets moved among visually indistinguishable distractors. The targets and distractors either stayed visible during continuous viewpoint changes or they turned invisible. The parametric variation of tracking load revealed load-dependent activations of the intraparietal sulcus, the superior parietal lobule, and the lateral occipital cortex in response to the attentive tracking task. Viewpoint changes with invisible objects that demanded retention and updating produced load-dependent activation only in the precuneus in line with its presumed involvement in updating spatial working memory.

Remapping attention in multiple object tracking

Which coordinate system do we use to track moving objects? In a previous study using smooth pursuit eye movements, we argued that targets are tracked in both retinal (retinotopic) and scene-centered (allocentric) coordinates (Howe, Pinto, & Horowitz, 2010). However, multiple object tracking typically also elicits saccadic eye movements, which may change how object locations are represented. Observers fixated a cross while tracking three targets out of six identical disks confined to move within an imaginary square. The fixation cross alternated between two locations, requiring observers to make repeated saccades. By moving (or not moving) the imaginary square in sync with the fixation cross, we could disrupt either (or both) coordinate systems. Surprisingly, tracking performance was much worse when the objects moved with the fixation cross, although this manipulation preserved the retinal image across saccades, thereby avoiding the visual disruptions normally associated with saccades. Instead, tracking performance was best when the allocentric coordinate system was preserved, suggesting that targets locations are maintained in that coordinate system across saccades. This is consistent with a theoretical framework in which the positions of a small set of attentional pointers are predictively updated in advance of a saccade.

The coordinate systems used in visual tracking

Tracking moving objects is a fundamental attentional operation. Here we ask which coordinate system is used to track objects: retinal (retinotopic), scene-centered (allocentric), or both? Observers tracked three of six disks that were confined to move within an imaginary square. By moving either the imaginary square (and thus the disks contained within), the fixation cross, or both, we could dramatically increase the disks' speeds in one coordinate system while leaving them unchanged in the other, so as to impair tracking in only one coordinate system at a time. Hindering tracking in either coordinate system reduced tracking ability by an equal amount, suggesting that observers are compelled to use both coordinate systems and cannot choose to track only in the unimpaired coordinate system.