Attentional selection of moving objects by a serial process

Signal switching may enhance processing power of the brain

Our ability to perceive multiple objects is mysterious. Sensory neurons are broadly tuned, producing potential overlap in the populations of neurons activated by each object in a scene. This overlap raises questions about how distinct information is retained about each item. We present a novel signal switching theory of neural representation, which posits that neural signals may interleave representations of individual items across time. Evidence for this theory comes from new statistical tools that overcome the limitations inherent to standard time-and-trial-pooled assessments of neural signals. Our theory has implications for diverse domains of neuroscience, including attention, figure binding/scene segregation, oscillations, and divisive normalization. The general concept of switching between functions could also lend explanatory power to theories of grounded cognition.

Multiple object tracking with extended occlusions

In everyday life, we often view objects through a limited aperture (e.g., soccer players on TV or cars slipping into our blind spot on a busy road), where objects often move out of view and reappear in a different place later. We modelled this situation in a series of multiple object tracking (MOT) experiments, in which we introduced a cover on the edges of the observed area and manipulated its width. This method introduced systematic occlusions, which were longer than those used in previous MOT studies. Experiment 1 (N = 50) showed that tracking under such conditions is possible, although difficult. An item-level analysis confirmed that people made more errors in targets that were covered longer and more often. In Experiment 2 (N = 50), we manipulated the tracking workload and found that the participants were less affected by the cover when the tracking load was low. In Experiment 3 (N = 50), we asked the participants to keep track of the objects' identities (multiple identity tracking [MIT]). Although MIT is subjectively more demanding, memorising identities improved performance in the most difficult cover conditions. Contrary to previous reports, we also found that even partial occlusions negatively affected tracking.

Visual spatial attention and spatial working memory do not draw on shared capacity-limited core processes

The extent to which similar capacity limits in visual attention and visual working memory indicate a common shared underlying mechanism is currently still debated. In the spatial domain, the multiple object tracking (MOT) task has been used to assess the relationship between spatial attention and spatial working memory though existing results have been inconclusive. In three dual task experiments, we examined the extent of interference between attention to spatial positions and memory for spatial positions. When the position monitoring task required keeping track of target identities through colour-location binding, we found a moderate detrimental effect of position monitoring on spatial working memory and an ambiguous interaction effect. However, when this task requirement was removed, load increases in neither task were detrimental to the other. The only very moderate interference effect that remained resided in an interaction between load types but was not consistent with shared capacity between tasks-rather it was consistent with content-related crosstalk between spatial representations. Contrary to propositions that spatial attention and spatial working memory may draw on a common shared set of core processes, these findings indicate that for a purely spatial task, perceptual attention and working memory appear to recruit separate core capacity-limited processes.

Position tracking and identity tracking are separate systems: Evidence from eye movements

How do we track multiple moving objects in our visual environment? Some investigators argue that tracking is based on a parallel mechanism (e.g., Cavanagh & Alvarez, 2005; Pylyshyn, 1989), others argue that tracking contains a serial component (e.g. Holcombe & Chen, 2013; Oksama & Hyönä, 2008). In the present study, we put previous theories into a direct test by registering observers' eye movements when they tracked identical moving targets (the MOT task) or when they tracked distinct object identities (the MIT task). The eye movement technique is a useful tool to study whether overt focal attention is exploited during tracking. We found a qualitative difference between these tasks in terms of eye movements. When the participants tracked only position information (MOT), the observers had a clear preference for keeping their eyes fixed for a rather long time on the same screen position. In contrast, active eye behavior was observed when the observers tracked the identities of moving objects (MIT). The participants updated over four target identities with overt attention shifts. These data suggest that there are two separate systems involved in multiple object tracking. The position tracking system keeps track of the positions of the moving targets in parallel without the need of overt attention shifts in the form of eye movements. On the other hand, the identity tracking system maintains identity-location bindings in a serial fashion by utilizing overt attention shifts.

Bottlenecks of motion processing during a visual glance: the leaky flask model

Where do the bottlenecks for information and attention lie when our visual system processes incoming stimuli? The human visual system encodes the incoming stimulus and transfers its contents into three major memory systems with increasing time scales, viz., sensory (or iconic) memory, visual short-term memory (VSTM), and long-term memory (LTM). It is commonly believed that the major bottleneck of information processing resides in VSTM. In contrast to this view, we show major bottlenecks for motion processing prior to VSTM. In the first experiment, we examined bottlenecks at the stimulus encoding stage through a partial-report technique by delivering the cue immediately at the end of the stimulus presentation. In the second experiment, we varied the cue delay to investigate sensory memory and VSTM. Performance decayed exponentially as a function of cue delay and we used the time-constant of the exponential-decay to demarcate sensory memory from VSTM. We then decomposed performance in terms of quality and quantity measures to analyze bottlenecks along these dimensions. In terms of the quality of information, two thirds to three quarters of the motion-processing bottleneck occurs in stimulus encoding rather than memory stages. In terms of the quantity of information, the motion-processing bottleneck is distributed, with the stimulus-encoding stage accounting for one third of the bottleneck. The bottleneck for the stimulus-encoding stage is dominated by the selection compared to the filtering function of attention. We also found that the filtering function of attention is operating mainly at the sensory memory stage in a specific manner, i.e., influencing only quantity and sparing quality. These results provide a novel and more complete understanding of information processing and storage bottlenecks for motion processing.

Information extraction during simultaneous motion processing

When confronted with multiple moving objects the visual system can process them in two stages: an initial stage in which a limited number of signals are processed in parallel (i.e. simultaneously) followed by a sequential stage. We previously demonstrated that during the simultaneous stage, observers could discriminate between presentations containing up to 5 vs. 6 spatially localized motion signals (Edwards & Rideaux, 2013). Here we investigate what information is actually extracted during the simultaneous stage and whether the simultaneous limit varies with the detail of information extracted. This was achieved by measuring the ability of observers to extract varied information from low detail, i.e. the number of signals presented, to high detail, i.e. the actual directions present and the direction of a specific element, during the simultaneous stage. The results indicate that the resolution of simultaneous processing varies as a function of the information which is extracted, i.e. as the information extraction becomes more detailed, from the number of moving elements to the direction of a specific element, the capacity to process multiple signals is reduced. Thus, when assigning a capacity to simultaneous motion processing, this must be qualified by designating the degree of information extraction.

How many motion signals can be simultaneously perceived?

Previous research indicates that the maximum number of motion signals that can be simultaneously perceived is 2, if they are defined only by direction differences, or 3 if they also differ in speed or depth (Greenwood & Edwards, 2006b). Those previous studies used transparent, spatially-sparse stimuli. Here we investigate this motion-number perception limit using spatially-localised stimuli that drive either the standard or form-specific motion systems (Edwards, 2009). Each motion signal was defined by four signal-dots that were arranged in either a square pattern (Square Condition), to drive the form-specific system, or a random pattern (Random Condition), to drive the standard motion-system. A temporal 2AFC procedure was used with each interval (150 ms duration) containing n or n+1 signals. The observer had to identify the interval containing the highest number of signals. The total number of dots in each interval was kept constant by varying the number of noise dots (dots that started off in the same spatial arrangement as the signal dots but then each of those dots moved in different directions). A mask was used at the end of each motion sequence to prohibit the use of iconic memory. In the Square Condition, up to five directions could be simultaneously perceived, and only 1 in the Variable condition. Decreasing the number of noise dots improved performance for the Variable condition, and increasing it decreased performance in the Square Condition. These results show that the previously observed limit of 3 is not a universal limit for motion perception and further, that signal-to-noise limits are a fundamental factor in determining the number of directions that can be simultaneously perceived. Hence the greater sensitivity to motion of the form-specific system makes it well suited to extracting the motion of multiple moving objects.

How Many Objects are You Worth? Quantification of the Self-Motion Load on Multiple Object Tracking

Perhaps walking and chewing gum is effortless, but walking and tracking moving objects is not. Multiple object tracking is impaired by walking from one location to another, suggesting that updating location of the self puts demands on object tracking processes. Here, we quantified the cost of self-motion in terms of the tracking load. Participants in a virtual environment tracked a variable number of targets (1-5) among distractors while either staying in one place or moving along a path that was similar to the objects' motion. At the end of each trial, participants decided whether a probed dot was a target or distractor. As in our previous work, self-motion significantly impaired performance in tracking multiple targets. Quantifying tracking capacity for each individual under move versus stay conditions further revealed that self-motion during tracking produced a cost to capacity of about 0.8 (±0.2) objects. Tracking your own motion is worth about one object, suggesting that updating the location of the self is similar, but perhaps slightly easier, than updating locations of objects.

Direction information in multiple object tracking is limited by a graded resource

Is multiple object tracking (MOT) limited by a fixed set of structures (slots), a limited but divisible resource, or both? Here, we answer this question by measuring the precision of the direction representation for tracked targets. The signature of a limited resource is a decrease in precision as the square root of the tracking load. The signature of fixed slots is a fixed precision. Hybrid models predict a rapid decrease to asymptotic precision. In two experiments, observers tracked moving disks and reported target motion direction by adjusting a probe arrow. We derived the precision of representation of correctly tracked targets using a mixture distribution analysis. Precision declined with target load according to the square-root law up to six targets. This finding is inconsistent with both pure and hybrid slot models. Instead, directional information in MOT appears to be limited by a continuously divisible resource.

Distinguishing between parallel and serial accounts of multiple object tracking

Humans can track multiple moving objects. Is this accomplished by attending to all the objects at the same time or do we attend to each object in turn? We addressed this question using a novel application of the classic simultaneous-sequential paradigm. We considered a display in which objects moved for only part of the time. In one condition, the objects moved sequentially, whereas in the other condition they all moved and paused simultaneously. A parallel model would predict that the targets are tracked independently, so the tracking of one target should not be influenced by the movement of another target. Thus, one would expect equal performance in the two conditions. Conversely, a simple serial account of object tracking would predict that an observer's accuracy should be greater in the sequential condition because in that condition, at any one time, fewer targets are moving and thus need to be attended. In fact, in our experiments we observed performance in the simultaneous condition to be equal to or greater than the performance in the sequential condition. This occurred regardless of the number of targets or how the targets were positioned in the visual field. These results are more directly in line with a parallel account of multiple object tracking.

The gender-specific face aftereffect is based in retinotopic not spatiotopic coordinates across several natural image transformations

In four experiments, we measured the gender-specific face-aftereffect following subject's eye movement, head rotation, or head movement toward the display and following movement of the adapting stimulus itself to a new test location. In all experiments, the face aftereffect was strongest at the retinal position, orientation, and size of the adaptor. There was no advantage for the spatiotopic location in any experiment nor was there an advantage for the location newly occupied by the adapting face after it moved in the final experiment. Nevertheless, the aftereffect showed a broad gradient of transfer across location, orientation and size that, although centered on the retinotopic values of the adapting stimulus, covered ranges far exceeding the tuning bandwidths of neurons in early visual cortices. These results are consistent with a high-level site of adaptation (e.g. FFA) where units of face analysis have modest coverage of visual field, centered in retinotopic coordinates, but relatively broad tolerance for variations in size and orientation.