Tracking unique objects

Gaze Behavior and Cognitive Performance on Tasks of Multiple Object Tracking and Multiple Identity Tracking by Handball Players and Non-Athletes

Multiple object tracking (MOT) and multiple identity tracking (MIT) each measure the ability to track moving objects visually. While prior investigators have mainly compared athletes and non-athletes on MOT, MIT more closely resembles dynamic real-life environments. Here we compared the performance and gaze behavior of handball players with non-athletes on both MOT and MIT. Since previous researchers have shown that MOT and MIT engage different eye movement strategies, we had participants track 3-5 targets among 10 moving objects. In MOT, the objects were identical, while in MIT they differed in shape and color. Although we observed no group differences for tracking accuracy, the eye movements of athletes were more target-oriented than those of non-athletes. We concluded that tasks and stimuli intended by researchers to demonstrate that athletes' show better object tracking than non-athletes should be specific to the athletes' type of sport and should use more perception-action coupled measures. An implication of this conclusion is that the differences in object tracking skills between athletes and non-athletes is highly specific to the skills demanded by the athletes' sport.

Gaze coherence reveals distinct tracking strategies in multiple object and multiple identity tracking

In dynamic environments, a central task of the attentional system is to keep track of objects changing their spatial location over time. In some instances, it is sufficient to track only the spatial locations of moving objects (i.e., multiple object tracking; MOT). In other instances, however, it is also important to maintain distinct identities of moving objects (i.e., multiple identity tracking; MIT). Despite previous research, it is not clear whether MOT and MIT performance emerge from the same tracking mechanism. In the present report, we study gaze coherence (i.e., the extent to which participants repeat their gaze behaviour when tracking the same object locations twice) across repeated MOT and MIT trials. We observed more substantial gaze coherence in repeated MOT trials compared to the repeated MIT trials or mixed MOT-MIT trial pairs. A subsequent simulation study suggests that MOT is based more on a grouping mechanism than MIT, whereas MIT is based more on a target-jumping mechanism than MOT. It thus appears unlikely that MOT and MIT emerge from the same basic tracking mechanism.

Multiple Object Tracking Without Pre-attentive Indexing

Multiple object tracking (MOT) involves simultaneous tracking of a certain number of target objects amongst a larger set of objects as they all move unpredictably over time. The prevalent explanation for successful target tracking by humans in MOT involving visually identical objects is based on the Visual Indexing Theory. This assumes that each target is indexed by a pointer using a non-conceptual mechanism to maintain an object's identity even as its properties change over time. Thus, successful tracking requires successful indexing and the absence of identification errors. Identity maintenance and successful tracking are measured in terms of identification (ID) and tracking accuracy respectively, with higher accuracy indicating better identity maintenance or better tracking. Existing evidence suggests that humans have high tracking accuracy despite poor identification accuracy, suggesting that it might be possible to perform MOT without indexing. Our work adds to existing evidence for this position through two experiments, and presents a computational model of multiple object tracking that does not require indexes. Our empirical results show that identification accuracy is aligned with tracking accuracy in humans for tracking up to three, but is lower when tracking more objects. Our computational model of MOT without indexing accounts for several empirical tracking accuracy patterns shown in earlier studies, reproduces the dissociation between tracking and identification accuracy produced earlier in the literature as well as in our experiments, and makes several novel predictions.

Multiple object tracking with extended occlusions

In everyday life, we often view objects through a limited aperture (e.g., soccer players on TV or cars slipping into our blind spot on a busy road), where objects often move out of view and reappear in a different place later. We modelled this situation in a series of multiple object tracking (MOT) experiments, in which we introduced a cover on the edges of the observed area and manipulated its width. This method introduced systematic occlusions, which were longer than those used in previous MOT studies. Experiment 1 (N = 50) showed that tracking under such conditions is possible, although difficult. An item-level analysis confirmed that people made more errors in targets that were covered longer and more often. In Experiment 2 (N = 50), we manipulated the tracking workload and found that the participants were less affected by the cover when the tracking load was low. In Experiment 3 (N = 50), we asked the participants to keep track of the objects' identities (multiple identity tracking [MIT]). Although MIT is subjectively more demanding, memorising identities improved performance in the most difficult cover conditions. Contrary to previous reports, we also found that even partial occlusions negatively affected tracking.

Difficulty limits of visual mental imagery

While past work has focused on the representational format of mental imagery, and the similarities of its operation and neural substrate to online perception, surprisingly little has tested the boundaries of the level of detail that mental imagery can generate. To answer this question, we take inspiration from the visual short-term memory literature, a related field which has found that memory capacity is affected by the number of items, whether they are unique, and whether and how they move. We test these factors of set size, color heterogeneity, and transformation in mental imagery through both subjective (Exp 1; Exp 2) and objective (Exp 2) measures - difficulty ratings and a change detection task, respectively - to determine the capacity limits of our mental imagery, and find that limits on mental imagery are similar to those for visual short-term memory. In Experiment 1, participants rated the difficulty of imagining 1-4 colored items as subjectively more difficult when there were more items, when the items had unique colors instead of an identical color, and when they scaled or rotated instead of merely linearly translating. Experiment 2 isolated these subjective difficulty ratings of rotation for uniquely colored items, and added a rotation distance manipulation (10° to 110°), again finding higher subjective difficulty for more items, and for when those items rotated farther; the objective measure showed a decrease in performance for more items, but not for rotational degree. Congruities between the subjective and objective results suggest similar costs, but some incongruities suggest that subjective reports can be overly optimistic, likely because they are biased by an illusion of detail.

Disentangling working memory from multiple-object tracking: Evidence from dual-task interferences

Multiple object tracking (MOT) is generally regarded as a pure attention-consuming task that draws heavily on attention resources. In the present study, we adopted a cross-channel visual-audio dual-task paradigm, i.e., the MOT task combined with the concurrent auditory N-back working memory task, to test whether working memory indeed plays a necessary role in the process of multiple tracking, as well as to further identify the specific types of working memory components involved in this process. Experiments 1a and 1b examined the relationship between the MOT task and nonspatial object working memory (OWM) processing by manipulating the tracking load and working memory load, respectively. Results in both experiments indicated that the concurrent nonspatial OWM task did not have a significant effect on the tracking capacity of the MOT task. In contrast, Experiments 2a and 2b examined the relationship between the MOT task and spatial working memory (SWM) processing by a similar approach. Results in both experiments indicated that the concurrent SWM task significantly impaired the tracking capacity of the MOT task, showing a gradual decrease with increasing SWM load. Overall, our study provides empirical evidence that multiple object tracking does involve working memory, primarily related to spatial working memory rather than nonspatial object working memory, which sheds more light on the mechanisms of multiple object tracking.

Inattentional blindness: Attentional set for efficient task success

Inattentional blindness is the failure to notice an unexpected object in plain sight when attention is otherwise engaged. We investigated what determines observers' attentional set in a dynamic-counting inattentional blindness paradigm, when task instructions and visual distinctiveness of task-relevant objects were either congruent or in opposition. In seven experiments, observers counted bounces by task-relevant objects, with the instruction either to count-by-shape (squares, diamonds, crosses) or count-by-colour (blue, purple). To manipulate visual distinctiveness, we varied the extent to which task-relevant and task-irrelevant objects looked different on two dimensions: shape and colour. When colour better distinguished task-relevant from task-irrelevant objects, observers-even if instructed count-by-shape-reported an unexpected object that matched the colour of task-relevant objects. Crucially, when instructed count-by-colour, but shape better distinguished task-relevant from task-irrelevant objects, observers reported an unexpected object that matched the shape of task-relevant objects. We conclude that observers set their attention to promote efficient task performance.

The role of motion in visual working memory for dynamic stimuli: More lagged but more precise representations of moving objects

While most visual working memory studies use static stimuli with unchanging features, objects in the real world are often dynamic, introducing significant differences in the surface feature information hitting the retina from the same object over time (e.g., changes in orientation, lighting, shadows). Previous research on dynamic stimuli has shown that change detection is improved if objects obey rules of physical motion, but it is unclear how memory for visual features interacts with object motion. In the current study, we investigated whether object motion facilitates greater temporal integration of continuously changing surface feature information. In a series of experiments, participants were asked to report the final color of continuously changing colored dots that were either moving or stationary on the screen. We found that the reported colors "lagged behind" the physical states of the dots when they were in motion. We also observed that the precision of memory responses was significantly higher for stimuli in the moving condition compared to the stationary condition. Together, these findings suggest that memory representation is improved - but lagged - for moving objects, consistent with the idea that object motion may facilitate integration of object information over longer intervals.

Characterizing motion prediction in small autonomous swarms

The use of robotic swarms has become increasingly common in research, industrial, and military domains for tasks such as collective exploration, coordinated movement, and collective localization. Despite the expanded use of robotic swarms, little is known about how swarms are perceived by human operators. To characterize human-swarm interactions, we evaluate how operators perceive swarm characteristics, including movement patterns, control schemes, and occlusion. In a series of experiments manipulating movement patterns and control schemes, participants tracked swarms on a computer screen until they were occluded from view, at which point participants were instructed to estimate the spatiotemporal dynamics of the occluded swarm by mouse click. In addition to capturing mouse click responses, eye tracking was used to capture participants eye movements while visually tracking swarms. We observed that manipulating control schemes had minimal impact on the perception of swarms, and that swarms are easier to track when they are visible compared to when they were occluded. Regarding swarm movements, a complex pattern of data emerged. For example, eye tracking indicates that participants more closely track a swarm in an arc pattern compared to sinusoid and linear movement patterns. When evaluating behavioral click-responses, data show that time is underestimated, and that spatial accuracy is reduced in complex patterns. Results suggest that measures of performance may capture different patterns of behavior, underscoring the need for multiple measures to accurately characterize performance. In addition, the lack of generalizable data across different movement patterns highlights the complexity involved in the perception of swarms of objects.

Exploring the effectiveness of auditory, visual, and audio-visual sensory cues in a multiple object tracking environment

Maintaining object correspondence among multiple moving objects is an essential task of the perceptual system in many everyday life activities. A substantial body of research has confirmed that observers are able to track multiple target objects amongst identical distractors based only on their spatiotemporal information. However, naturalistic tasks typically involve the integration of information from more than one modality, and there is limited research investigating whether auditory and audio-visual cues improve tracking. In two experiments, we asked participants to track either five target objects or three versus five target objects amongst similarly indistinguishable distractor objects for 14 s. During the tracking interval, the target objects bounced occasionally against the boundary of a centralised orange circle. A visual cue, an auditory cue, neither or both coincided with these collisions. Following the motion interval, the participants were asked to indicate all target objects. Across both experiments and both set sizes, our results indicated that visual and auditory cues increased tracking accuracy although visual cues were more effective than auditory cues. Audio-visual cues, however, did not increase tracking performance beyond the level of purely visual cues for both high and low load conditions. We discuss the theoretical implications of our findings for multiple object tracking as well as for the principles of multisensory integration.

Tracking multiple fish

Although the Multiple Object Tracking (MOT) task is a widely used experimental method for studying divided attention, tracking objects in the real world usually looks different. For example, in the real world, objects are usually clearly distinguishable from each other and also possess different movement patterns. One such case is tracking groups of creatures, such as tracking fish in an aquarium. We used movies of fish in an aquarium and measured general tracking performance in this task (Experiment 1). In Experiment 2, we compared tracking accuracy within-subjects in fish tracking, tracking typical MOT stimuli, and in a third condition using standard MOT uniform objects which possessed movement patterns similar to the real fish. This third condition was added to further examine the impact of different motion characteristics on tracking performance. Results within a Bayesian framework showed that tracking real fish shares similarities with tracking simple objects in a typical laboratory MOT task. Furthermore, we observed a close relationship between performance in both laboratory MOT tasks (typical and fish-like) and real fish tracking, suggesting that the commonly used laboratory MOT task possesses a good level of ecological validity.

Finding meaning in "wrong responses": The multiple object-awareness paradigm shows that visual awareness is probabilistic

Visual information that observers perceive and remember at any given moment guides behavior in daily life. However, binary alternative-forced choice responses, often used in visual research, limit the report of the visual information that observers perceive and remember. We used a new multiple object-awareness paradigm where observers can use multiple clicks to find a target. We calculated visual awareness capacity based on the first-attempt accuracy and the total number of clicks, respectively. Results showed that the capacity estimated by the clicks in guessing from N was significantly greater than that estimated by the first-attempt accuracy. Further, analysis found that if observers could not locate the target in their first attempt, they were more likely to click closer to the target or on stimuli that matched its color. In addition, we found that even when observers used the same number of clicks to find a target (2 or 3), the average distance was shorter when observers reported high-level subjective visibility. The findings are compatible with the partial awareness hypothesis, and the visual ensembles and summary statistics hypothesis, which hold that visual awareness is probabilistic. These results also support the visual short-term memory models where many items are stored but with a resolution or noise level that depends on the number of items in memory.

How do humans group non-rigid objects in multiple object tracking?: Evidence from grouping by self-rotation

Previous studies on perceptual grouping found that people can use spatiotemporal and featural information to group spatially separated rigid objects into a unit while tracking moving objects. However, few studies have tested the role of objects' self-motion information in perceptual grouping, although it is of great significance to the motion perception in the three-dimensional space. In natural environments, objects always move in translation and rotation at the same time. The self-rotation of the objects seriously destroys objects' rigidity and topology, creates conflicting movement signals and results in crowding effects. Thus, this study sought to examine the specific role played by self-rotation information on grouping spatially separated non-rigid objects through a modified multiple object tracking (MOT) paradigm with self-rotating objects. Experiment 1 found that people could use self-rotation information to group spatially separated non-rigid objects, even though this information was deleterious for attentive tracking and irrelevant to the task requirements, and people seemed to use it strategically rather than automatically. Experiment 2 provided stronger evidence that this grouping advantage did come from the self-rotation per se rather than surface-level cues arising from self-rotation (e.g. similar 2D motion signals and common shapes). Experiment 3 changed the stimuli to more natural 3D cubes to strengthen the impression of self-rotation and again found that self-rotation improved grouping. Finally, Experiment 4 demonstrated that grouping by self-rotation and grouping by changing shape were statistically comparable but additive, suggesting that they were two different sources of the object information. Thus, grouping by self-rotation mainly benefited from the perceptual differences in motion flow fields rather than in deformation. Overall, this study is the first attempt to identify self-motion as a new feature that people can use to group objects in dynamic scenes and shed light on debates about what entities/units we group and what kinds of information about a target we process while tracking objects.

Capturing the objects of vision with neural networks

Human visual perception carves a scene at its physical joints, decomposing the world into objects, which are selectively attended, tracked and predicted as we engage our surroundings. Object representations emancipate perception from the sensory input, enabling us to keep in mind that which is out of sight and to use perceptual content as a basis for action and symbolic cognition. Human behavioural studies have documented how object representations emerge through grouping, amodal completion, proto-objects and object files. By contrast, deep neural network models of visual object recognition remain largely tethered to sensory input, despite achieving human-level performance at labelling objects. Here, we review related work in both fields and examine how these fields can help each other. The cognitive literature provides a starting point for the development of new experimental tasks that reveal mechanisms of human object perception and serve as benchmarks driving the development of deep neural network models that will put the object into object recognition.

Attentional capture in multiple object tracking

Attentional processes are generally assumed to be involved in multiple object tracking (MOT). The attentional capture paradigm is regularly used to study conditions of attentional control. It has up to now not been used to assess influences of sudden onset distractor stimuli in MOT. We investigated whether attentional capture does occur in MOT: Are onset distractors processed at all in dynamic attentional tasks? We found that sudden onset distractors were effective in lowering probe detection, thus demonstrating attentional capture. Tracking performance as dependent measure was not affected. The attentional capture effect persisted in conditions of higher tracking load (Experiment 2) and was dramatically increased in lower presentation frequency of the onset distractor (Experiment 3). Tracking performance was shown to suffer only when onset distractors were presented serially with very short time gaps in between, thus effectively disturbing re-engaging attention on the tracking set (Experiment 4). We discuss that rapid dis- and re-engagement of the attention process on target objects and an additional more basic process that continuously provides location information allow managing strong disruptions of attention during tracking.

Adaptive target enhancement determines levels of guidability in Multiple Object Tracking

Flexible resource models suggest that attentional deployment in Multiple Object Tracking either shows in target enhancement or distractor inhibition or both. In order to gauge the influence of these processes we manipulated distinctiveness of targets and distractors along a single dimension by finely grading gray values of objects. We established that a quantitative increase in distinctiveness results in a quantitative increase in tracking performance thus demonstrating a new finding. Further increases of distinctiveness beyond a certain degree produced no further improvements in tracking. This effect, however, was moderated by levels of task difficulty thus providing evidence for the relativity of this effect. With higher task difficulty higher degrees of distinctiveness still resulted in a significant performance gain (Experiment 1). A follow up experiment generalized this finding. By manipulating speed of objects, a situation with a yet higher task difficulty could be established showing that an increase of an already striking distinctiveness can produce an even further performance gain - provided higher task difficulty (Experiment 2). Finally we tested the hypothesis that target enhancement and distractor inhibition are influenced by feature distinctiveness. Although we found possible indications for distractor inhibition, we could not demonstrate an effect of feature distinctiveness on distractor inhibition. Target enhancement, however, was substantially influenced by variation of feature distinctiveness (Experiment 3).

Ensemble perception during multiple-object tracking

Multiple-object tracking studies consistently reveal attentive tracking limits of approximately three to five items. How do factors such as visual grouping and ensemble perception impact these capacity limits? Which heuristics lead to the perception of multiple objects as a group? This work investigates the role of grouping on multiple-object tracking ability, and more specifically, in identifying the heuristics that lead to the formation and perception of ensembles within dynamic contexts. First, we show that group tracking limits are approximately four groups of objects and are independent of the number of items that compose the groups. Further, we show that group tracking performance declines as inter-object spacing increases. We also demonstrate the role of group rigidity in tracking performance in that disruptions to common fate negatively impact ensemble tracking ability. The findings from this work contribute to our overall understanding of the perception of dynamic groups of objects. They characterize the properties that determine the formation and perception of dynamic object ensembles. In addition, they inform development and design decisions considering cognitive limitations involving tracking groups of objects.

Distinguishing the neural mechanism of attentional control and working memory in feature-based attentive tracking

Surface features are an important component in attentive tracking. However, the neural mechanisms underlying how features affect attentive tracking remain unknown. The present fMRI study addressed this issue by manipulating the intragroup feature complexity and intergroup feature similarity. In particular, this study distinguished the different neural mechanisms of intragroup feature complexity and intergroup feature similarity by investigating the roles of attentional control and working memory in dynamic feature-based attentive tracking. Behavioral and neuroimaging evidence showed that when targets are distinct from distractors, the intragroup feature complexity of the targets, rather than that of the distractors, mainly increases the visual working memory load and significantly activates the frontoparietal cortical circuit. Thus, the involvement of working memory in feature-based attentive tracking is modulated by goal-directed attention control. In addition, when targets are similar to distractors, the intergroup feature similarity (i.e., target-distractor similarity) mainly affects the allocation of attention. Specifically, target-distractor similarity affects the goal-directed attention toward the targets in a stimulus-driven way and induces an interaction between the ventral and dorsal attention networks.

How Do Humans Perform in Multiple Object Tracking With Unstable Features

In real-world scenarios, objects’ surface features sometimes change as they move, impairing the continuity of objects. However, it is still unknown how our visual system adapts to this dynamic change. Hence, the present study investigated the role of feature changes in attentive tracking through a modified multiple object tracking (MOT) task. The feature heterogeneity and feature stability were manipulated in two experiments. The results from Experiment 1 showed that the tracking performance under feature-changed condition was lower than that under the feature-fixed condition only when the objects were four colors grouped or all unique, suggesting that the performance decrease was moderated by the feature heterogeneity. In Experiment 2, we further examined this effect by manipulating the frequency of feature change. The results showed that when the target set was one color or two colors grouped (the color grouping for the distractor set corresponded with it), the tracking performance decreased significantly as the feature-change frequency increased. However, this trend was not the case when the objects were of the same color or eight unique colors. In addition, a relatively consistent effect appeared both in Experiments 1 and 2. When objects have unique features, the tracking performance decreased significantly as the increase of feature heterogeneity in each frequency of feature changes. Taken together, we concluded that unstable features could be utilized in attentive tracking, and the extent to which the observers relied on surface feature information to assist tracking depended on the level of feature heterogeneity and the frequency of feature change.

Multiple-target tracking in human and machine vision

Humans are able to track multiple objects at any given time in their daily activities—for example, we can drive a car while monitoring obstacles, pedestrians, and other vehicles. Several past studies have examined how humans track targets simultaneously and what underlying behavioral and neural mechanisms they use. At the same time, computer-vision researchers have proposed different algorithms to track multiple targets automatically. These algorithms are useful for video surveillance, team-sport analysis, video analysis, video summarization, and human–computer interaction. Although there are several efficient biologically inspired algorithms in artificial intelligence, the human multiple-target tracking (MTT) ability is rarely imitated in computer-vision algorithms. In this paper, we review MTT studies in neuroscience and biologically inspired MTT methods in computer vision and discuss the ways in which they can be seen as complementary.

Multiple-target tracking (MTT) is a challenging task vital for both a human’s daily life and for many artificial intelligent systems, such as those used for urban traffic control. Neuroscientists are interested in discovering the underlying neural mechanisms that successfully exploit cognitive resources, e.g., spatial attention or memory, during MTT. Computer-vision specialists aim to develop powerful MTT algorithms based on advanced models or data-driven computational methods. In this paper, we review MTT studies from both communities and discuss how findings from cognitive studies can inspire developers to construct higher performing MTT algorithms. Moreover, some directions have been proposed through which MTT algorithms could raise new questions in the cognitive science domain, and answering them can shed light on neural processes underlying MTT.

Multiple identity tracking strategies vary by age: An ERP study

Top-down modulation underlies our ability to focus attention on task-relevant stimuli and ignore irrelevant distractions. Although age-related differences in neural correlates of top-down modulation have been investigated in multiple studies using variety of tasks (Gazzaley et al., 2005; Störmer et al., 2013), the effect of age on top-down modulation in a multiple identity tracking (MIT) task is still unknown. Thus, we investigated age-related differences in the MIT task by employing event-related potentials (ERPs). Participants tracked ten uniquely colored disks, two of which were randomly designated as targets at the beginning of each trial; the targets moved among four stationary distractors (serving as ERP baseline) and four moving distractors. Each type of stimulus was probed during the trial to capture differential patterns of brain activation. Tracking performance was similar across age groups. ERP data showed that younger adults performed the MIT task by enhancing the unique identities associated with targets relative to distractors through feature-based tracking. Older adults showed a pattern of distractor suppression engaging both location- and feature-based tracking strategies. Thus, our findings suggest that compared to younger adults, older adults engage greater levels of neural activity to achieve the same level of performance. These findings are discussed in light of theories of cognitive aging.

Eye Behavior During Multiple Object Tracking and Multiple Identity Tracking

We review all published eye-tracking studies to date that have used eye movements to examine multiple object (MOT) or multiple identity tracking (MIT). In both tasks, observers dynamically track multiple moving objects. In MOT the objects are identical, whereas in MIT they have distinct identities. In MOT, observers prefer to fixate on blank space, which is often the center of gravity formed by the moving targets (centroid). In contrast, in MIT observers have a strong preference for the target-switching strategy, presumably to refresh and maintain identity-location bindings for the targets. To account for the qualitative differences between MOT and MIT, two mechanisms have been posited, a position tracking (MOT) and an identity tracking (MOT & MIT) mechanism. Eye-tracking studies of MOT have also demonstrated that observers execute rescue saccades toward targets in danger of becoming occluded or are about to change direction after a collision. Crowding attracts the eyes close to it in order to increase visual acuity for the crowded objects to prevent target loss. It is suggested that future studies should concentrate more on MIT, as MIT more closely resembles tracking in the real world.

Two-year-olds succeed at MIT: Multiple identity tracking in 20- and 25-month-old infants

Infants' ability to remember objects and their locations emerges during the first year of life. However, not much is known about infants' ability to track objects' identities in a dynamic environment. Here, we tailored the delayed match retrieval eye-tracking paradigm to study infants' ability to track two object identities during occlusion-an infant version of multiple identity tracking (MIT). Delayed match retrieval uses virtual "cards" as stimuli that are first shown face up, exposing to-be-remembered information, and then turned face down, occluding it. Here, cards were subject to movement during the face-down occlusion period. We used complex non-nameable objects as card faces to discourage verbal rehearsal. In three experiments (N = 110), we compared infants' ability to track object identities when two previously exposed cards were static (Experiment 1), were moved into new positions along the same trajectory (Experiment 2), or were moved along different trajectories (Experiment 3) while face down. We found that 20-month-olds could remember two object identities when static; however, it was not until 25 months of age that infants could track when movement was introduced. Our results show that the ability to track multiple identities in visual working memory is present by 25 months.

The Global Properties of Objects Play the Main Role in Facilitating Multiple Object Tracking Performance

Previous research has revealed the uniqueness-facilitation effect in the multiple object tracking (MOT) task: simple distinct identities and surface features of moving targets could facilitate attentional tracking. By adapting compound stimuli, the present study investigated whether the global or local properties played the main role in the uniqueness-facilitation effect in the MOT task. The uniqueness of local properties, of global properties or of both local and global properties were considered. Observers’ tracking performance in alternative conditions were compared with that in the homogeneous condition wherein all stimuli have identical local and global properties. Results from two experiments suggest that the global properties played the key role in facilitating tracking. The distinctiveness of local properties can also facilitate tracking with global properties being homogeneous. However, when the stimuli’s global properties are distinct from each other—whether the local properties being unique or not—observers’ tracking performance can achieve the same level as that in the unitary-uniqueness condition wherein the moving objects were distinct unitary letters. These results revealed a global superiority effect in the MOT task. Finally, the facilitation effects of the global and local properties might depend on the stimulus sparsity. When the compound stimuli had fewer local elements, the uniqueness facilitation effect on tracking decreased.

Perception in dynamic scenes: What is your Heider capacity?

The classic animation experiment by Heider and Simmel (1944) revealed that humans have a strong tendency to impose narrative even on displays showing interactions between simple geometric shapes. In their most famous animation with three simple shapes, observers almost inevitably interpreted them as rational agents with intentions, desires, and beliefs ("That nasty big triangle!"). Much work on dynamic scenes has identified basic visual properties that can make shapes seem animate. Here, we investigate the limits on the ability to use narrative to share information about animated scenes. We created 30 second Heider-style cartoons with 3-9 items. Item trajectories were generated automatically by a simple set of rules, but without a script. In Experiments 1 and 2, 10 observers wrote short narratives for each cartoon. Next, new observers were shown a cartoon and then presented with a narrative generated for that specific cartoon or one generated for a different cartoon having the same items. Observers rated the fit of the narrative to the cartoon on a scale from 1 (clearly does not fit) to 5 (clearly fits). Performance declined markedly when the number of items was larger than 3. Experiment 3 had observers determine if a short clip of a cartoon came from a longer clip. Experiment 4 had observers determine which of two narratives fit a cartoon. Finally, in Experiment 5, narratives always mentioned every item in a display. In all cases of matching narrative to cartoon, performance drops most dramatically between 3 and 4 items. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The effects of colour complexity and similarity on multiple object tracking performance

Surface features can be used during multiple object tracking (MOT). Previous studies suggested that surface features might be stored in visual working memory to assist object tracking, and attentive tracking and visual working memory share common attentional resources. However, it is still unknown whether features of both the target and distractor sets will be stored, or features of the target and distractor sets are processed differently. Moreover, how feature distinctiveness and similarity between the target and distractor sets affect tracking and allocation of attentional resources are still not clear. First, we manipulated the colour complexity of the target set (CT) and the colour complexity of the distractor set (CD), respectively, in two experiments, where colours of the target and distractor sets were always distinct, to test their effects on tracking performance. If features of the target and distractor sets are stored, manipulating feature complexity of the target and distractor sets would significantly affect tracking performance. Second, this study tested whether tracking performance was affected by different levels of distinctiveness between the target and distractor sets (DTD) and explored how distinctiveness affected tracking and allocation of attentional resources. Results showed that DTD and CT significantly affect tracking performance and allocation of attentional resources, but not CD. These results indicated that when targets and distractors have distinct features, only the surface features of the targets are maintained in visual working memory. And when targets have the same colour with the distractors, they are more difficult and consume more attentional resources to track.

Model of Multiple Identity Tracking (MOMIT) 2.0: Resolving the serial vs. parallel controversy in tracking

The present study investigated whether during tracking of multiple moving objects with distinct identities only one identity is tracked at each moment (serial tracking) or whether multiple identities can be tracked simultaneously (parallel tracking). By adopting the gaze-contingent display change technique, we manipulated in real time the presence/absence of object identities during tracking. The data on performance accuracy revealed a serial tracking pattern for facial images and a parallel pattern for color discs: when tracking faces, the presence/absence of only the currently foveated identity impacted the performance, whereas when tracking colors, the presence of multiple identities across the visual field led to improved tracking performance. This pattern is consistent with the identifiability of the different types of objects in the visual field. The eye movements during MIT showed a bias towards visiting and dwelling on individual targets when facial identities were present and towards visiting the blank areas between targets when color identities were present. Nevertheless, the eye visits were predominately on individual targets regardless of the type of objects and the presence of object identities. The eye visits to targets were beneficial for target tracking, particularly in face tracking. We propose the Model of Multiple Identity Tracking (MOMIT) 2.0 which accounts for the results and reconcile the serial vs. parallel controversy. The model suggests that observers cooperatively use attention, eye movements, perception, and working memory for dynamic tracking. Tracking appears more serial when high-resolution information needs to be sampled and maintained for discriminating the targets, whereas it appears more parallel when low-resolution information is sufficient.

A New Multiple Object Awareness Paradigm Shows that Imperfect Knowledge of Object Location Is Still Knowledge

Keeping track of multiple moving items is a fundamental perceptual skill. Nevertheless, our capacity in tasks such as multiple object tracking (MOT) [1, 2], multiple identity tracking (MIT) [3, 4], or change detection [5] is very low-typically just two to four items. In standard tracking tasks, observers monitor M out of N items as they move about the screen. At testing time, observers are asked to mark all tracked items or to state if a specific item is a target. If they are wrong about an item, they are considered completely wrong. Similarly, in an identity tracking experiment, observers might be asked, "Where is the horse?" If they mark the neighboring location where the cow is hidden, again, they are marked completely wrong. If an observer knew approximately, but not exactly, where the horse was, she would get no credit for that knowledge. In our multiple object awareness task, observers continue to click on locations until they locate the target for that trial so that any imprecise knowledge can be used in observers' responses. A model in which location information degrades with time after an item is attended can be used to estimate a new capacity of multiple object awareness-the number of items about which the observer has some location information. This capacity is at least double the usual capacity estimates for identity tracking.

It is not good to talk: conversation has a fixed interference cost on attention regardless of difficulty

It is well-documented that telephone conversations lead to impaired driving performance. Kunar et al. (Psychon Bull Rev 15:1135–1140, 2008) showed that this deficit was, in part, due to a dual-task cost of conversation on sustained visual attention. Using a multiple object tracking (MOT) task they found that the act of conversing on a hands-free telephone resulted in slower response times and increased errors compared to when participants performed the MOT task alone. The current study investigates whether the dual-task impairment of conversation on sustained attention is affected by conversation difficulty or task difficulty, and whether there was a dual-task deficit on attention when participants overheard half a conversation. Experiment 1 manipulated conversation difficulty by asking participants to discuss either easy questions or difficult questions. The results showed that there was no difference in the dual-task cost depending on conversation difficulty. Experiment 2 showed a similar dual-task deficit of attention in both an easy and a difficult visual search task. Experiments 3 and 4 showed that in contrast to work using a dot tracking and choice reaction time task (Emberson et al., Psychol Sci 21:1383–1388, 2010), there was little deficit on MOT performance of hearing half a conversation, provided people heard the conversations in their native language. The results are discussed in terms of a resource-depleted account of attentional resources showing a fixed conversational-interference cost on attention.

A Vector Field Design Approach to Animated Transitions

Animated transitions can be effective in explaining and exploring a small number of visualizations where there are drastic changes in the scene over a short interval of time. This is especially true if data elements cannot be visually distinguished by other means. Current research in animated transitions has mainly focused on linear transitions (all elements follow straight line paths) or enhancing coordinated motion through bundling of linear trajectories. In this paper, we introduce animated transition design, a technique to build smooth, non-linear transitions for clustered data with either minimal or no user involvement. The technique is flexible and simple to implement, and has the additional advantage that it explicitly enhances coordinated motion and can avoid crowding, which are both important factors to support object tracking in a scene. We investigate its usability, provide preliminary evidence for the effectiveness of this technique through metric evaluations and user study and discuss limitations and future directions.

Chasing Animals With Split Attention: Are Animals Prioritized in Visual Tracking?

Some evolutionary psychologists have hypothesized that animals have priority in human attention. That is, they should be detected and selected more efficiently than other types of objects, especially man-made ones. Such a priority mechanism should automatically deploy more attentional resources and dynamic monitoring toward animal stimuli than nonanimals. Consequently, we postulated that variations of the multiple object or identity tracking and multiple event monitoring tasks should be particularly suitable paradigms for addressing the animate monitoring hypothesis, given their dynamic properties and dependency on divided attention. We used images of animals and artifacts and found neither a substantial sign of improvement in tracking the positions associated with animal stimuli nor a significant distracting effect of animals. We also failed to observe a significant prioritization in orders of response for positions associated with animals. While we observed an advantage for animals in event monitoring, this appeared to be dependent on properties of the task, as confirmed in further experiments. Moreover, we observed a small but inconclusive advantage for animals in identity accuracy. Thus, under certain conditions, some bias toward animals could be observed, but the evidence was weak and inconclusive. To conclude, effect sizes were generally small and not conclusively in favor of the expected attentional bias for animals. We found moderate to strong evidence that images of animals do not improve positional tracking, do not act as more effective distractors, are not selected prior to artifacts in the response phase, and are not easier to monitor for changes in size.

Event monitoring: Can we detect more than one event at a time?

A prior study by Wu and Wolfe found that the capacity for event monitoring (e.g. did an item change its state?) is more limited than for classic multiple object tracking. That limited capacity, K, could arise from either of two situations. It could be that people can detect K events simultaneously or it could be that they can successfully detect just one event at a time while monitoring K out of a total of N items. In the three different experiments of the present study, observers were asked to monitor a set of moving objects while watching for two critical events occurring in that set. Observers' performance can be well described by a model that includes an ability to detect two changes at once. Our results suggest that the capacity for event monitoring is further limited when tracking an additional event, but within the monitored set, people can detect at least two events simultaneously.

Binding object features to locations: Does the "spatial congruency bias" update with object movement?

One of the fundamental challenges of visual cognition is how our visual systems combine information about an object's features with its spatial location. A recent phenomenon related to object-location binding, the "spatial congruency bias," revealed that two objects are more likely to be perceived as having the same identity or features if they appear in the same spatial location, versus if the second object appears in a different location. The spatial congruency bias suggests that irrelevant location information is automatically encoded with and bound to other object properties, biasing perceptual judgments. Here we further explored this new phenomenon and its role in object-location binding by asking what happens when an object moves to a new location: Is the spatial congruency bias sensitive to spatiotemporal contiguity cues, or does it remain linked to the original object location? Across four experiments, we found that the spatial congruency bias remained strongly linked to the original object location. However, under certain circumstances-for instance, when the first object paused and remained visible for a brief time after the movement-the congruency bias was found at both the original location and the updated location. These data suggest that the spatial congruency bias is based more on low-level visual information than on spatiotemporal contiguity cues, and reflects a type of object-location binding that is primarily tied to the original object location and that may only update to the object's new location if there is time for the features to be re-encoded and rebound following the movement.

Cortical Circuit for Binding Object Identity and Location During Multiple-Object Tracking

Sustained multifocal attention for moving targets requires binding object identities with their locations. The brain mechanisms of identity-location binding during attentive tracking have remained unresolved. In 2 functional magnetic resonance imaging experiments, we measured participants’ hemodynamic activity during attentive tracking of multiple objects with equivalent (multiple-object tracking) versus distinct (multiple identity tracking, MIT) identities. Task load was manipulated parametrically. Both tasks activated large frontoparietal circuits. MIT led to significantly increased activity in frontoparietal and temporal systems subserving object recognition and working memory. These effects were replicated when eye movements were prohibited. MIT was associated with significantly increased functional connectivity between lateral temporal and frontal and parietal regions. We propose that coordinated activity of this network subserves identity-location binding during attentive tracking.

Hue distinctiveness overrides category in determining performance in multiple object tracking

The visual distinctiveness between targets and distractors can significantly facilitate performance in multiple object tracking (MOT), in which color is a feature that has been commonly used. However, the processing of color can be more than "visual." Color is continuous in chromaticity, while it is commonly grouped into discrete categories (e.g., red, green). Evidence from color perception suggested that color categories may have a unique role in visual tasks independent of its chromatic appearance. Previous MOT studies have not examined the effect of chromatic and categorical distinctiveness on tracking separately. The current study aimed to reveal how chromatic (hue) and categorical distinctiveness of color between the targets and distractors affects tracking performance. With four experiments, we showed that tracking performance was largely facilitated by the increasing hue distance between the target set and the distractor set, suggesting that perceptual grouping was formed based on hue distinctiveness to aid tracking. However, we found no color categorical effect, because tracking performance was not significantly different when the targets and distractors were from the same or different categories. It was concluded that the chromatic distinctiveness of color overrides category in determining tracking performance, suggesting a dominant role of perceptual feature in MOT.

Comparing eye movements during position tracking and identity tracking: No evidence for separate systems

There is an ongoing debate as to whether people track multiple moving objects in a serial fashion or with a parallel mechanism. One recent study compared eye movements when observers tracked identical objects (Multiple Object Tracking-MOT task) versus when they tracked the identities of different objects (Multiple Identity Tracking-MIT task). Distinct eye-movement patterns were found and attributed to two separate tracking systems. However, the same results could be caused by differences in the stimuli viewed during tracking. In the present study, object identities in the MIT task were invisible during tracking, so observers performed MOT and MIT tasks with identical stimuli. Observer were able to track either position and identity depending on the task. There was no difference in eye movements between position tracking and identity tracking. This result suggests that, while observers can use different eye-movement strategies in MOT and MIT, it is not necessary.

The semantic category-based grouping in the Multiple Identity Tracking task

In the Multiple Identity Tracking (MIT) task, categorical distinctions between targets and distractors have been found to facilitate tracking (Wei, Zhang, Lyu, & Li in Frontiers in Psychology, 7, 589, 2016). The purpose of this study was to further investigate the reasons for the facilitation effect, through six experiments. The results of Experiments 1-3 excluded the potential explanations of visual distinctiveness, attentional distribution strategy, and a working memory mechanism, respectively. When objects' visual information was preserved and categorical information was removed, the facilitation effect disappeared, suggesting that the visual distinctiveness between targets and distractors was not the main reason for the facilitation effect. Moreover, the facilitation effect was not the result of strategically shifting the attentional distribution, because the targets received more attention than the distractors in all conditions. Additionally, the facilitation effect did not come about because the identities of targets were encoded and stored in visual working memory to assist in the recovery from tracking errors; when working memory was disturbed by the object identities changing during tracking, the facilitation effect still existed. Experiments 4 and 5 showed that observers grouped targets together and segregated them from distractors on the basis of their categorical information. By doing this, observers could largely avoid distractor interference with tracking and improve tracking performance. Finally, Experiment 6 indicated that category-based grouping is not an automatic, but a goal-directed and effortful, strategy. In summary, the present findings show that a semantic category-based target-grouping mechanism exists in the MIT task, which is likely to be the major reason for the tracking facilitation effect.

Brain activation of semantic category-based grouping in multiple identity tracking task

Using Multiple Identity Tracking task and the functional magnetic resonance imaging (fMRI) technology, the present study aimed to isolate and visualize the functional anatomy of neural systems involved in the semantic category-based grouping process. Three experiment conditions were selected and compared: the category-based targets grouping (TG) condition, the targets-distractors grouping (TDG) condition and the homogenous condition. In the TG condition, observers could utilize the categorical distinction between targets and distractors, to construct a uniform presentation of targets, that is, to form a group of the targets to facilitate tracking. In the TDG condition, half the targets and half the distractors belonged to the same category. Observers had to inhibit the grouping of targets and distractors in one category to complete tracking. In the homogenous condition, where targets and distractors consisted of the same objects, no grouping could be formed. The "TG-Homogenous" contrast (p<0.01) revealed the activation of the left fusiform and the pars triangularis of inferior frontal gyrus (IFG). The "TG-TDG" contrast only revealed the activation of the left anterior cingulate gyrus (ACC). The fusiform and IFG pars triangularis might participate in the representation of semantic knowledge, IFG pars triangularis might relate intensely with the classification of semantic categories. The ACC might be responsible for the initiation and maintenance of grouping representation.

Angry faces are tracked more easily than neutral faces during multiple identity tracking

We investigated whether and how emotional facial expressions affect sustained attention in face tracking. In a multiple-identity and object tracking paradigm, participants tracked multiple target faces that continuously moved around together with several distractor faces, and subsequently reported where each target face had moved to. The emotional expression (angry, happy, and neutral) of the target and distractor faces was manipulated. Tracking performance was better when the target faces were angry rather than neutral, whereas angry distractor faces did not affect tracking. The effect persisted when the angry faces were presented upside-down and when surface features of the faces were irrelevant to the ongoing task. There was only suggestive and weak evidence for a facilitatory effect of happy targets and a distraction effect of happy distractors in comparison to neutral faces. The results show that angry expressions on the target faces can facilitate sustained attention on the targets via increased vigilance, yet this effect likely depends on both emotional information and visual features of the angry faces.

Predicting rhesus monkey eye movements during natural-image search

There are three prominent factors that can predict human visual-search behavior in natural scenes: the distinctiveness of a location (salience), similarity to the target (relevance), and features of the environment that predict where the object might be (context). We do not currently know how well these factors are able to predict macaque visual search, which matters because it is arguably the most popular model for asking how the brain controls eye movements. Here we trained monkeys to perform the pedestrian search task previously used for human subjects. Salience, relevance, and context models were all predictive of monkey eye fixations and jointly about as precise as for humans. We attempted to disrupt the influence of scene context on search by testing the monkeys with an inverted set of the same images. Surprisingly, the monkeys were able to locate the pedestrian at a rate similar to that for upright images. The best predictions of monkey fixations in searching inverted images were obtained by rotating the results of the model predictions for the original image. The fact that the same models can predict human and monkey search behavior suggests that the monkey can be used as a good model for understanding how the human brain enables natural-scene search.

Multiple event monitoring

Suppose you were monitoring a group of people in order to determine if anyone of them did something suspicious (e.g., putting down a bag) or if any two interacted in a suspicious manner (e.g., trading bags). How large a group could you monitor successfully? This paper reports on six experiments in which observers monitor a group of entities, watching for an event. Whether the event was performed by a single entity or was an interaction between a pair, the capacity for event monitoring was two to three items. This was lower than the multiple object tracking capacity for the same stimuli (approximately six items). Capacity was essentially the same whether entities were identical circles or unique cartoon animals; nor was capacity changed by an added requirement to identify the entities involved in an event. Event monitoring appears to be related to, but not identical to, multiple object tracking.

On the Theorem of Correspondence

In a recent paper, Mammen (Integrative Psychological and Behavioral Science, 50, 196-233, 2016a) brought novel arguments into the discussion concerning the importance of being able to single out and track objects through space and time. Mammen offered a formal account of two basic, yet distinct, ways in which we as human beings encounter objects in the real world, that is, sense and choice categories. In this paper I discuss aspects of his theory and in particular the Theorem of Correspondence. I shall attempt to argue that Mammen's formal account is indeed a novel and powerful analytical generic tool allowing us to see the important relevance in different domains of being able to establish choice categories. Meanwhile, I will attempt to show that evidence from the so-called multiple object tracking studies -- even though these use highly artificial stimuli -- provide compelling evidence in support of Mammen's formal account.