Functional specialization for feature-based and symmetry-based groupings in multiple object tracking

Categorical encoding of moving colors during location tracking

Categorical perception (CP) describes our tendency to perceive the visual world in a categorical manner, suggesting that high-level cognition may affect perception. While most studies are conducted in static visual scenes, Sun and colleagues found CP effects of color in multiple object tracking (MOT). This study used functional magnetic resonance imaging to investigate the neural mechanism behind the categorical effects of color in MOT. Categorical effects were associated with activities in a broad range of brain regions, including both the ventral (V4, middle temporal gyrus) and dorsal pathways (MT + /V5, inferior parietal lobule) of feature processing, as well as frontal regions (middle frontal gyrus, medial superior frontal gyrus). We proposed that these regions are hierarchically organized and responsible for distinct functions. The color-selective V4 encodes color categories, making cross-category colors more discriminable than within-category colors. Meanwhile, the language and/or semantic regions encode the verbal information of the colors. Both visual and nonvisual codes of color categories then modulate the activities of motion-sensitive MT + areas and frontal areas responsible for attentional processes.

Inattentional blindness: Attentional set for efficient task success

Inattentional blindness is the failure to notice an unexpected object in plain sight when attention is otherwise engaged. We investigated what determines observers' attentional set in a dynamic-counting inattentional blindness paradigm, when task instructions and visual distinctiveness of task-relevant objects were either congruent or in opposition. In seven experiments, observers counted bounces by task-relevant objects, with the instruction either to count-by-shape (squares, diamonds, crosses) or count-by-colour (blue, purple). To manipulate visual distinctiveness, we varied the extent to which task-relevant and task-irrelevant objects looked different on two dimensions: shape and colour. When colour better distinguished task-relevant from task-irrelevant objects, observers-even if instructed count-by-shape-reported an unexpected object that matched the colour of task-relevant objects. Crucially, when instructed count-by-colour, but shape better distinguished task-relevant from task-irrelevant objects, observers reported an unexpected object that matched the shape of task-relevant objects. We conclude that observers set their attention to promote efficient task performance.

How do humans group non-rigid objects in multiple object tracking?: Evidence from grouping by self-rotation

Previous studies on perceptual grouping found that people can use spatiotemporal and featural information to group spatially separated rigid objects into a unit while tracking moving objects. However, few studies have tested the role of objects' self-motion information in perceptual grouping, although it is of great significance to the motion perception in the three-dimensional space. In natural environments, objects always move in translation and rotation at the same time. The self-rotation of the objects seriously destroys objects' rigidity and topology, creates conflicting movement signals and results in crowding effects. Thus, this study sought to examine the specific role played by self-rotation information on grouping spatially separated non-rigid objects through a modified multiple object tracking (MOT) paradigm with self-rotating objects. Experiment 1 found that people could use self-rotation information to group spatially separated non-rigid objects, even though this information was deleterious for attentive tracking and irrelevant to the task requirements, and people seemed to use it strategically rather than automatically. Experiment 2 provided stronger evidence that this grouping advantage did come from the self-rotation per se rather than surface-level cues arising from self-rotation (e.g. similar 2D motion signals and common shapes). Experiment 3 changed the stimuli to more natural 3D cubes to strengthen the impression of self-rotation and again found that self-rotation improved grouping. Finally, Experiment 4 demonstrated that grouping by self-rotation and grouping by changing shape were statistically comparable but additive, suggesting that they were two different sources of the object information. Thus, grouping by self-rotation mainly benefited from the perceptual differences in motion flow fields rather than in deformation. Overall, this study is the first attempt to identify self-motion as a new feature that people can use to group objects in dynamic scenes and shed light on debates about what entities/units we group and what kinds of information about a target we process while tracking objects.

Distinguishing the neural mechanism of attentional control and working memory in feature-based attentive tracking

Surface features are an important component in attentive tracking. However, the neural mechanisms underlying how features affect attentive tracking remain unknown. The present fMRI study addressed this issue by manipulating the intragroup feature complexity and intergroup feature similarity. In particular, this study distinguished the different neural mechanisms of intragroup feature complexity and intergroup feature similarity by investigating the roles of attentional control and working memory in dynamic feature-based attentive tracking. Behavioral and neuroimaging evidence showed that when targets are distinct from distractors, the intragroup feature complexity of the targets, rather than that of the distractors, mainly increases the visual working memory load and significantly activates the frontoparietal cortical circuit. Thus, the involvement of working memory in feature-based attentive tracking is modulated by goal-directed attention control. In addition, when targets are similar to distractors, the intergroup feature similarity (i.e., target-distractor similarity) mainly affects the allocation of attention. Specifically, target-distractor similarity affects the goal-directed attention toward the targets in a stimulus-driven way and induces an interaction between the ventral and dorsal attention networks.

Multiple-target tracking in human and machine vision

Humans are able to track multiple objects at any given time in their daily activities—for example, we can drive a car while monitoring obstacles, pedestrians, and other vehicles. Several past studies have examined how humans track targets simultaneously and what underlying behavioral and neural mechanisms they use. At the same time, computer-vision researchers have proposed different algorithms to track multiple targets automatically. These algorithms are useful for video surveillance, team-sport analysis, video analysis, video summarization, and human–computer interaction. Although there are several efficient biologically inspired algorithms in artificial intelligence, the human multiple-target tracking (MTT) ability is rarely imitated in computer-vision algorithms. In this paper, we review MTT studies in neuroscience and biologically inspired MTT methods in computer vision and discuss the ways in which they can be seen as complementary.

Multiple-target tracking (MTT) is a challenging task vital for both a human’s daily life and for many artificial intelligent systems, such as those used for urban traffic control. Neuroscientists are interested in discovering the underlying neural mechanisms that successfully exploit cognitive resources, e.g., spatial attention or memory, during MTT. Computer-vision specialists aim to develop powerful MTT algorithms based on advanced models or data-driven computational methods. In this paper, we review MTT studies from both communities and discuss how findings from cognitive studies can inspire developers to construct higher performing MTT algorithms. Moreover, some directions have been proposed through which MTT algorithms could raise new questions in the cognitive science domain, and answering them can shed light on neural processes underlying MTT.

The effects of colour complexity and similarity on multiple object tracking performance

Surface features can be used during multiple object tracking (MOT). Previous studies suggested that surface features might be stored in visual working memory to assist object tracking, and attentive tracking and visual working memory share common attentional resources. However, it is still unknown whether features of both the target and distractor sets will be stored, or features of the target and distractor sets are processed differently. Moreover, how feature distinctiveness and similarity between the target and distractor sets affect tracking and allocation of attentional resources are still not clear. First, we manipulated the colour complexity of the target set (CT) and the colour complexity of the distractor set (CD), respectively, in two experiments, where colours of the target and distractor sets were always distinct, to test their effects on tracking performance. If features of the target and distractor sets are stored, manipulating feature complexity of the target and distractor sets would significantly affect tracking performance. Second, this study tested whether tracking performance was affected by different levels of distinctiveness between the target and distractor sets (DTD) and explored how distinctiveness affected tracking and allocation of attentional resources. Results showed that DTD and CT significantly affect tracking performance and allocation of attentional resources, but not CD. These results indicated that when targets and distractors have distinct features, only the surface features of the targets are maintained in visual working memory. And when targets have the same colour with the distractors, they are more difficult and consume more attentional resources to track.