Hue distinctiveness overrides category in determining performance in multiple object tracking

Categorical encoding of moving colors during location tracking

Categorical perception (CP) describes our tendency to perceive the visual world in a categorical manner, suggesting that high-level cognition may affect perception. While most studies are conducted in static visual scenes, Sun and colleagues found CP effects of color in multiple object tracking (MOT). This study used functional magnetic resonance imaging to investigate the neural mechanism behind the categorical effects of color in MOT. Categorical effects were associated with activities in a broad range of brain regions, including both the ventral (V4, middle temporal gyrus) and dorsal pathways (MT + /V5, inferior parietal lobule) of feature processing, as well as frontal regions (middle frontal gyrus, medial superior frontal gyrus). We proposed that these regions are hierarchically organized and responsible for distinct functions. The color-selective V4 encodes color categories, making cross-category colors more discriminable than within-category colors. Meanwhile, the language and/or semantic regions encode the verbal information of the colors. Both visual and nonvisual codes of color categories then modulate the activities of motion-sensitive MT + areas and frontal areas responsible for attentional processes.

Inattentional blindness: Attentional set for efficient task success

Inattentional blindness is the failure to notice an unexpected object in plain sight when attention is otherwise engaged. We investigated what determines observers' attentional set in a dynamic-counting inattentional blindness paradigm, when task instructions and visual distinctiveness of task-relevant objects were either congruent or in opposition. In seven experiments, observers counted bounces by task-relevant objects, with the instruction either to count-by-shape (squares, diamonds, crosses) or count-by-colour (blue, purple). To manipulate visual distinctiveness, we varied the extent to which task-relevant and task-irrelevant objects looked different on two dimensions: shape and colour. When colour better distinguished task-relevant from task-irrelevant objects, observers-even if instructed count-by-shape-reported an unexpected object that matched the colour of task-relevant objects. Crucially, when instructed count-by-colour, but shape better distinguished task-relevant from task-irrelevant objects, observers reported an unexpected object that matched the shape of task-relevant objects. We conclude that observers set their attention to promote efficient task performance.

Language Modulates Categorical Effects of Moving Color Objects

Categorical perception (CP) of color claims that colors from different linguistic categories are discriminated more easily than those from the same category, suggesting that language may interact with visual perception. However, controversy remains regarding whether CP effects derive from language. Recently, CP effects were found in a dynamic paradigm named multiple object tracking (MOT). Here, we investigated whether this kind of CP is dependent on online use of language. We found that the CP effects are reduced by verbal interference when the participants were required to memorize color words during tracking (Experiment 2) but not when the interference stimuli were digits (Experiment 1). Our study suggested that the CP effects in tracking depend partly on online use of language and that the difficulty of verbal interference tasks influences the capability of disrupting CP.

Multiple-target tracking in human and machine vision

Humans are able to track multiple objects at any given time in their daily activities—for example, we can drive a car while monitoring obstacles, pedestrians, and other vehicles. Several past studies have examined how humans track targets simultaneously and what underlying behavioral and neural mechanisms they use. At the same time, computer-vision researchers have proposed different algorithms to track multiple targets automatically. These algorithms are useful for video surveillance, team-sport analysis, video analysis, video summarization, and human–computer interaction. Although there are several efficient biologically inspired algorithms in artificial intelligence, the human multiple-target tracking (MTT) ability is rarely imitated in computer-vision algorithms. In this paper, we review MTT studies in neuroscience and biologically inspired MTT methods in computer vision and discuss the ways in which they can be seen as complementary.

Multiple-target tracking (MTT) is a challenging task vital for both a human’s daily life and for many artificial intelligent systems, such as those used for urban traffic control. Neuroscientists are interested in discovering the underlying neural mechanisms that successfully exploit cognitive resources, e.g., spatial attention or memory, during MTT. Computer-vision specialists aim to develop powerful MTT algorithms based on advanced models or data-driven computational methods. In this paper, we review MTT studies from both communities and discuss how findings from cognitive studies can inspire developers to construct higher performing MTT algorithms. Moreover, some directions have been proposed through which MTT algorithms could raise new questions in the cognitive science domain, and answering them can shed light on neural processes underlying MTT.

Tracking within-category colors is easier: Color categories modulate location processing in a dynamic visual task

The categorical perception (CP) of color describes the phenomenon that colors across categories (e.g., blue and green) are more discriminable than within-category colors (e.g., green) even when the perceptual distance is controlled. While most studies are conducted in a static visual scene, the current study investigated the effect of color categories when tracking multiple colored objects within two experiments. The targets or distractors were either from the same color category or from two different categories, and the perceptual distance was controlled across all conditions. We found that location tracking was facilitated when the targets or the distractors shared the same color category in the tracking task, which required location tracking and color memorization at the same time, compared with when they were from two different categories (Experiment 1). By contrast, when location tracking was the sole demand and no explicit color processing was required, the target CP effects, but not the distractor CP effects, persisted (Experiment 2). We conclude that color categories can dynamically modulate preattention visual processing when color working memory is involved, but that attention is necessary to produce CP effects when color is irrelevant to the task.

The effects of colour complexity and similarity on multiple object tracking performance

Surface features can be used during multiple object tracking (MOT). Previous studies suggested that surface features might be stored in visual working memory to assist object tracking, and attentive tracking and visual working memory share common attentional resources. However, it is still unknown whether features of both the target and distractor sets will be stored, or features of the target and distractor sets are processed differently. Moreover, how feature distinctiveness and similarity between the target and distractor sets affect tracking and allocation of attentional resources are still not clear. First, we manipulated the colour complexity of the target set (CT) and the colour complexity of the distractor set (CD), respectively, in two experiments, where colours of the target and distractor sets were always distinct, to test their effects on tracking performance. If features of the target and distractor sets are stored, manipulating feature complexity of the target and distractor sets would significantly affect tracking performance. Second, this study tested whether tracking performance was affected by different levels of distinctiveness between the target and distractor sets (DTD) and explored how distinctiveness affected tracking and allocation of attentional resources. Results showed that DTD and CT significantly affect tracking performance and allocation of attentional resources, but not CD. These results indicated that when targets and distractors have distinct features, only the surface features of the targets are maintained in visual working memory. And when targets have the same colour with the distractors, they are more difficult and consume more attentional resources to track.