1
|
Kamkar S, Ghezloo F, Moghaddam HA, Borji A, Lashgari R. Multiple-target tracking in human and machine vision. PLoS Comput Biol 2020; 16:e1007698. [PMID: 32271746 PMCID: PMC7144962 DOI: 10.1371/journal.pcbi.1007698] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
Abstract
Humans are able to track multiple objects at any given time in their daily activities—for example, we can drive a car while monitoring obstacles, pedestrians, and other vehicles. Several past studies have examined how humans track targets simultaneously and what underlying behavioral and neural mechanisms they use. At the same time, computer-vision researchers have proposed different algorithms to track multiple targets automatically. These algorithms are useful for video surveillance, team-sport analysis, video analysis, video summarization, and human–computer interaction. Although there are several efficient biologically inspired algorithms in artificial intelligence, the human multiple-target tracking (MTT) ability is rarely imitated in computer-vision algorithms. In this paper, we review MTT studies in neuroscience and biologically inspired MTT methods in computer vision and discuss the ways in which they can be seen as complementary. Multiple-target tracking (MTT) is a challenging task vital for both a human’s daily life and for many artificial intelligent systems, such as those used for urban traffic control. Neuroscientists are interested in discovering the underlying neural mechanisms that successfully exploit cognitive resources, e.g., spatial attention or memory, during MTT. Computer-vision specialists aim to develop powerful MTT algorithms based on advanced models or data-driven computational methods. In this paper, we review MTT studies from both communities and discuss how findings from cognitive studies can inspire developers to construct higher performing MTT algorithms. Moreover, some directions have been proposed through which MTT algorithms could raise new questions in the cognitive science domain, and answering them can shed light on neural processes underlying MTT.
Collapse
Affiliation(s)
- Shiva Kamkar
- Machine Vision and Medical Image Processing Laboratory, Faculty of Electrical and Computer Engineering, K. N. Toosi University of Technology, Tehran, Iran
- Brain Engineering Research Center, Institute for Research in Fundamental Sciences (IPM), Tehran, Iran
| | - Fatemeh Ghezloo
- Brain Engineering Research Center, Institute for Research in Fundamental Sciences (IPM), Tehran, Iran
| | - Hamid Abrishami Moghaddam
- Machine Vision and Medical Image Processing Laboratory, Faculty of Electrical and Computer Engineering, K. N. Toosi University of Technology, Tehran, Iran
- * E-mail: (RL); (HAM)
| | - Ali Borji
- HCL America, Manhattan, New York City, United States of America
| | - Reza Lashgari
- Brain Engineering Research Center, Institute for Research in Fundamental Sciences (IPM), Tehran, Iran
- * E-mail: (RL); (HAM)
| |
Collapse
|
2
|
Wei L, Zhang X, Li Z, Hu B, Li X. The Global Properties of Objects Play the Main Role in Facilitating Multiple Object Tracking Performance. Front Psychol 2019; 10:924. [PMID: 31105626 PMCID: PMC6499005 DOI: 10.3389/fpsyg.2019.00924] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2019] [Accepted: 04/08/2019] [Indexed: 11/14/2022] Open
Abstract
Previous research has revealed the uniqueness-facilitation effect in the multiple object tracking (MOT) task: simple distinct identities and surface features of moving targets could facilitate attentional tracking. By adapting compound stimuli, the present study investigated whether the global or local properties played the main role in the uniqueness-facilitation effect in the MOT task. The uniqueness of local properties, of global properties or of both local and global properties were considered. Observers’ tracking performance in alternative conditions were compared with that in the homogeneous condition wherein all stimuli have identical local and global properties. Results from two experiments suggest that the global properties played the key role in facilitating tracking. The distinctiveness of local properties can also facilitate tracking with global properties being homogeneous. However, when the stimuli’s global properties are distinct from each other—whether the local properties being unique or not—observers’ tracking performance can achieve the same level as that in the unitary-uniqueness condition wherein the moving objects were distinct unitary letters. These results revealed a global superiority effect in the MOT task. Finally, the facilitation effects of the global and local properties might depend on the stimulus sparsity. When the compound stimuli had fewer local elements, the uniqueness facilitation effect on tracking decreased.
Collapse
Affiliation(s)
- Liuqing Wei
- Department of Psychology, Faculty of Education, Hubei University, Wuhan, China.,Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education, Faculty of Psychology, Beijing Normal University, Beijing, China
| | - Xuemin Zhang
- Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education, Faculty of Psychology, Beijing Normal University, Beijing, China.,State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.,Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing, China
| | - Zhen Li
- eMetric, LLC., San Antonio, TX, United States
| | - Bin Hu
- Gansu Provincial Key Laboratory of Wearable Computing, School of Information Science and Engineering, Lanzhou University, Lanzhou, China.,CAS Center for Excellence in Brain Science and Intelligence Technology, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, Shanghai, China.,Beijing Institute for Brain Disorders, Capital Medical University, Beijing, China
| | - Xiaowei Li
- Gansu Provincial Key Laboratory of Wearable Computing, School of Information Science and Engineering, Lanzhou University, Lanzhou, China
| |
Collapse
|
3
|
Wang C, Hu L, Talhelm T, Zhang X. The effects of colour complexity and similarity on multiple object tracking performance. Q J Exp Psychol (Hove) 2018; 72:1903-1912. [DOI: 10.1177/1747021818817388] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Surface features can be used during multiple object tracking (MOT). Previous studies suggested that surface features might be stored in visual working memory to assist object tracking, and attentive tracking and visual working memory share common attentional resources. However, it is still unknown whether features of both the target and distractor sets will be stored, or features of the target and distractor sets are processed differently. Moreover, how feature distinctiveness and similarity between the target and distractor sets affect tracking and allocation of attentional resources are still not clear. First, we manipulated the colour complexity of the target set (CT) and the colour complexity of the distractor set (CD), respectively, in two experiments, where colours of the target and distractor sets were always distinct, to test their effects on tracking performance. If features of the target and distractor sets are stored, manipulating feature complexity of the target and distractor sets would significantly affect tracking performance. Second, this study tested whether tracking performance was affected by different levels of distinctiveness between the target and distractor sets (DTD) and explored how distinctiveness affected tracking and allocation of attentional resources. Results showed that DTD and CT significantly affect tracking performance and allocation of attentional resources, but not CD. These results indicated that when targets and distractors have distinct features, only the surface features of the targets are maintained in visual working memory. And when targets have the same colour with the distractors, they are more difficult and consume more attentional resources to track.
Collapse
Affiliation(s)
- Chundi Wang
- Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education, Faculty of Psychology, Beijing Normal University, Beijing, China
| | - Luming Hu
- Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education, Faculty of Psychology, Beijing Normal University, Beijing, China
| | - Thomas Talhelm
- Booth School of Business, The University of Chicago, Chicago, IL, USA
| | - Xuemin Zhang
- Beijing Key Laboratory of Applied Experimental Psychology, National Demonstration Center for Experimental Psychology Education, Faculty of Psychology, Beijing Normal University, Beijing, China
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing, China
| |
Collapse
|
4
|
Wang C, Hu L, Hu S, Xu Y, Zhang X. Functional specialization for feature-based and symmetry-based groupings in multiple object tracking. Cortex 2018; 108:265-275. [DOI: 10.1016/j.cortex.2018.09.005] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2018] [Revised: 05/12/2018] [Accepted: 09/05/2018] [Indexed: 10/28/2022]
|
5
|
Hue distinctiveness overrides category in determining performance in multiple object tracking. Atten Percept Psychophys 2017; 80:374-386. [PMID: 29238912 DOI: 10.3758/s13414-017-1466-7] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The visual distinctiveness between targets and distractors can significantly facilitate performance in multiple object tracking (MOT), in which color is a feature that has been commonly used. However, the processing of color can be more than "visual." Color is continuous in chromaticity, while it is commonly grouped into discrete categories (e.g., red, green). Evidence from color perception suggested that color categories may have a unique role in visual tasks independent of its chromatic appearance. Previous MOT studies have not examined the effect of chromatic and categorical distinctiveness on tracking separately. The current study aimed to reveal how chromatic (hue) and categorical distinctiveness of color between the targets and distractors affects tracking performance. With four experiments, we showed that tracking performance was largely facilitated by the increasing hue distance between the target set and the distractor set, suggesting that perceptual grouping was formed based on hue distinctiveness to aid tracking. However, we found no color categorical effect, because tracking performance was not significantly different when the targets and distractors were from the same or different categories. It was concluded that the chromatic distinctiveness of color overrides category in determining tracking performance, suggesting a dominant role of perceptual feature in MOT.
Collapse
|
6
|
Abstract
In the Multiple Identity Tracking (MIT) task, categorical distinctions between targets and distractors have been found to facilitate tracking (Wei, Zhang, Lyu, & Li in Frontiers in Psychology, 7, 589, 2016). The purpose of this study was to further investigate the reasons for the facilitation effect, through six experiments. The results of Experiments 1-3 excluded the potential explanations of visual distinctiveness, attentional distribution strategy, and a working memory mechanism, respectively. When objects' visual information was preserved and categorical information was removed, the facilitation effect disappeared, suggesting that the visual distinctiveness between targets and distractors was not the main reason for the facilitation effect. Moreover, the facilitation effect was not the result of strategically shifting the attentional distribution, because the targets received more attention than the distractors in all conditions. Additionally, the facilitation effect did not come about because the identities of targets were encoded and stored in visual working memory to assist in the recovery from tracking errors; when working memory was disturbed by the object identities changing during tracking, the facilitation effect still existed. Experiments 4 and 5 showed that observers grouped targets together and segregated them from distractors on the basis of their categorical information. By doing this, observers could largely avoid distractor interference with tracking and improve tracking performance. Finally, Experiment 6 indicated that category-based grouping is not an automatic, but a goal-directed and effortful, strategy. In summary, the present findings show that a semantic category-based target-grouping mechanism exists in the MIT task, which is likely to be the major reason for the tracking facilitation effect.
Collapse
|
7
|
Wei L, Zhang X, Lyu C, Hu S, Li Z. Brain activation of semantic category-based grouping in multiple identity tracking task. PLoS One 2017; 12:e0177709. [PMID: 28505166 PMCID: PMC5432174 DOI: 10.1371/journal.pone.0177709] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2016] [Accepted: 05/02/2017] [Indexed: 11/19/2022] Open
Abstract
Using Multiple Identity Tracking task and the functional magnetic resonance imaging (fMRI) technology, the present study aimed to isolate and visualize the functional anatomy of neural systems involved in the semantic category-based grouping process. Three experiment conditions were selected and compared: the category-based targets grouping (TG) condition, the targets-distractors grouping (TDG) condition and the homogenous condition. In the TG condition, observers could utilize the categorical distinction between targets and distractors, to construct a uniform presentation of targets, that is, to form a group of the targets to facilitate tracking. In the TDG condition, half the targets and half the distractors belonged to the same category. Observers had to inhibit the grouping of targets and distractors in one category to complete tracking. In the homogenous condition, where targets and distractors consisted of the same objects, no grouping could be formed. The "TG-Homogenous" contrast (p<0.01) revealed the activation of the left fusiform and the pars triangularis of inferior frontal gyrus (IFG). The "TG-TDG" contrast only revealed the activation of the left anterior cingulate gyrus (ACC). The fusiform and IFG pars triangularis might participate in the representation of semantic knowledge, IFG pars triangularis might relate intensely with the classification of semantic categories. The ACC might be responsible for the initiation and maintenance of grouping representation.
Collapse
Affiliation(s)
- Liuqing Wei
- School of Psychology, Beijing Key Lab of Applied Experimental Psychology, Beijing Normal University, Beijing, China
| | - Xuemin Zhang
- School of Psychology, Beijing Key Lab of Applied Experimental Psychology, Beijing Normal University, Beijing, China
- State Key Lab of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing, China
- * E-mail:
| | - Chuang Lyu
- School of Psychology, Beijing Key Lab of Applied Experimental Psychology, Beijing Normal University, Beijing, China
| | - Siyuan Hu
- School of Psychology, Beijing Key Lab of Applied Experimental Psychology, Beijing Normal University, Beijing, China
- State Key Lab of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Center for Collaboration and Innovation in Brain and Learning Sciences, Beijing Normal University, Beijing, China
| | - Zhen Li
- eMetric, LLC, San Antonio, Texas, United States of America
| |
Collapse
|