Greedy Batch-Based Minimum-Cost Flows for Tracking Multiple Objects

Effective Multi-Object Tracking via Global Object Models and Object Constraint Learning

Effective multi-object tracking is still challenging due to the trade-off between tracking accuracy and speed. Because the recent multi-object tracking (MOT) methods leverage object appearance and motion models so as to associate detections between consecutive frames, the key for effective multi-object tracking is to reduce the computational complexity of learning both models. To this end, this work proposes global appearance and motion models to discriminate multiple objects instead of learning local object-specific models. In concrete detail, it learns a global appearance model using contrastive learning between object appearances. In addition, we learn a global relation motion model using relative motion learning between objects. Moreover, this paper proposes object constraint learning for improving tracking efficiency. This study considers the discriminability of the models as a constraint, and learns both models when inconsistency with the constraint occurs. Therefore, object constraint learning differs from the conventional online learning for multi-object tracking which updates learnable parameters per frame. This work incorporates global models and object constraint learning into the confidence-based association method, and compare our tracker with the state-of-the-art methods on public available MOT Challenge datasets. As a result, we achieve 64.5% MOTA (multi-object tracking accuracy) and 6.54 Hz tracking speed on the MOT16 test dataset. The comparison results show that our methods can contribute to improve tracking accuracy and tracking speed together.

An Object Point Set Inductive Tracker for Multi-Object Tracking and Segmentation

Multi-object tracking and segmentation (MOTS) is a derivative task of multi-object tracking (MOT). The new setting encourages the learning of more discriminative high-quality embeddings. In this paper, we focus on the problem of exploring the relationship between the segmenter and the tracker, and propose an efficient Object Point set Inductive Tracker (OPITrack) based on it. First, we discover that after a single attention layer, the high-dimensional, key point embedding will show feature averaging. To alleviate this phenomenon, we propose an embedding generalization training strategy for sparse training and dense testing. This strategy allows the network to increase randomness in training and encourages the tracker to learn more discriminative features. In addition, to learn the desired embedding space, we propose a general Trip-hard sample augmentation loss. The loss uses patches that are not distinguishable by the segmenter to join the feature learning and force the embedding network to learn the difference between false positives and true positives. Our method was validated on two MOTS benchmark datasets and achieved promising results. In addition, our OPITrack can achieve better performance for the raw model while costing less video memory (VRAM) at training time.

Bayesian Multi-Targets Strategy to Track Apis mellifera Movements at Colony Level

Simple Summary

The number of honey bee, Apis mellifera L., colonies has reduced around the globe, and one potential cause is their unintended exposure to sublethal stressors such as agricultural pesticides. The quantification of such effects at colony level is a very complex task due to the innumerable collective activities done by the individual within colonies. Here, we present a Bayesian and computational approach capable of tracking the movements of bees within colonies, which allows the comparison of the collective activities of colonies that received bees previously exposed to uncontaminated diets or to diets containing sublethal concentrations of an agricultural pesticide (a commercial formulation containing the synthetic fungicides thiophanate-methyl and chlorothalonil). Our Bayesian tracking technique proved successful and superior to comparable algorithms, allowing the estimation of dynamical parameters such as entropy and kinetic energy. Our efforts demonstrated that fungicide-contaminated colonies behaved differently from uncontaminated colonies, as the former exhibited anticipated collective activities in peripheral hive areas and had reduced swarm entropy and kinetic energies. Such findings may facilitate the electronic monitoring of potential unintended effects in social pollinators, at colony level, mediated by environmental stressors (e.g., pesticides, electromagnetic fields, noise, and light intensities) alone or in combination.

Abstract

Interactive movements of bees facilitate the division and organization of collective tasks, notably when they need to face internal or external environmental challenges. Here, we present a Bayesian and computational approach to track the movement of several honey bee, Apis mellifera, workers at colony level. We applied algorithms that combined tracking and Kernel Density Estimation (KDE), allowing measurements of entropy and Probability Distribution Function (PDF) of the motion of tracked organisms. We placed approximately 200 recently emerged and labeled bees inside an experimental colony, which consists of a mated queen, approximately 1000 bees, and a naturally occurring beehive background. Before release, labeled bees were fed for one hour with uncontaminated diets or diets containing a commercial mixture of synthetic fungicides (thiophanate-methyl and chlorothalonil). The colonies were filmed (12 min) at the 1st hour, 5th and 10th days after the bees’ release. Our results revealed that the algorithm tracked the labeled bees with great accuracy. Pesticide-contaminated colonies showed anticipated collective activities in peripheral hive areas, far from the brood area, and exhibited reduced swarm entropy and energy values when compared to uncontaminated colonies. Collectively, our approach opens novel possibilities to quantify and predict potential alterations mediated by pollutants (e.g., pesticides) at the bee colony-level.

Single-Image Dehazing via Compositional Adversarial Network

Single-image dehazing has been an important topic given the commonly occurred image degradation caused by adverse atmosphere aerosols. The key to haze removal relies on an accurate estimation of global air-light and the transmission map. Most existing methods estimate these two parameters using separate pipelines which reduces the efficiency and accumulates errors, thus leading to a suboptimal approximation, hurting the model interpretability, and degrading the performance. To address these issues, this article introduces a novel generative adversarial network (GAN) for single-image dehazing. The network consists of a novel compositional generator and a novel deeply supervised discriminator. The compositional generator is a densely connected network, which combines fine-scale and coarse-scale information. Benefiting from the new generator, our method can directly learn the physical parameters from data and recover clean images from hazy ones in an end-to-end manner. The proposed discriminator is deeply supervised, which enforces that the output of the generator to look similar to the clean images from low-level details to high-level structures. To the best of our knowledge, this is the first end-to-end generative adversarial model for image dehazing, which simultaneously outputs clean images, transmission maps, and air-lights. Extensive experiments show that our method remarkably outperforms the state-of-the-art methods. Furthermore, to facilitate future research, we create the HazeCOCO dataset which is currently the largest dataset for single-image dehazing.

Tracking-by-Counting: Using Network Flows on Crowd Density Maps for Tracking Multiple Targets

State-of-the-art multi-object tracking (MOT) methods follow the tracking-by-detection paradigm, where object trajectories are obtained by associating per-frame outputs of object detectors. In crowded scenes, however, detectors often fail to obtain accurate detections due to heavy occlusions and high crowd density. In this paper, we propose a new MOT paradigm, tracking-by-counting, tailored for crowded scenes. Using crowd density maps, we jointly model detection, counting, and tracking of multiple targets as a network flow program, which simultaneously finds the global optimal detections and trajectories of multiple targets over the whole video. This is in contrast to prior MOT methods that either ignore the crowd density and thus are prone to errors in crowded scenes, or rely on a suboptimal two-step process using heuristic density-aware point-tracks for matching targets. Our approach yields promising results on public benchmarks of various domains including people tracking, cell tracking, and fish tracking.

Variational Inference for 3-D Localization and Tracking of Multiple Targets Using Multiple Cameras

This paper proposes a novel unified framework to solve the 3-D localization and tracking problem that occurs multiple camera settings with overlapping views. The main challenge is to overcome the uncertainty of the back projection arising from the challenges of ground point detection in an environment that includes severe occlusions and the unknown heights of people. To tackle this challenge, we establish a Bayesian learning framework that maximizes a posterior over the trajectory assignments and 3-D positions for given detections from multiple cameras. To solve the Bayesian learning problem in a tractable form, we develop an expectation-maximization scheme based on the variation inference approximation, where the probability distributions are designed to follow Boltzmann distributions of seven terms that are induced from multicamera tracking settings. The experimental results show that the proposed method outperforms the state-of-the-art methods on the challenging multicamera data sets.

Interacting Tracklets for Multi-object Tracking

In this paper, we propose to exploit the interactions between non-associable tracklets to facilitate multi-object tracking. We introduce two types of tracklet interactions, close interaction and distant interaction. The close interaction imposes physical constraints between two temporally overlapping tracklets and more importantly, allows us to learn local classifiers to distinguish targets that are close to each other in the spatiotemporal domain. The distant interaction, on the other hand, accounts for the higher-order motion and appearance consistency between two temporally isolated tracklets. Our approach is modeled as a binary labeling problem and solved using the efficient Quadratic Pseudo-Boolean Optimization (QPBO). It yields promising tracking performance on the challenging PETS09 and MOT16 dataset. Our code will be made publicly available upon the acceptance of the manuscript.

Robust 3D Hand Pose Estimation From Single Depth Images Using Multi-View CNNs

Articulated hand pose estimation is one of core technologies in human-computer interaction. Despite the recent progress, most existing methods still cannot achieve satisfactory performance, partly due to the difficulty of the embedded high-dimensional nonlinear regression problem. Most existing data-driven methods directly regress 3D hand pose from 2D depth image, which cannot fully utilize the depth information. In this paper, we propose a novel multi-view convolutional neural network (CNN)-based approach for 3D hand pose estimation. To better exploit 3D information in the depth image, we project the point cloud generated from the query depth image onto multiple views of two projection settings and integrate them for more robust estimation. Multi-view CNNs are trained to learn the mapping from projected images to heat-maps, which reflect probability distributions of joints on each view. These multi-view heat-maps are then fused to estimate the optimal 3D hand pose with learned pose priors, and the unreliable information in multi-view heat-maps is suppressed using a view selection method. Experimental results show that the proposed method is superior to the state-of-the-art methods on two challenging data sets. Furthermore, a cross-data set experiment also validates that our proposed approach has good generalization ability.