Person Reidentification in a Distributed Camera Network Framework

Revisiting Person Re-Identification by Camera Selection

Person re-identification (Re-ID) is a fundamental task in visual surveillance. Given a query image of the target person, conventional Re-ID focuses on the pairwise similarities between the candidate images and the query. However, conventional Re-ID does not evaluate the consistency of the retrieval results of whether the most similar images ranked in each place contain the same person, which is risky in some applications such as missing out a place where the patient passed will hinder the epidemiological investigation. In this work, we investigate a more challenging task: consistently and successfully retrieving the target person in all camera views. We define the task as continuous person Re-ID and propose a corresponding evaluation metric termed overall Rank-K accuracy. Different from the conventional Re-ID, any incorrect retrieval under an individual camera view that raises an inconsistency will fail the continuous Re-ID. Consequently, the defective cameras, in which the images are hard to be automatically associated with the images from other views, strongly degrade the performance of continuous person Re-ID. Since the camera deployment is crucial for continuous tracking across camera views, we rethink person Re-ID from the perspective of camera deployment and assess the quality of a camera network by performing continuous Re-ID. Moreover, we propose to automatically detect the defective cameras that greatly hamper the continuous Re-ID. Because brute-force search is costly when the camera network becomes complicated, we explicitly model the visual relations as well as the spatial relations among cameras and develop a relational deep Q-network to select the properly deployed cameras and the un-selected cameras are regarded as the defective cameras. Since most existing datasets do not provide topology information about the camera network, they are unsuitable for investigating the importance of spatial relations on camera selection. Thus, we collect a new dataset including 20 cameras with topology information. Compared with randomly removing cameras, the experimental results show that our method can effectively detect the defective cameras so that people could take further operations on these cameras in practice (https://www.isee-ai.cn/∼yixing/MCCPD.html).

Multi-Camera Trajectory Forecasting With Trajectory Tensors

We introduce the problem of multi-camera trajectory forecasting (MCTF), which involves predicting the trajectory of a moving object across a network of cameras. While multi-camera setups are widespread for applications such as surveillance and traffic monitoring, existing trajectory forecasting methods typically focus on single-camera trajectory forecasting (SCTF), limiting their use for such applications. Furthermore, using a single camera limits the field-of-view available, making long-term trajectory forecasting impossible. We address these shortcomings of SCTF by developing an MCTF framework that simultaneously uses all estimated relative object locations from several viewpoints and predicts the object's future location in all possible viewpoints. Our framework follows a Which-When-Where approach that predicts in which camera(s) the objects appear and when and where within the camera views they appear. To this end, we propose the concept of trajectory tensors: a new technique to encode trajectories across multiple camera views and the associated uncertainties. We develop several encoder-decoder MCTF models for trajectory tensors and present extensive experiments on our own database (comprising 600 hours of video data from 15 camera views) created particularly for the MCTF task. Results show that our trajectory tensor models outperform coordinate trajectory-based MCTF models and existing SCTF methods adapted for MCTF. Code is available from: https://github.com/olly-styles/Trajectory-Tensors.

Efficient Person Search via Expert-Guided Knowledge Distillation

The person search problem aims to find the target person in the scene images, which presents high demands for both effectiveness and efficiency. In this paper, we present a unified person search framework which jointly handles the two demands for real-world applications. We explore the technique of knowledge distillation (KD), which allows the student network to share capabilities of the deep expert networks with much fewer parameters and less computing time. To achieve this, we describe an efficient person search network and a set of deep and well-engineered expert networks, to build a tiny and compact model that can approximate the representations of the expert networks in a multitask learning manner. We present extensive experiments on three customized student networks with different scales of networks and show strong performance compared to the state-of-the-art methods on both mean average precision and top-1 accuracies. We further demonstrate the efficiency of the proposed network at 120 frames/s in the feedforward time with only a little sacrifice on the accuracy.

Person Reidentification via Unsupervised Cross-View Metric Learning

Person reidentification (Re-ID) aims to match observations of individuals across multiple nonoverlapping camera views. Recently, metric learning-based methods have played important roles in addressing this task. However, metrics are mostly learned in supervised manners, of which the performance relies heavily on the quantity and quality of manual annotations. Meanwhile, metric learning-based algorithms generally project person features into a common subspace, in which the extracted features are shared by all views. However, it may result in information loss since these algorithms neglect the view-specific features. Besides, they assume person samples of different views are taken from the same distribution. Conversely, these samples are more likely to obey different distributions due to view condition changes. To this end, this paper proposes an unsupervised cross-view metric learning method based on the properties of data distributions. Specifically, person samples in each view are taken from a mixture of two distributions: one models common prosperities among camera views and the other focuses on view-specific properties. Based on this, we introduce a shared mapping to explore the shared features. Meanwhile, we construct view-specific mappings to extract and project view-related features into a common subspace. As a result, samples in the transformed subspace follow the same distribution and are equipped with comprehensive representations. In this paper, these mappings are learned in an unsupervised manner by clustering samples in the projected space. Experimental results on five cross-view datasets validate the effectiveness of the proposed method.

A New Deep Learning Method Based on Unsupervised Domain Adaptation and Re-ranking in Person Re-identification

Person re-identification (Re-ID) is a research hot spot in the field of intelligent video analysis, and it is also a challenging task. As the number of samples grows larger, traditional metric and feature learning methods fall into bottleneck, while it just meets the needs of deep learning algorithm, which perform very well in person re-identification. Although they have achieved good results in the field of supervised learning, their application in real-world scenarios is not very satisfactory. This is mainly because in the real world, a huge number of labeled images are hard to obtain, and even if they are obtained, the cost is expensive. Meanwhile, the performance of deep learning in unsupervised metrics is not ideal. For solving the problem, we propose a new method based on unsupervised domain adaptation (UDA) and re-ranking, and name it UDA[Formula: see text]. As for this method, we first train a camera-aware style transfer model to gain camstyle images. Then we further reduce the difference between the domain of the target and source by using invariant feature, and further improve their commonality. In addition, re-ranking is also introduced to optimize the matching results. This method can not only reduce the cost of obtaining labeled data, but also improve the accuracy. Experimental results show that our method can outperform the most advanced method by 4% on Rank-1 and 14% on mAP. The results also better confirm the effectiveness of Re-ranking module and provide a new idea for domain adaptation by unsupervised methods in the future.

Person Reidentification via Discrepancy Matrix and Matrix Metric

Person reidentification (re-id), as an important task in video surveillance and forensics applications, has been widely studied. Previous research efforts toward solving the person re-id problem have primarily focused on constructing robust vector description by exploiting appearance's characteristic, or learning discriminative distance metric by labeled vectors. Based on the cognition and identification process of human, we propose a new pattern, which transforms the feature description from characteristic vector to discrepancy matrix. In particular, in order to well identify a person, it converts the distance metric from vector metric to matrix metric, which consists of the intradiscrepancy projection and interdiscrepancy projection parts. We introduce a consistent term and a discriminative term to form the objective function. To solve it efficiently, we utilize a simple gradient-descent method under the alternating optimization process with respect to the two projections. Experimental results on public datasets demonstrate the effectiveness of the proposed pattern as compared with the state-of-the-art approaches.

Discriminant Context Information Analysis for Post-Ranking Person Re-Identification

Existing approaches for person re-identification are mainly based on creating distinctive representations or on learning optimal metrics. The achieved results are then provided in the form of a list of ranked matching persons. It often happens that the true match is not ranked first but it is in the first positions. This is mostly due to the visual ambiguities shared between the true match and other "similar" persons. At the current state, there is a lack of a study of such visual ambiguities which limit the re-identification performance within the first ranks. We believe that an analysis of the similar appearances of the first ranks can be helpful in detecting, hence removing, such visual ambiguities. We propose to achieve such a goal by introducing an unsupervised post-ranking framework. Once the initial ranking is available, content and context sets are extracted. Then, these are exploited to remove the visual ambiguities and to obtain the discriminant feature space which is finally exploited to compute the new ranking. An in-depth analysis of the performance achieved on three public benchmark data sets support our believes. For every data set, the proposed method remarkably improves the first ranks results and outperforms the state-of-the-art approaches.