Research on the Recognition and Tracking of Group-Housed Pigs' Posture Based on Edge Computing

The existing algorithms for identifying and tracking pigs in barns generally have a large number of parameters, relatively complex networks and a high demand for computational resources, which are not suitable for deployment in embedded-edge nodes on farms. A lightweight multi-objective identification and tracking algorithm based on improved YOLOv5s and DeepSort was developed for group-housed pigs in this study. The identification algorithm was optimized by: (i) using a dilated convolution in the YOLOv5s backbone network to reduce the number of model parameters and computational power requirements; (ii) adding a coordinate attention mechanism to improve the model precision; and (iii) pruning the BN layers to reduce the computational requirements. The optimized identification model was combined with DeepSort to form the final Tracking by Detecting algorithm and ported to a Jetson AGX Xavier edge computing node. The algorithm reduced the model size by 65.3% compared to the original YOLOv5s. The algorithm achieved a recognition precision of 96.6%; a tracking time of 46 ms; and a tracking frame rate of 21.7 FPS, and the precision of the tracking statistics was greater than 90%. The model size and performance met the requirements for stable real-time operation in embedded-edge computing nodes for monitoring group-housed pigs.

Label Metric for Multi-Class Multi-Target Tracking under Hierarchical Multilevel Classification

Aiming at multiple quantities and types of targets, multi-class multi-target tracking usually faces not only cardinality errors, but also mis-classification problems. Considering its performance evaluation, the traditional optimal subpattern assignment (OSPA) method tends to calculate a separate metric for each class of targets, or introduce other indexes such as the classification error rate, which decreases the value of OSPA as a comprehensive single metric. This paper proposed a hierarchical multi-level class label for multi-class multi-target tracking under hierarchical multilevel classification, which can synthetically measure the state errors, cardinality error, and mis-classification. The hierarchical multi-level class label is introduced as an attached label to finite sets based on the hierarchical tree-structured categorization. A Wasserstein distance type metric then can be defined among the distribution represented by any two labels. The proposed label metric is a mathematic metric, and its advantages are illustrated by examples in several cases.

Multi-Target State and Extent Estimation for High Resolution Automotive Sensor Detections

This paper discusses the perception and tracking of individual as well as group targets as applied to multi-lane public traffic. Target tracking problem is formulated as a two hierarchical layer problem-on the first layer, a multi-target tracking problem based on multiple detections is distinguished in the measurement space, and on the second (top) layer, group target tracking with birth and death as well as merging and splitting of group target tracks as they evolve in a dynamic scene is represented. This configuration enhances the multi-target tracking performance in situations including but not limited to target initialization(birth), target occlusion, missed detections, unresolved measurement, target maneuver, etc. In addition, group tracking exposes complex individual target interactions to help in situation assessment which is challenging to capture otherwise.

An Improved Q-Learning-Based Sensor-Scheduling Algorithm for Multi-Target Tracking

Target tracking is an essential issue in wireless sensor networks (WSNs). Compared with single-target tracking, how to guarantee the performance of multi-target tracking is more challenging because the system needs to balance the tracking resource for each target according to different target properties and network status. However, the balance of tracking task allocation is rarely considered in those prior sensor-scheduling algorithms, which may result in the degradation of tracking accuracy for some targets and additional system energy consumption. To address this issue, we propose in this paper an improved Q-learning-based sensor-scheduling algorithm for multi-target tracking (MTT-SS). First, we devise an entropy weight method (EWM)-based strategy to evaluate the priority of targets being tracked according to target properties and network status. Moreover, we develop a Q-learning-based task allocation mechanism to obtain a balanced resource scheduling result in multi-target-tracking scenarios. Simulation results demonstrate that our proposed algorithm can obtain a significant enhancement in terms of tracking accuracy and energy efficiency compared with the existing sensor-scheduling algorithms.

Multi-Feature Matching GM-PHD Filter for Radar Multi-Target Tracking

Multi-target tracking (MTT) is one of the most important functions of radar systems. Traditional multi-target tracking methods based on data association convert multi-target tracking problems into single-target tracking problems. When the number of targets is large, the amount of computation increases exponentially. The Gaussian mixture probability hypothesis density (GM-PHD) filtering based on a random finite set (RFS) provides an effective method to solve multi-target tracking problems without the requirement of explicit data association. However, it is difficult to track targets accurately in real-time with dense clutter and low detection probability. To solve this problem, this paper proposes a multi-feature matching GM-PHD (MFGM-PHD) filter for radar multi-target tracking. Using Doppler and amplitude information contained in radar echo to modify the weights of Gaussian components, the weight of the clutter can be greatly reduced and the target can be distinguished from clutter. Simulations show that the proposed MFGM-PHD filter can improve the accuracy of multi-target tracking as well as the real-time performance with high clutter density and low detection probability.

Design of a Robust System Architecture for Tracking Vehicle on Highway Based on Monocular Camera

Multi-Target tracking is a central aspect of modeling the environment of autonomous vehicles. A mono camera is a necessary component in the autonomous driving system. One of the biggest advantages of the mono camera is it can give out the type of vehicle and cameras are the only sensors able to interpret 2D information such as road signs or lane markings. Besides this, it has the advantage of estimating the lateral velocity of the moving object. The mono camera is now being used by companies all over the world to build autonomous vehicles. In the expressway scenario, the forward-looking camera can generate a raw picture to extract information from and finally achieve tracking multiple vehicles at the same time. A multi-object tracking system, which is composed of a convolution neural network module, depth estimation module, kinematic state estimation module, data association module, and track management module, is needed. This paper applies the YOLO detection algorithm combined with the depth estimation algorithm, Extend Kalman Filter, and Nearest Neighbor algorithm with a gating trick to build the tracking system. Finally, the tracking system is tested on the vehicle equipped with a forward mono camera, and the results show that the lateral and longitudinal position and velocity can satisfy the need for Adaptive Cruise Control (ACC), Navigation On Pilot (NOP), Auto Emergency Braking (AEB), and other applications.

Multi-Target Tracking Using Windowed Fourier Single-Pixel Imaging

The single-pixel imaging (SPI) technique enables the tracking of moving targets at a high frame rate. However, when extended to the problem of multi-target tracking, there is no effective solution using SPI yet. Thus, a multi-target tracking method using windowed Fourier single-pixel imaging (WFSI) is proposed in this paper. The WFSI technique uses a series of windowed Fourier basis patterns to illuminate the target. This method can estimate the displacements of K independently moving targets by implementing 6K measurements and calculating 2K windowed Fourier coefficients, which is a measurement method with low redundancy. To enhance the capability of the proposed method, we propose a joint estimation approach for multi-target displacement, which solves the problem where different targets in close proximity cannot be distinguished. Using the independent and joint estimation approaches, multi-target tracking can be implemented with WFSI. The accuracy of the proposed multi-target tracking method is verified by numerical simulation to be less than 2 pixels. The tracking effectiveness is analyzed by a video experiment. This method provides, for the first time, an effective idea of multi-target tracking using SPI.

Space Debris Tracking with the Poisson Labeled Multi-Bernoulli Filter

This paper presents a Bayesian filter based solution to the Space Object (SO) tracking problem using simulated optical telescopic observations. The presented solution utilizes the Probabilistic Admissible Region (PAR) approach, which is an orbital admissible region that adheres to the assumption of independence between newborn targets and surviving SOs. These SOs obey physical energy constraints in terms of orbital semi-major axis length and eccentricity within a range of orbits of interest. In this article, Low Earth Orbit (LEO) SOs are considered. The solution also adopts the Partially Uniform Birth (PUB) intensity, which generates uniformly distributed births in the sensor field of view. The measurement update then generates a particle SO distribution. In this work, a Poisson Labeled Multi-Bernoulli (PLMB) multi-target tracking filter is proposed, using the PUB intensity model for the multi-target birth density, and a PAR for the spatial density to determine the initial orbits of SOs. Experiments are demonstrated using simulated SO trajectories created from real Two-Line Element data, with simulated measurements from twelve telescopes located in observatories, which form part of the Falcon telescope network. Optimal Sub-Pattern Assignment (OSPA) and CLEAR MOT metrics demonstrate encouraging multi-SO tracking results even under very low numbers of observations per SO pass.

A Robust SMC-PHD Filter for Multi-Target Tracking with Unknown Heavy-Tailed Measurement Noise

In multi-target tracking, the sequential Monte Carlo probability hypothesis density (SMC-PHD) filter is a practical algorithm. Influenced by outliers under unknown heavy-tailed measurement noise, the SMC-PHD filter suffers severe performance degradation. In this paper, a robust SMC-PHD (RSMC-PHD) filter is proposed. In the proposed filter, Student-t distribution is introduced to describe the unknown heavy-tailed measurement noise where the degrees of freedom (DOF) and the scale matrix of the Student-t distribution are respectively modeled as a Gamma distribution and an inverse Wishart distribution. Furthermore, the variational Bayesian (VB) technique is employed to infer the unknown DOF and scale matrix parameters while the recursion estimation framework of the RSMC-PHD filter is derived. In addition, considering that the introduced Student- t distribution might lead to an overestimation of the target number, a strategy is applied to modify the updated weight of each particle. Simulation results demonstrate that the proposed filter is effective with unknown heavy-tailed measurement noise.

PHD Filter for Object Tracking in Road Traffic Applications Considering Varying Detectability

This paper considers the object detection and tracking problem in a road traffic situation from a traffic participant’s perspective. The information source is an automotive radar which is attached to the ego vehicle. The scenario characteristics are varying object visibility due to occlusion and multiple detections of a vehicle during a scanning interval. The goal is to maintain and report the state of undetected though possibly present objects. The proposed algorithm is based on the multi-object Probability Hypothesis Density filter. Because the PHD filter has no memory, the estimate of the number of objects present can change abruptly due to erroneous detections. To reduce this effect, we model the occlusion of the object to calculate the state-dependent detection probability. Thus, the filter can maintain unnoticed but probably valid hypotheses for a more extended period. We use the sequential Monte Carlo method with clustering for implementing the filter. We distinguish between detected, undetected, and hidden particles within our framework, whose purpose is to track hidden but likely present objects. The performance of the algorithm is demonstrated using highway radar measurements.

Output-Only Damage Detection of Shear Building Structures Using an Autoregressive Model-Enhanced Optimal Subpattern Assignment Metric

This paper proposes a novel output-only structural damage indicator by incorporating the pole-based optimal subpattern assignment distance with autoregressive models to localize and relatively assess the severity of damages for sheared structures. Autoregressive models can model dynamic systems well, while their model poles can represent the state of the dynamic systems. Structural damage generally causes changes in the dynamic characteristics (especially the natural frequency, mode shapes and damping ratio) of structures. Since the poles of the autoregressive models can solve the modal parameters of the structure, the poles have a close relationship with the modal parameters so that the changes in the poles of its autoregressive model reflect structural damages. Therefore, we can identify the damage by tracking the shifts in the dynamic system poles. The optimal subpattern assignment distance, which is the performance evaluator in multi-target tracking algorithms to measure the metric between true and estimated tracks, enables the construction of damage sensitive indicator from system poles using the Hungarian algorithm. The proposed approach has been validated with a five-story shear-building using numerical simulations and experimental verifications, which are subjected to excitations of white noise, El Centro earthquake and sinusoidal wave with frequencies sweeping, respectively; the results indicate that this approach can localize and quantify structural damages effectively in an output-only and data-driven way.

Joint Dwell Time and Bandwidth Optimization for Multi-Target Tracking in Radar Network Based on Low Probability of Intercept

Radar network systems have been demonstrated to offer numerous advantages for target tracking. In this paper, a low probability of intercept (LPI)-based joint dwell time and bandwidth optimization strategy is proposed for multi-target tracking in a radar network. Since the Bayesian Cramer-Rao lower bound (BCRLB) provides a lower bound on parameter estimation, it can be utilized as the accuracy metric for target tracking. In this strategy, in order to improve the LPI performance of the radar network, the total dwell time consumption of the underlying system is minimized, while guaranteeing a predetermined tracking accuracy. There are two adaptable parameters in the optimization problem: one for dwell time, and the other for bandwidth allocation. Since the nonlinear programming-based genetic algorithm (NPGA) can solve the nonlinear problem well, we develop a method based upon NPGA to solve the resulting problem. The simulation results demonstrate that the proposed strategy has superiority over traditional algorithms, and can achieve a better LPI performance of this radar network.

On-Line Visual Tracking with Occlusion Handling

One of the core challenges in visual multi-target tracking is occlusion. This is especially important in applications such as video surveillance and sports analytics. While offline batch processing algorithms can utilise future measurements to handle occlusion effectively, online algorithms have to rely on current and past measurements only. As such, it is markedly more challenging to handle occlusion in online applications. To address this problem, we propagate information over time in a way that it generates a sense of déjà vu when similar visual and motion features are observed. To achieve this, we extend the Generalized Labeled Multi-Bernoulli (GLMB) filter, originally designed for tracking point-sized targets, to be used in visual multi-target tracking. The proposed algorithm includes a novel false alarm detection/removal and label recovery methods capable of reliably recovering tracks that are even lost for a substantial period of time. We compare the performance of the proposed method with the state-of-the-art methods in challenging datasets using standard visual tracking metrics. Our comparisons show that the proposed method performs favourably compared to the state-of-the-art methods, particularly in terms of ID switches and fragmentation metrics which signifies occlusion.

Adaptive Estimation of Spatial Clutter Measurement Density Using Clutter Measurement Probability for Enhanced Multi-Target Tracking

The point detections obtained from radars or sonars in surveillance environments include clutter measurements, as well as target measurements. Target tracking with these data requires data association, which distinguishes the detections from targets and clutter. Various algorithms have been proposed for clutter measurement density estimation to achieve accurate and robust target tracking with the point detections. Among them, the spatial clutter measurement density estimator (SCMDE) computes the sparsity of clutter measurement, which is the reciprocal of the clutter measurement density. The SCMDE considers all adjacent measurements only as clutter, so the estimated clutter measurement density is biased for multi-target tracking applications, which may result in degraded target tracking performance. Through the study in this paper, a major source of tracking performance degradation with the existing SCMDE for multi-target tracking is analyzed, and the use of the clutter measurement probability is proposed as a remedy. It is also found that the expansion of the volume of the hyper-sphere for each sparsity order reduces the bias of clutter measurement density estimates. Based on the analysis, we propose a new adaptive clutter measurement density estimation method called SCMDE for multi-target tracking (MTT-SCMDE). The proposed method is applied to multi-target tracking, and the improvement of multi-target tracking performance is shown by a series of Monte Carlo simulation runs and a real radar data test. The clutter measurement density estimation performance and target tracking performance are also analyzed for various sparsity orders.

Multi-Target Localization and Tracking Using TDOA and AOA Measurements Based on Gibbs-GLMB Filtering

This paper deals with mobile multi-target detection and tracking. In the traditional method, there are uncertainties such as misdetection and false alarm in the measurement data, and it will be inevitable having to deal with the data association. To solve the target trajectory and state estimation problem under a cluttered environment, this paper proposes a non-concurrent multi-target acoustic localization tracking method based on the Gibbs-generalized labelled multi-Bernoulli (Gibbs-GLMB) filter and considers an acoustic array of a fixed arrangement for the tracking of targets by joint time difference of arrival (TDOA) and angle of arrival (AOA) measurements. Firstly, the TDOAs are calculated by using the generalized cross-correlation algorithm (GCC) and the AOAs are derived from the received signal directions. Secondly, we assume the independence of the targets and fuse the measurements which are used to track the multiple targets via the Gibbs-GLMB filter. Finally, the effectiveness of the method is verified by Monte Carlo simulation experiments.

Linear-Time Direct Data Assignment Algorithm for Passive Sensor Measurements

To solve the problem of passive sensor data association in multi-sensor multi-target tracking, a novel linear-time direct data assignment (DDA) algorithm is proposed in this paper. Different from existing methods which solve the data association problem in the measurement domain, the proposed algorithm solves the problem directly in the target state domain. The number and state of candidate targets are preset in the region of interest, which can avoid the problem of combinational explosion. The time complexity of the proposed algorithm is linear with the number of sensors and targets while that of the existing algorithms are exponential. Computer simulations show that the proposed algorithm can achieve almost the same association accuracy as the existing algorithms, but the time consumption can be significantly reduced.

A Computationally Efficient Labeled Multi-Bernoulli Smoother for Multi-Target Tracking

A forward–backward labeled multi-Bernoulli (LMB) smoother is proposed for multi-target tracking. The proposed smoother consists of two components corresponding to forward LMB filtering and backward LMB smoothing, respectively. The former is the standard LMB filter and the latter is proved to be closed under LMB prior. It is also shown that the proposed LMB smoother can improve both the cardinality estimation and the state estimation, and the major computational complexity is linear with the number of targets. Implementation based on the Sequential Monte Carlo method in a representative scenario has demonstrated the effectiveness and computational efficiency of the proposed smoother in comparison to existing approaches.

An Adaptive Real-Time Detection Algorithm for Dim and Small Photoelectric GSO Debris

Geosynchronous orbit (GSO) is the ideal orbit for communication, navigation, meteorology and other satellites, but the space of GSO is limited, and there are still a large number of space debris threatening the safety of spacecraft. Therefore, real-time detection of GSO debris is necessary to avoid collision accidents. Because radar is limited by transmitting power and operating distance, it is difficult to detect GSO debris, so photoelectric detection becomes the mainstream way to detect GSO debris. This paper presents an adaptive real-time detection algorithm for GSO debris in the charge coupled device (CCD) images. The main work is as follows: An image adaptive fast registration algorithm and an enhanced dilation difference algorithm are proposed. Combining with mathematical morphology, threshold segmentation and global nearest neighbor (GNN) multi-target tracking algorithm, the functions of image background suppression, registration, suspected target extraction and multi-target tracking are realized. The processing results of a large number of measured data show that the algorithm can detect dim geostationary earth orbit (GEO) and non-GEO debris in GSO belt stably and efficiently, and the processing speed meets the real-time requirements, with strong adaptive ability, and has high practical application value.

Recent Advances in Stochastic Sensor Control for Multi-Object Tracking

In many multi-object tracking applications, the sensor(s) may have controllable states. Examples include movable sensors in multi-target tracking applications in defence, and unmanned air vehicles (UAVs) as sensors in multi-object systems used in civil applications such as inspection and fault detection. Uncertainties in the number of objects (due to random appearances and disappearances) as well as false alarms and detection uncertainties collectively make the above problem a highly challenging stochastic sensor control problem. Numerous solutions have been proposed to tackle the problem of precise control of sensor(s) for multi-object detection and tracking, and, in this work, recent contributions towards the advancement in the domain are comprehensively reviewed. After an introduction, we provide an overview of the sensor control problem and present the key components of sensor control solutions in general. Then, we present a categorization of the existing methods and review those methods under each category. The categorization includes a new generation of solutions called selective sensor control that have been recently developed for applications where particular objects of interest need to be accurately detected and tracked by controllable sensors.

SLAM in Dynamic Environments: A Deep Learning Approach for Moving Object Tracking Using ML-RANSAC Algorithm

The important problem of Simultaneous Localization and Mapping (SLAM) in dynamic environments is less studied than the counterpart problem in static settings. In this paper, we present a solution for the feature-based SLAM problem in dynamic environments. We propose an algorithm that integrates SLAM with multi-target tracking (SLAMMTT) using a robust feature-tracking algorithm for dynamic environments. A novel implementation of RANdomSAmple Consensus (RANSAC) method referred to as multilevel-RANSAC (ML-RANSAC) within the Extended Kalman Filter (EKF) framework is applied for multi-target tracking (MTT). We also apply machine learning to detect features from the input data and to distinguish moving from stationary objects. The data stream from LIDAR and vision sensors are fused in real-time to detect objects and depth information. A practical experiment is designed to verify the performance of the algorithm in a dynamic environment. The unique feature of this algorithm is its ability to maintain tracking of features even when the observations are intermittent whereby many reported algorithms fail in such situations. Experimental validation indicates that the algorithm is able to perform consistent estimates in a fast and robust manner suggesting its feasibility for real-time applications.

Doppler Data Association Scheme for Multi-Target Tracking in an Active Sonar System

In many wireless sensors, the target kinematic states include location and Doppler information that can be observed from a time series of range and velocity measurements. In this work, we present a tracking strategy for comprising target velocity components as part of the measurement supplement procedure and evaluate the advantages of the proposed scheme. Data association capability can be considered as the key performance for multi-target tracking in an active sonar system. Then, we proposed an enhanced Doppler data association (DDA) scheme which exploits target range and target velocity components for linear multi-target tracking. If the target velocity measurements are not incorporated into target kinematic state tracking, the linear filter bank for the combination of target velocity components can be implemented. Finally, a significant enhancement in the multi-target tracking capability provided by the proposed DDA scheme with the linear multi-target combined probabilistic data association method is demonstrated in a sonar underwater scenario.

Information Fusion for Industrial Mobile Platform Safety via Track-Before-Detect Labeled Multi-Bernoulli Filter

This paper presents a novel Track-Before-Detect (TBD) Labeled Multi-Bernoulli (LMB) filter tailored for industrial mobile platform safety applications. At the core of the developed solution is two techniques for fusion of color and edge information in visual tracking. We derive an application specific separable likelihood function that captures the geometric shape of the human targets wearing safety vests. We use a novel geometric shape likelihood along with a color likelihood to devise two Bayesian updates steps which fuse shape and color related information. One approach is sequential and the other is based on weighted Kullback–Leibler average (KLA). Experimental results show that the KLA based fusion variant of the proposed algorithm outperforms both the sequential update based variant and a state-of-art method in terms of the performance metrics commonly used in computer vision literature.

Adaptive Target Birth Intensity Multi-Bernoulli Filter with Noise-Based Threshold

Adaptively modeling the target birth intensity while maintaining the filtering efficiency is a challenging issue in multi-target tracking (MTT). Generally, the target birth probability is predefined as a constant and only the target birth density is considered in existing adaptive birth models, resulting in deteriorated target tracking accuracy, especially in the target appearing cases. In addition, the existing adaptive birth models also give rise to a decline in operation efficiency on account of the extra birth modeling calculations. To properly adapt the real variation of the number of newborn targets and improve the multi-target tracking performance, a novel fast sequential Monte Carlo (SMC) adaptive target birth intensity cardinality balanced multi-target multi-Bernoulli (CBMeMBer) filter is proposed in this paper. Through adaptively conducting the target birth probability in a pre-processing step, which incorporates the information of current measurements to correct the pre-setting of the target birth probability, the proposed filter can truly adapt target birth cases and achieve better tracking accuracy. Moreover, the implementation efficiency can be improved significantly by employing a measurement noise-based threshold in the likelihood calculations of the multi-target updating. Simulation results verify the effectiveness of the proposed filter.

Multi-Objective Optimization Based Multi-Bernoulli Sensor Selection for Multi-Target Tracking

This paper presents a novel multi-objective optimization based sensor selection method for multi-target tracking in sensor networks. The multi-target states are modelled as multi-Bernoulli random finite sets and the multi-Bernoulli filter is used to propagate the multi-target posterior density. The proposed method is designed to select the sensor that provides the most reliable cardinality estimate, since more accurate cardinality estimate indicates more accurate target states. In the multi-Bernoulli filter, the updated multi-target density is a multi-Bernoulli random finite set formed by a union of legacy tracks and measurement-updated tracks. The legacy track and the measurement-updated track have different theoretical and physical meanings, and hence these two kinds of tracks are considered separately in the sensor management problem. Specifically, two objectives are considered: (1) maximizing the mean cardinality of the measurement-updated tracks, (2) minimizing the cardinality variance of the legacy tracks. Considering the conflicting objectives simultaneously is a multi-objective optimization problem. Tradeoff solutions between two conflicting objectives will be derived. Theoretical analysis and examples show that the proposed approach is effective and direct. The performance of the proposed method is demonstrated using two scenarios with different levels of observability of targets in the passive sensor network.

Sensor Selection for Decentralized Large-Scale Multi-Target Tracking Network

A new optimization algorithm of sensor selection is proposed in this paper for decentralized large-scale multi-target tracking (MTT) network within a labeled random finite set (RFS) framework. The method is performed based on a marginalized δ-generalized labeled multi-Bernoulli RFS. The rule of weighted Kullback-Leibler average (KLA) is used to fuse local multi-target densities. A new metric, named as the label assignment (LA) metric, is proposed to measure the distance for two labeled sets. The lower bound of LA metric based mean square error between the labeled multi-target state set and its estimate is taken as the optimized objective function of sensor selection. The proposed bound is obtained by the information inequality to RFS measurement. Then, we present the sequential Monte Carlo and Gaussian mixture implementations for the bound. Another advantage of the bound is that it provides a basis for setting the weights of KLA. The coordinate descent method is proposed to compromise the computational cost of sensor selection and the accuracy of MTT. Simulations verify the effectiveness of our method under different signal-to- noise ratio scenarios.

Automated Intracellular Calcium Profiles Extraction from Endothelial Cells Using Digital Fluorescence Images

Endothelial cells perform a wide variety of fundamental functions for the cardiovascular system, their proliferation and migration being strongly regulated by their intracellular calcium concentration. Hence it is extremely important to carefully measure endothelial calcium signals under different stimuli. A proposal to automate the intracellular calcium profiles extraction from fluorescence image sequences is presented. Digital image processing techniques were combined with a multi-target tracking approach supported by Kalman estimation. The system was tested with image sequences from two different stimuli. The first one was a chemical stimulus, that is, ATP, which caused small movements in the cells trajectories, thereby suggesting that the bath application of the agonist does not generate significant artifacts. The second one was a mechanical stimulus delivered by a glass microelectrode, which caused major changes in cell trajectories. The importance of the tracking block is evidenced since more accurate profiles were extracted, mainly for cells closest to the stimulated area. Two important contributions of this work are the automatic relocation of the region of interest assigned to the cells and the possibility of data extraction from big image sets in efficient and expedite way. The system may adapt to different kind of cell images and may allow the extraction of other useful features.

Constrained Multi-Sensor Control Using a Multi-Target MSE Bound and a δ-GLMB Filter

The existing multi-sensor control algorithms for multi-target tracking (MTT) within the random finite set (RFS) framework are all based on the distributed processing architecture, so the rule of generalized covariance intersection (GCI) has to be used to obtain the multi-sensor posterior density. However, there has still been no reliable basis for setting the normalized fusion weight of each sensor in GCI until now. Therefore, to avoid the GCI rule, the paper proposes a new constrained multi-sensor control algorithm based on the centralized processing architecture. A multi-target mean-square error (MSE) bound defined in our paper is served as cost function and the multi-sensor control commands are just the solutions that minimize the bound. In order to derive the bound by using the generalized information inequality to RFS observation, the error between state set and its estimation is measured by the second-order optimal sub-pattern assignment metric while the multi-target Bayes recursion is performed by using a δ-generalized labeled multi-Bernoulli filter. An additional benefit of our method is that the proposed bound can provide an online indication of the achievable limit for MTT precision after the sensor control. Two suboptimal algorithms, which are mixed penalty function (MPF) method and complex method, are used to reduce the computation cost of solving the constrained optimization problem. Simulation results show that for the constrained multi-sensor control system with different observation performance, our method significantly outperforms the GCI-based Cauchy-Schwarz divergence method in MTT precision. Besides, when the number of sensors is relatively large, the computation time of the MPF and complex methods is much shorter than that of the exhaustive search method at the expense of completely acceptable loss of tracking accuracy.

Dual Sensor Control Scheme for Multi-Target Tracking

Sensor control is a challenging issue in the field of multi-target tracking. It involves multi-target state estimation and the optimal control of the sensor. To maximize the overall utility of the surveillance system, we propose a dual sensor control scheme. This work is formulated in the framework of partially observed Markov decision processes (POMDPs) with Mahler’s finite set statistics (FISST). To evaluate the performance associated with each control action, a key element is to design an appropriate metric. From a task-driven perspective, we utilize a metric to minimize the posterior distance between the sensor and the target. This distance-related metric promotes the design of a dual sensor control scheme. Moreover, we introduce a metric to maximize the predicted average probability of detection, which will improve the efficiency by avoiding unnecessary update processes. Simulation results indicate that the performance of the proposed algorithm is significantly superior to the existing methods.

δ-Generalized Labeled Multi-Bernoulli Filter Using Amplitude Information of Neighboring Cells

The amplitude information (AI) of echoed signals plays an important role in radar target detection and tracking. A lot of research shows that the introduction of AI enables the tracking algorithm to distinguish targets from clutter better and then improves the performance of data association. The current AI-aided tracking algorithms only consider the signal amplitude in the range-azimuth cell where measurement exists. However, since radar echoes always contain backscattered signals from multiple cells, the useful information of neighboring cells would be lost if directly applying those existing methods. In order to solve this issue, a new δ-generalized labeled multi-Bernoulli (δ-GLMB) filter is proposed. It exploits the AI of radar echoes from neighboring cells to construct a united amplitude likelihood ratio, and then plugs it into the update process and the measurement-track assignment cost matrix of the δ-GLMB filter. Simulation results show that the proposed approach has better performance in target’s state and number estimation than that of the δ-GLMB only using single-cell AI in low signal-to-clutter-ratio (SCR) environment.

A Student's t Mixture Probability Hypothesis Density Filter for Multi-Target Tracking with Outliers

In multi-target tracking, the outliers-corrupted process and measurement noises can reduce the performance of the probability hypothesis density (PHD) filter severely. To solve the problem, this paper proposed a novel PHD filter, called Student’s t mixture PHD (STM-PHD) filter. The proposed filter models the heavy-tailed process noise and measurement noise as a Student’s t distribution as well as approximates the multi-target intensity as a mixture of Student’s t components to be propagated in time. Then, a closed PHD recursion is obtained based on Student’s t approximation. Our approach can make full use of the heavy-tailed characteristic of a Student’s t distribution to handle the situations with heavy-tailed process and the measurement noises. The simulation results verify that the proposed filter can overcome the negative effect generated by outliers and maintain a good tracking accuracy in the simultaneous presence of process and measurement outliers.

Markov Chain Realization of Joint Integrated Probabilistic Data Association

A practical probabilistic data association filter is proposed for tracking multiple targets in clutter. The number of joint data association events increases combinatorially with the number of measurements and the number of targets, which may become computationally impractical for even small numbers of closely located targets in real target-tracking applications in heavily cluttered environments. In this paper, a Markov chain model is proposed to generate a set of feasible joint events (FJEs) for multiple target tracking that is used to approximate the multi-target data association probabilities and the probabilities of target existence of joint integrated probabilistic data association (JIPDA). A Markov chain with the transition probabilities obtained from the integrated probabilistic data association (IPDA) for single-target tracking is designed to generate a random sequence composed of the predetermined number of FJEs without incurring additional computational cost. The FJEs generated are adjusted for the multi-target tracking environment. A computationally tractable set of these random sequences is utilized to evaluate the track-to-measurement association probabilities such that the computational burden is substantially reduced compared to the JIPDA algorithm. By a series of simulations, the track confirmation rates and target retention statistics of the proposed algorithm are compared with the other existing algorithms including JIPDA to show the effectiveness of the proposed algorithm.

An Automatic Multi-Target Independent Analysis Framework for Non-Planar Infrared-Visible Registration

In this paper, we propose a novel automatic multi-target registration framework for non-planar infrared-visible videos. Previous approaches usually analyzed multiple targets together and then estimated a global homography for the whole scene, however, these cannot achieve precise multi-target registration when the scenes are non-planar. Our framework is devoted to solving the problem using feature matching and multi-target tracking. The key idea is to analyze and register each target independently. We present a fast and robust feature matching strategy, where only the features on the corresponding foreground pairs are matched. Besides, new reservoirs based on the Gaussian criterion are created for all targets, and a multi-target tracking method is adopted to determine the relationships between the reservoirs and foreground blobs. With the matches in the corresponding reservoir, the homography of each target is computed according to its moving state. We tested our framework on both public near-planar and non-planar datasets. The results demonstrate that the proposed framework outperforms the state-of-the-art global registration method and the manual global registration matrix in all tested datasets.

Image-Based Multi-Target Tracking through Multi-Bernoulli Filtering with Interactive Likelihoods

We develop an interactive likelihood (ILH) for sequential Monte Carlo (SMC) methods for image-based multiple target tracking applications. The purpose of the ILH is to improve tracking accuracy by reducing the need for data association. In addition, we integrate a recently developed deep neural network for pedestrian detection along with the ILH with a multi-Bernoulli filter. We evaluate the performance of the multi-Bernoulli filter with the ILH and the pedestrian detector in a number of publicly available datasets (2003 PETS INMOVE, Australian Rules Football League (AFL) and TUD-Stadtmitte) using standard, well-known multi-target tracking metrics (optimal sub-pattern assignment (OSPA) and classification of events, activities and relationships for multi-object trackers (CLEAR MOT)). In all datasets, the ILH term increases the tracking accuracy of the multi-Bernoulli filter.

Multi-Target Tracking Using an Improved Gaussian Mixture CPHD Filter

The cardinalized probability hypothesis density (CPHD) filter is an alternative approximation to the full multi-target Bayesian filter for tracking multiple targets. However, although the joint propagation of the posterior intensity and cardinality distribution in its recursion allows more reliable estimates of the target number than the PHD filter, the CPHD filter suffers from the spooky effect where there exists arbitrary PHD mass shifting in the presence of missed detections. To address this issue in the Gaussian mixture (GM) implementation of the CPHD filter, this paper presents an improved GM-CPHD filter, which incorporates a weight redistribution scheme into the filtering process to modify the updated weights of the Gaussian components when missed detections occur. In addition, an efficient gating strategy that can adaptively adjust the gate sizes according to the number of missed detections of each Gaussian component is also presented to further improve the computational efficiency of the proposed filter. Simulation results demonstrate that the proposed method offers favorable performance in terms of both estimation accuracy and robustness to clutter and detection uncertainty over the existing methods.

Adaptive Collaborative Gaussian Mixture Probability Hypothesis Density Filter for Multi-Target Tracking

In this paper, an adaptive collaborative Gaussian Mixture Probability Hypothesis Density (ACo-GMPHD) filter is proposed for multi-target tracking with automatic track extraction. Based on the evolutionary difference between the persistent targets and the birth targets, the measurements are adaptively partitioned into two parts, persistent and birth measurement sets, for updating the persistent and birth target Probability Hypothesis Density, respectively. Furthermore, the collaboration mechanism of multiple probability hypothesis density (PHDs) is established, where tracks can be automatically extracted. Simulation results reveal that the proposed filter yields considerable computational savings in processing requirements and significant improvement in tracking accuracy.

Improved Bearings-Only Multi-Target Tracking with GM-PHD Filtering

In this paper, an improved nonlinear Gaussian mixture probability hypothesis density (GM-PHD) filter is proposed to address bearings-only measurements in multi-target tracking. The proposed method, called the Gaussian mixture measurements-probability hypothesis density (GMM-PHD) filter, not only approximates the posterior intensity using a Gaussian mixture, but also models the likelihood function with a Gaussian mixture instead of a single Gaussian distribution. Besides, the target birth model of the GMM-PHD filter is assumed to be partially uniform instead of a Gaussian mixture. Simulation results show that the proposed filter outperforms the GM-PHD filter embedded with the extended Kalman filter (EKF) and the unscented Kalman filter (UKF).

Cardinality Balanced Multi-Target Multi-Bernoulli Filter with Error Compensation

The cardinality balanced multi-target multi-Bernoulli (CBMeMBer) filter developed recently has been proved an effective multi-target tracking (MTT) algorithm based on the random finite set (RFS) theory, and it can jointly estimate the number of targets and their states from a sequence of sensor measurement sets. However, because of the existence of systematic errors in sensor measurements, the CBMeMBer filter can easily produce different levels of performance degradation. In this paper, an extended CBMeMBer filter, in which the joint probability density function of target state and systematic error is recursively estimated, is proposed to address the MTT problem based on the sensor measurements with systematic errors. In addition, an analytic implementation of the extended CBMeMBer filter is also presented for linear Gaussian models. Simulation results confirm that the proposed algorithm can track multiple targets with better performance.

Multi-Target State Extraction for the SMC-PHD Filter

The sequential Monte Carlo probability hypothesis density (SMC-PHD) filter has been demonstrated to be a favorable method for multi-target tracking. However, the time-varying target states need to be extracted from the particle approximation of the posterior PHD, which is difficult to implement due to the unknown relations between the large amount of particles and the PHD peaks representing potential target locations. To address this problem, a novel multi-target state extraction algorithm is proposed in this paper. By exploiting the information of measurements and particle likelihoods in the filtering stage, we propose a validation mechanism which aims at selecting effective measurements and particles corresponding to detected targets. Subsequently, the state estimates of the detected and undetected targets are performed separately: the former are obtained from the particle clusters directed by effective measurements, while the latter are obtained from the particles corresponding to undetected targets via clustering method. Simulation results demonstrate that the proposed method yields better estimation accuracy and reliability compared to existing methods.

Multi-Target Joint Detection and Estimation Error Bound for the Sensor with Clutter and Missed Detection

The error bound is a typical measure of the limiting performance of all filters for the given sensor measurement setting. This is of practical importance in guiding the design and management of sensors to improve target tracking performance. Within the random finite set (RFS) framework, an error bound for joint detection and estimation (JDE) of multiple targets using a single sensor with clutter and missed detection is developed by using multi-Bernoulli or Poisson approximation to multi-target Bayes recursion. Here, JDE refers to jointly estimating the number and states of targets from a sequence of sensor measurements. In order to obtain the results of this paper, all detectors and estimators are restricted to maximum a posteriori (MAP) detectors and unbiased estimators, and the second-order optimal sub-pattern assignment (OSPA) distance is used to measure the error metric between the true and estimated state sets. The simulation results show that clutter density and detection probability have significant impact on the error bound, and the effectiveness of the proposed bound is verified by indicating the performance limitations of the single-sensor probability hypothesis density (PHD) and cardinalized PHD (CPHD) filters for various clutter densities and detection probabilities.

Multi-Target Tracking Based on Multi-Bernoulli Filter with Amplitude for Unknown Clutter Rate

Knowledge of the clutter rate is of critical importance in multi-target Bayesian tracking. However, estimating the clutter rate is a difficult problem in practice. In this paper, an improved multi-Bernoulli filter based on random finite sets for multi-target Bayesian tracking accommodating non-linear dynamic and measurement models, as well as unknown clutter rate, is proposed for radar sensors. The proposed filter incorporates the amplitude information into the state and measurement spaces to improve discrimination between actual targets and clutters, while adaptively generating the new-born object random finite sets using the measurements to eliminate reliance on prior random finite sets. A sequential Monte-Carlo implementation of the proposed filter is presented, and simulations are used to demonstrate the proposed filter's improvements in estimation accuracy of the target number and corresponding multi-target states, as well as the clutter rate.

FISST based method for multi-target tracking in the image plane of optical sensors

A finite set statistics (FISST)-based method is proposed for multi-target tracking in the image plane of optical sensors. The method involves using signal amplitude information in probability hypothesis density (PHD) filter which is derived from FISST to improve multi-target tracking performance. The amplitude of signals generated by the optical sensor is modeled first, from which the amplitude likelihood ratio between target and clutter is derived. An alternative approach is adopted for the situations where the signal noise ratio (SNR) of target is unknown. Then the PHD recursion equations incorporated with signal information are derived and the Gaussian mixture (GM) implementation of this filter is given. Simulation results demonstrate that the proposed method achieves significantly better performance than the generic PHD filter. Moreover, our method has much lower computational complexity in the scenario with high SNR and dense clutter.

Penalty dynamic programming algorithm for dim targets detection in sensor systems

In order to detect and track multiple maneuvering dim targets in sensor systems, an improved dynamic programming track-before-detect algorithm (DP-TBD) called penalty DP-TBD (PDP-TBD) is proposed. The performances of tracking techniques are used as a feedback to the detection part. The feedback is constructed by a penalty term in the merit function, and the penalty term is a function of the possible target state estimation, which can be obtained by the tracking methods. With this feedback, the algorithm combines traditional tracking techniques with DP-TBD and it can be applied to simultaneously detect and track maneuvering dim targets. Meanwhile, a reasonable constraint that a sensor measurement can originate from one target or clutter is proposed to minimize track separation. Thus, the algorithm can be used in the multi-target situation with unknown target numbers. The efficiency and advantages of PDP-TBD compared with two existing methods are demonstrated by several simulations.