Pedestrian detection via classification on Riemannian manifolds

Hyperscanning EEG and Classification Based on Riemannian Geometry for Festive and Violent Mental State Discrimination

Interactions between two brains constitute the essence of social communication. Daily movements are commonly executed during social interactions and are determined by different mental states that may express different positive or negative behavioral intent. In this context, the effective recognition of festive or violent intent before the action execution remains crucial for survival. Here, we hypothesize that the EEG signals contain the distinctive features characterizing movement intent already expressed before movement execution and that such distinctive information can be identified by state-of-the-art classification algorithms based on Riemannian geometry. We demonstrated for the first time that a classifier based on covariance matrices and Riemannian geometry can effectively discriminate between neutral, festive, and violent mental states only on the basis of non-invasive EEG signals in both the actor and observer participants. These results pave the way for new electrophysiological discrimination of mental states based on non-invasive EEG recordings and cutting-edge machine learning techniques.

Riemann-Based Algorithms Assessment for Single- and Multiple-Trial P300 Classification in Non-Optimal Environments

The P300 wave is commonly used in Brain-Computer Interface technology due to its higher bit rates when compared to other BCI paradigms. P300 classification pipelines based on Riemannian Geometry provide accuracies on par with state-of-the-art pipelines, without having the need for spatial filters, and also possess the ability to be calibrated with little data. In this study, five different P300 detection pipelines are compared, with three of them using Riemannian Geometry as either feature extraction or classification algorithms. The goal of this study is to assess the viability of Riemannian Geometry-based methods in non-optimal environments with sudden background noise changes, rather than maximizing classification accuracy values. For fifteen subjects, the average single-trial accuracy obtained for each pipeline was: 56.06% for Linear Discriminant Analysis (LDA), 72.13% for Bayesian Linear Discriminant Analysis (BLDA), 63.56% for Riemannian Minimum Distance to Mean (MDM), 69.22% for Riemannian Tangent Space with Logistic Regression (TS-LogR), and 63.30% for Riemannian Tangent Space with Support Vector Machine (TS-SVM). The results are higher for the pipelines based on BLDA and TS-LogR, suggesting that they could be viable methods for the detection of the P300 component when maximizing the bit rate is needed. For multiple-trial classification, the BLDA pipeline converged faster towards higher average values, closely followed by the TS-LogR pipeline. The two remaining Riemannian methods' accuracy also increases with the number of trials, but towards a lower value compared to the aforementioned ones. Single-stimulus detection metrics revealed that the TS-LogR pipeline can be a viable classification method, as its results are only slightly lower than those obtained with BLDA. P300 waveforms were also analyzed to check for evidence of the component being elicited. Finally, a questionnaire was used to retrieve the most intuitive focusing methods employed by the subjects.

Automatic Analysis of Facial Expressions Based on Deep Covariance Trajectories

In this article, we propose a new approach for facial expression recognition (FER) using deep covariance descriptors. The solution is based on the idea of encoding local and global deep convolutional neural network (DCNN) features extracted from still images, in compact local and global covariance descriptors. The space geometry of the covariance matrices is that of symmetric positive definite (SPD) matrices. By conducting the classification of static facial expressions using a support vector machine (SVM) with a valid Gaussian kernel on the SPD manifold, we show that deep covariance descriptors are more effective than the standard classification with fully connected layers and softmax. Besides, we propose a completely new and original solution to model the temporal dynamic of facial expressions as deep trajectories on the SPD manifold. As an extension of the classification pipeline of covariance descriptors, we apply SVM with valid positive definite kernels derived from global alignment for deep covariance trajectories classification. By performing extensive experiments on the Oulu-CASIA, CK+, static facial expression in the wild (SFEW), and acted facial expressions in the wild (AFEW) data sets, we show that both the proposed static and dynamic approaches achieve the state-of-the-art performance for FER outperforming many recent approaches.

Hierarchical Gaussian Descriptors with Application to Person Re-Identification

Describing the color and textural information of a person image is one of the most crucial aspects of person re-identification (re-id). Although a covariance descriptor has been successfully applied to person re-id, it loses the local structure of a region and mean information of pixel features, both of which tend to be the major discriminative information for person re-id. In this paper, we present novel meta-descriptors based on a hierarchical Gaussian distribution of pixel features, in which both mean and covariance information are included in patch and region level descriptions. More specifically, the region is modeled as a set of multiple Gaussian distributions, each of which represents the appearance of a local patch. The characteristics of the set of Gaussian distributions are again described by another Gaussian distribution. Because the space of Gaussian distribution is not a linear space, we embed the parameters of the distribution into a point of Symmetric Positive Definite (SPD) matrix manifold in both steps. We show, for the first time, that normalizing the scale of the SPD matrix enhances the hierarchical feature representation on this manifold. Additionally, we develop feature norm normalization methods with the ability to alleviate the biased trends that exist on the SPD matrix descriptors. The experimental results conducted on five public datasets indicate the effectiveness of the proposed descriptors and the two types of normalizations.

Decoding multiclass motor imagery EEG from the same upper limb by combining Riemannian geometry features and partial least squares regression

OBJECTIVE

Due to low spatial resolution and poor signal-to-noise ratio of electroencephalogram (EEG), high accuracy classifications still suffer from lots of obstacles in the context of motor imagery (MI)-based brain-machine interface (BMI) systems. Particularly, it is extremely challenging to decode multiclass MI EEG from the same upper limb. This research proposes a novel feature learning approach to address the classification problem of 6-class MI tasks, including imaginary elbow flexion/extension, wrist supination/pronation, and hand close/open within the unilateral upper limb.

APPROACH

Instead of the traditional common spatial pattern (CSP) or filter-bank CSP (FBCSP) manner, the Riemannian geometry (RG) framework involving Riemannian distance and Riemannian mean was directly adopted to extract tangent space (TS) features from spatial covariance matrices of the MI EEG trials. Subsequently, to reduce the dimensionality of the TS features, the algorithm of partial least squares regression was applied to obtain more separable and compact feature representations.

MAIN RESULTS

The performance of the learned RG feature representations was validated by a linear discriminative analysis and support vector machine classifier, with an average accuracy of 80.50% and 79.70% on EEG dataset collected from 12 participants, respectively.

SIGNIFICANCE

These results demonstrate that compared with CSP and FBCSP features, the proposed approach can significantly increase the decoding accuracy for multiclass MI tasks from the same upper limb. This approach is promising and could potentially be applied in the context of MI-based BMI control of a robotic arm or a neural prosthesis for motor disabled patients with highly impaired upper limb.

Background Noise Filtering and Distribution Dividing for Crowd Counting

Crowd counting is a challenging problem due to the diverse crowd distribution and background interference. In this paper, we propose a new approach for head size estimation to reduce the impact of different crowd scale and background noise. Different from just using local information of distance between human heads, the global information of the people distribution in the whole image is also under consideration. We obey the order of far- to near-region (small to large) to spread head size, and ensure that the propagation is uninterrupted by inserting dummy head points. The estimated head size is further exploited, such as dividing the crowd into parts of different densities and generating a high-fidelity head mask. On the other hand, we design three different head mask usage mechanisms and the corresponding head masks to analyze where and which mask could lead to better background filtering1. Based on the learned masks, two competitive models are proposed which can perform robust crowd estimation against background noise and diverse crowd scale. We evaluate the proposed method on three public crowd counting datasets of ShanghaiTech [2], UCFQNRF [3] and UCFCC_50 [4]. Experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art crowd counting approaches.

Counting Crowds with Perspective Distortion Correction via Adaptive Learning

The goal of crowd counting is to estimate the number of people in the image. Presently, use regression to count people number became a mainstream method. It is worth noting that, with the development of convolutional neural networks (CNN), methods that are based on CNN have become a research hotspot. It is a more interesting topic that how to locate the site of the person in the image than simply predicting the number of people in the image. The perspective transformation present is still a challenge, because perspective distortion will cause differences in the size of the crowd in the image. To devote perspective distortion and locate the site of the person more accuracy, we design a novel framework named Adaptive Learning Network (CAL). We use the VGG as the backbone. After each pooling layer is output, we collect the 1/2, 1/4, 1/8, and 1/16 features of the original image and combine them with the weights learned by an adaptive learning branch. The object of our adaptive learning branch is each image in the datasets. By combining the output features of different sizes of each image, the challenge of drastic changes in the size of the image crowd due to perspective transformation is reduced. We conducted experiments on four population counting data sets (i.e., ShanghaiTech Part A, ShanghaiTech Part B, UCF_CC_50 and UCF-QNRF), and the results show that our model has a good performance.

Optimal Transport in Reproducing Kernel Hilbert Spaces: Theory and Applications

In this paper, we present a mathematical and computational framework for comparing and matching distributions in reproducing kernel Hilbert spaces (RKHS). This framework, called optimal transport in RKHS, is a generalization of the optimal transport problem in input spaces to (potentially) infinite-dimensional feature spaces. We provide a computable formulation of Kantorovich's optimal transport in RKHS. In particular, we explore the case in which data distributions in RKHS are Gaussian, obtaining closed-form expressions of both the estimated Wasserstein distance and optimal transport map via kernel matrices. Based on these expressions, we generalize the Bures metric on covariance matrices to infinite-dimensional settings, providing a new metric between covariance operators. Moreover, we extend the correlation alignment problem to Hilbert spaces, giving a new strategy for matching distributions in RKHS. Empirically, we apply the derived formulas under the Gaussianity assumption to image classification and domain adaptation. In both tasks, our algorithms yield state-of-the-art performances, demonstrating the effectiveness and potential of our framework.

Batch Mode Active Learning on the Riemannian Manifold for Automated Scoring of Nuclear Pleomorphism in Breast Cancer

Breast cancer is the most prevalent invasive type of cancer among women. The mortality rate of the disease can be reduced considerably through timely prognosis and felicitous treatment planning, by utilizing the computer aided detection and diagnosis techniques. With the advent of whole slide image (WSI) scanners for digitizing the histopathological tissue samples, there is a drastic increase in the availability of digital histopathological images. However, these samples are often unlabeled and hence they need labeling to be done through manual annotations by domain experts and experienced pathologists. But this annotation process required for acquiring high quality large labeled training set for nuclear atypia scoring is a tedious, expensive and time consuming job. Active learning techniques have achieved widespread acceptance in reducing this human effort in annotating the data samples. In this paper, we explore the possibilities of active learning on nuclear pleomorphism scoring over a non-Euclidean framework, the Riemannian manifold. Active learning technique adopted for the cancer grading is in the batch-mode framework, that adaptively identifies the apt batch size along with the batch of instances to be queried, following a submodular optimization framework. Samples for annotation are selected considering the diversity and redundancy between the pair of samples, based on the kernelized Riemannian distance measures such as log-Euclidean metrics and the two Bregman divergences - Stein and Jeffrey divergences. Results of the adaptive Batch Mode Active Learning on the Riemannian metric show a superior performance when compared with the state-of-the-art techniques for breast cancer nuclear pleomorphism scoring, as it makes use of the information from the unlabeled samples.

PaDNet: Pan-Density Crowd Counting

Crowd counting is a highly challenging problem in computer vision and machine learning. Most previous methods have focused on consistent density crowds, i.e., either a sparse or a dense crowd, meaning they performed well in global estimation while neglecting local accuracy. To make crowd counting more useful in the real world, we propose a new perspective, named pan-density crowd counting, which aims to count people in varying density crowds. Specifically, we propose the Pan-Density Network (PaDNet) which is composed of the following critical components. First, the Density-Aware Network (DAN) contains multiple subnetworks pretrained on scenarios with different densities. This module is capable of capturing pandensity information. Second, the Feature Enhancement Layer (FEL) effectively captures the global and local contextual features and generates a weight for each density-specific feature. Third, the Feature Fusion Network (FFN) embeds spatial context and fuses these density-specific features. Further, the metrics Patch MAE (PMAE) and Patch RMSE (PRMSE) are proposed to better evaluate the performance on the global and local estimations. Extensive experiments on four crowd counting benchmark datasets, the ShanghaiTech, the UCF-CC-50, the UCSD, and the UCFQNRF, indicate that PaDNet achieves state-of-the-art recognition performance and high robustness in pan-density crowd counting.

One Shot Crowd Counting with Deep Scale Adaptive Neural Network

This paper aims to utilize the deep learning architecture to break through the limitations of camera perspective, image background, uneven crowd density distribution and pedestrian occlusion to estimate crowd density accurately. In this paper, we proposed a new neural network called Deep Scale-Adaptive Convolutional Neural Network (DSA-CNN), which can convert a single crowd image to density map for crowd counting directly. For a crowd image with any size and resolution, our algorithm can output the density map of the crowd image by end-to-end method and finally estimate the number of the crowd in the image. The proposed DSA-CNN consists of two parts: the seven layers CNN network structure and DSA modules. In order to ensure the proposed method is robust to camera perspective effect, DSA-CNN has adopted different sizes of filters in the network and combines them ingeniously. In order to reduce the depth of the data to increase the speed of training, the proposed method utilized 1 × 1 filter in DSA module. To validate the effectiveness of the proposed model, we conducted comparative experiments on four popular public datasets (ShanghiTech dataset, UCF_CC_50 dataset, WorldExpo’10 dataset and UCSD dataset). We compare the proposed method with other well-known algorithms on the MAE and MSE indicators, such as MCNN, Switching-CNN, CSRNet, CP-CNN and Cascaded-MTL. Experimental results show that the proposed method has excellent performance. In addition, we found that the proposed model is easily trained, which further increases the usability of the proposed model.

Multiscale Superpixelwise Locality Preserving Projection for Hyperspectral Image Classification

Manifold learning is a powerful dimensionality reduction tool for a hyperspectral image (HSI) classification to relieve the curse of dimensionality and to reveal the intrinsic low-dimensional manifold. However, a specific characteristic of HSIs, i.e., irregular spatial dependency, is not taken into consideration in the method design, which can yield many spatially homogenous subregions in an HSI scence. Conventional manifold learning methods, such as a locality preserving projection (LPP), pursue a unified projection on the entire HSI, while neglecting the local homogeneities on the HSI manifold caused by those spatially homogenous subregions. In this work, we propose a novel multiscale superpixelwise LPP (MSuperLPP) for HSI classification to overcome the challenge. First, we partition an HSI into homogeneous subregions with a multiscale superpixel segmentation. Then, on each scale, subregion specific LPPs and the associated preliminary classifications are performed. Finally, we aggregate the classification results from all scales using a decision fusion strategy to achieve the final result. Experimental results on three real hyperspectral data sets validate the effectiveness of our method.

Classification of Marine Vessels with Multi-Feature Structure Fusion

The classification of marine vessels is one of the important problems of maritime traffic. To fully exploit the complementarity between different features and to more effectively identify marine vessels, a novel feature structure fusion method based on spectral regression discriminant analysis (SF-SRDA) was proposed. Firstly, we selected the different convolutional neural network features that better describe the characteristics of ships, and constructed the features based on graphs by the similarity metric. Then we weighed the concatenate multi-feature and fused their structures according to the linear relationship assumption. Finally, we constructed the optimization formula to solve the fusion features and structure by using spectral regression discriminant analyses. Experiments on the VAIS dataset show that the proposed SF-SRDA method can reduce the feature dimension from the original 102,400 dimensions to 5 dimensions, that the classification accuracy of visible images can reach 87.60%, and that that of the infrared image can reach 74.68% at daytime. The experimental results demonstrate that the proposed method can not only extract the optimal features from the original redundant feature space, but also greatly reduce the dimensions of the feature. Furthermore, the classification performance of SF-SRDA also gets a promising result.

A Systematic Evaluation and Benchmark for Person Re-Identification: Features, Metrics, and Datasets

Person re-identification (re-id) is a critical problem in video analytics applications such as security and surveillance. The public release of several datasets and code for vision algorithms has facilitated rapid progress in this area over the last few years. However, directly comparing re-id algorithms reported in the literature has become difficult since a wide variety of features, experimental protocols, and evaluation metrics are employed. In order to address this need, we present an extensive review and performance evaluation of single- and multi-shot re-id algorithms. The experimental protocol incorporates the most recent advances in both feature extraction and metric learning. To ensure a fair comparison, all of the approaches were implemented using a unified code library that includes 11 feature extraction algorithms and 22 metric learning and ranking techniques. All approaches were evaluated using a new large-scale dataset that closely mimics a real-world problem setting, in addition to 16 other publicly available datasets: VIPeR, GRID, CAVIAR, DukeMTMC4ReID, 3DPeS, PRID, V47, WARD, SAIVT-SoftBio, CUHK01, CHUK02, CUHK03, RAiD, iLIDSVID, HDA+, and Market1501. The evaluation codebase and results will be made publicly available for community use.

Sparse Representation Over Learned Dictionaries on the Riemannian Manifold for Automated Grading of Nuclear Pleomorphism in Breast Cancer

Breast cancer is found to be the most pervasive type of cancer among women. Computer aided detection and diagnosis of cancer at the initial stages can increase the chances of recovery and thus reduce the mortality rate through timely prognosis and adequate treatment planning. The nuclear atypia scoring or histopathological breast tumor grading remains to be a challenging problem due to the various artifacts and variabilities introduced during slide preparation and also because of the complexity in the structure of the underlying tissue patterns. Inspired by the success of symmetric positive definite (SPD) matrices in many of the challenging tasks in machine learning and computer vision, a sparse coding and dictionary learning on SPD matrices is proposed in this paper for the breast tumor grading. The proposed covariance-based SPD matrices form a Riemannian manifold and are represented as the sparse combination of Riemannian dictionary atoms. Non-linearity of the SPD manifold is tackled by embedding into the reproducing kernel Hilbert space using kernels derived from log-Euclidean metric, Jeffrey and Stein divergences and compared with the non-kernel-based affine invariant Riemannian metric. The novelty of the work lies in exploiting the kernel approach for the Hilbert space embedding of the Riemannian manifold, that can achieve a better discrimination of the breast cancer tissues, following a sparse representation over learned dictionaries and henceforth it outperforms many of the state-of-the-art algorithms in breast cancer grading in terms of quantitative and qualitative analysis.

Can a Subjective Questionnaire Be Used as Brain-Computer Interface Performance Predictor?

Predicting a subject's ability to use a Brain Computer Interface (BCI) is one of the major issues in the BCI domain. Relevant applications of forecasting BCI performance include the ability to adapt the BCI to the needs and expectations of the user, assessing the efficiency of BCI use in stroke rehabilitation, and finally, homogenizing a research population. A limited number of recent studies have proposed the use of subjective questionnaires, such as the Motor Imagery Questionnaire Revised-Second Edition (MIQ-RS). However, further research is necessary to confirm the effectiveness of this type of subjective questionnaire as a BCI performance estimation tool. In this study we aim to answer the following questions: can the MIQ-RS be used to estimate the performance of an MI-based BCI? If not, can we identify different markers that could be used as performance estimators? To answer these questions, we recorded EEG signals from 35 healthy volunteers during BCI use. The subjects had previously completed the MIQ-RS questionnaire. We conducted an offline analysis to assess the correlation between the questionnaire scores related to Kinesthetic and Motor imagery tasks and the performances of four classification methods. Our results showed no significant correlation between BCI performance and the MIQ-RS scores. However, we reveal that BCI performance is correlated to habits and frequency of practicing manual activities.

Cross Euclidean-to-Riemannian Metric Learning with Application to Face Recognition from Video

Riemannian manifolds have been widely employed for video representations in visual classification tasks including video-based face recognition. The success mainly derives from learning a discriminant Riemannian metric which encodes the non-linear geometry of the underlying Riemannian manifolds. In this paper, we propose a novel metric learning framework to learn a distance metric across a Euclidean space and a Riemannian manifold to fuse average appearance and pattern variation of faces within one video. The proposed metric learning framework can handle three typical tasks of video-based face recognition: Video-to-Still, Still-to-Video and Video-to-Video settings. To accomplish this new framework, by exploiting typical Riemannian geometries for kernel embedding, we map the source Euclidean space and Riemannian manifold into a common Euclidean subspace, each through a corresponding high-dimensional Reproducing Kernel Hilbert Space (RKHS). With this mapping, the problem of learning a cross-view metric between the two source heterogeneous spaces can be converted to learning a single-view Euclidean distance metric in the target common Euclidean space. By learning information on heterogeneous data with the shared label, the discriminant metric in the common space improves face recognition from videos. Extensive experiments on four challenging video face databases demonstrate that the proposed framework has a clear advantage over the state-of-the-art methods in the three classical video-based face recognition scenarios.

A Comprehensive Look at Coding Techniques on Riemannian Manifolds

Core to many learning pipelines is visual recognition such as image and video classification. In such applications, having a compact yet rich and informative representation plays a pivotal role. An underlying assumption in traditional coding schemes [e.g., sparse coding (SC)] is that the data geometrically comply with the Euclidean space. In other words, the data are presented to the algorithm in vector form and Euclidean axioms are fulfilled. This is of course restrictive in machine learning, computer vision, and signal processing, as shown by a large number of recent studies. This paper takes a further step and provides a comprehensive mathematical framework to perform coding in curved and non-Euclidean spaces, i.e., Riemannian manifolds. To this end, we start by the simplest form of coding, namely, bag of words. Then, inspired by the success of vector of locally aggregated descriptors in addressing computer vision problems, we will introduce its Riemannian extensions. Finally, we study Riemannian form of SC, locality-constrained linear coding, and collaborative coding. Through rigorous tests, we demonstrate the superior performance of our Riemannian coding schemes against the state-of-the-art methods on several visual classification tasks, including head pose classification, video-based face recognition, and dynamic scene recognition.

Jointly Learning Deep Features, Deformable Parts, Occlusion and Classification for Pedestrian Detection

Feature extraction, deformation handling, occlusion handling, and classification are four important components in pedestrian detection. Existing methods learn or design these components either individually or sequentially. The interaction among these components is not yet well explored. This paper proposes that they should be jointly learned in order to maximize their strengths through cooperation. We formulate these four components into a joint deep learning framework and propose a new deep network architecture (Code available on www.ee.cuhk.edu.hk/wlouyang/projects/ouyangWiccv13Joint/index.html). By establishing automatic, mutual interaction among components, the deep model has average miss rate 8.57 percent/11.71 percent on the Caltech benchmark dataset with new/original annotations.

RAPID: Measuring Deformation of Biological Tissues from MR Images Through the Riemannian Pseudo Kernel

Due to the nonlinear deformation of nonrigid and nonuniform tissues, it is challenging to accurately measure the displacements of feature points distributed on the inner parts, boundaries, and separatrices of tissue layers. To address this challenge, we propose a feature point matching technique called RAPID to measure MR 2D slice deformation of nonuniform and nonrigid biological tissues. We propose to use the covariance of several neighboring point statistics computed around a keypoint, as the keypoint descriptor. Inspired by the kernel methods, we advocate adopting a Riemannian pseudo kernel to map SPD matrices to a high dimensional Hilbert space, where the Euclidean geometry applies. We compare our RAPID with two existing schemes (i.e., SIFT and SURF). Our experimental results show that our RAPID is superior to SIFT and SURF, because the benefits offered by RAPID are two-fold. First, our RAPID increases the number of matched data points. Second, RAPID substantially improves the key-point matching accuracy of SIFT and SURF.

Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methods

Representing images and videos with Symmetric Positive Definite (SPD) matrices, and considering the Riemannian geometry of the resulting space, has been shown to yield high discriminative power in many visual recognition tasks. Unfortunately, computation on the Riemannian manifold of SPD matrices -especially of high-dimensional ones- comes at a high cost that limits the applicability of existing techniques. In this paper, we introduce algorithms able to handle high-dimensional SPD matrices by constructing a lower-dimensional SPD manifold. To this end, we propose to model the mapping from the high-dimensional SPD manifold to the low-dimensional one with an orthonormal projection. This lets us formulate dimensionality reduction as the problem of finding a projection that yields a low-dimensional manifold either with maximum discriminative power in the supervised scenario, or with maximum variance of the data in the unsupervised one. We show that learning can be expressed as an optimization problem on a Grassmann manifold and discuss fast solutions for special cases. Our evaluation on several classification tasks evidences that our approach leads to a significant accuracy gain over state-of-the-art methods.

Human Motion Segmentation via Robust Kernel Sparse Subspace Clustering

Studies on human motion have attracted a lot of attentions. Human motion capture data, which much more precisely records human motion than videos do, has been widely used in many areas. Motion segmentation is an indispensable step for many related applications, but current segmentation methods for motion capture data do not effectively model some important characteristics of motion capture data, such as Riemannian manifold structure and containing non-Gaussian noise. In this paper, we convert the segmentation of motion capture data into a temporal subspace clustering problem. Under the framework of sparse subspace clustering, we propose to use the geodesic exponential kernel to model the Riemannian manifold structure, use correntropy to measure the reconstruction error, use the triangle constraint to guarantee temporal continuity in each cluster and use multi-view reconstruction to extract the relations between different joints. Therefore, exploiting some special characteristics of motion capture data, we propose a new segmentation method, which is robust to non-Gaussian noise, since correntropy is a localized similarity measure. We also develop an efficient optimization algorithm based on block coordinate descent method to solve the proposed model. Our optimization algorithm has a linear complexity while sparse subspace clustering is originally a quadratic problem. Extensive experiment results both on simulated noisy data set and real noisy data set demonstrate the advantage of the proposed method.Studies on human motion have attracted a lot of attentions. Human motion capture data, which much more precisely records human motion than videos do, has been widely used in many areas. Motion segmentation is an indispensable step for many related applications, but current segmentation methods for motion capture data do not effectively model some important characteristics of motion capture data, such as Riemannian manifold structure and containing non-Gaussian noise. In this paper, we convert the segmentation of motion capture data into a temporal subspace clustering problem. Under the framework of sparse subspace clustering, we propose to use the geodesic exponential kernel to model the Riemannian manifold structure, use correntropy to measure the reconstruction error, use the triangle constraint to guarantee temporal continuity in each cluster and use multi-view reconstruction to extract the relations between different joints. Therefore, exploiting some special characteristics of motion capture data, we propose a new segmentation method, which is robust to non-Gaussian noise, since correntropy is a localized similarity measure. We also develop an efficient optimization algorithm based on block coordinate descent method to solve the proposed model. Our optimization algorithm has a linear complexity while sparse subspace clustering is originally a quadratic problem. Extensive experiment results both on simulated noisy data set and real noisy data set demonstrate the advantage of the proposed method.

Pedestrian Detection with Semantic Regions of Interest

For many pedestrian detectors, background vs. foreground errors heavily influence the detection quality. Our main contribution is to design semantic regions of interest that extract the foreground target roughly to reduce the background vs. foreground errors of detectors. First, we generate a pedestrian heat map from the input image with a full convolutional neural network trained on the Caltech Pedestrian Dataset. Next, semantic regions of interest are extracted from the heat map by morphological image processing. Finally, the semantic regions of interest divide the whole image into foreground and background to assist the decision-making of detectors. We test our approach on the Caltech Pedestrian Detection Benchmark. With the help of our semantic regions of interest, the effects of the detectors have varying degrees of improvement. The best one exceeds the state-of-the-art.

CSP-TSM: Optimizing the performance of Riemannian tangent space mapping using common spatial pattern for MI-BCI

BACKGROUND

Classification of electroencephalography (EEG) signals for motor imagery based brain computer interface (MI-BCI) is an exigent task and common spatial pattern (CSP) has been extensively explored for this purpose. In this work, we focused on developing a new framework for classification of EEG signals for MI-BCI.

METHOD

We propose a single band CSP framework for MI-BCI that utilizes the concept of tangent space mapping (TSM) in the manifold of covariance matrices. The proposed method is named CSP-TSM. Spatial filtering is performed on the bandpass filtered MI EEG signal. Riemannian tangent space is utilized for extracting features from the spatial filtered signal. The TSM features are then fused with the CSP variance based features and feature selection is performed using Lasso. Linear discriminant analysis (LDA) is then applied to the selected features and finally classification is done using support vector machine (SVM) classifier.

RESULTS

The proposed framework gives improved performance for MI EEG signal classification in comparison with several competing methods. Experiments conducted shows that the proposed framework reduces the overall classification error rate for MI-BCI by 3.16%, 5.10% and 1.70% (for BCI Competition III dataset IVa, BCI Competition IV Dataset I and BCI Competition IV Dataset IIb, respectively) compared to the conventional CSP method under the same experimental settings.

CONCLUSION

The proposed CSP-TSM method produces promising results when compared with several competing methods in this paper. In addition, the computational complexity is less compared to that of TSM method. Our proposed CSP-TSM framework can be potentially used for developing improved MI-BCI systems.

Pedestrian Detection Algorithm Based on Improved Convolutional Neural Network

This paper proposes a new multi-feature detection method of local pedestrian based on a convolutional neural network (CNN), which provides a reliable basis for multi-feature fusion in pedestrian detection. According to the standard of pedestrian detection ratio, the pedestrian under the detection window would be segmented, using the sample labels to guide the local characteristics of CNN learning, the supervised learning after the network can obtain the local feature fusion more pedestrian description ability. Finally, a large number of experiments have been performed. The experimental results show that the local features of the neural network are better than those of most pedestrian features and combination features.