Topology preserving non-negative matrix factorization for face recognition

Correntropy-Induced Discriminative Nonnegative Sparse Coding for Robust Palmprint Recognition

Palmprint recognition has been widely studied for security applications. However, there is a lack of in-depth investigations on robust palmprint recognition. Regression analysis being intuitively interpretable on robustness design inspires us to propose a correntropy-induced discriminative nonnegative sparse coding method for robust palmprint recognition. Specifically, we combine the correntropy metric and l₁-norm to present a powerful error estimator that gains flexibility and robustness to various contaminations by cooperatively detecting and correcting errors. Furthermore, we equip the error estimator with a tailored discriminative nonnegative sparse regularizer to extract significant nonnegative features. We manage to explore an analytical optimization approach regarding this unified scheme and figure out a novel efficient method to address the challenging non-negative constraint. Finally, the proposed coding method is extended for robust multispectral palmprint recognition. Namely, we develop a constrained particle swarm optimizer to search for the feasible parameters to fuse the extracted robust features of different spectrums. Extensive experimental results on both contactless and contact-based multispectral palmprint databases verify the flexibility and robustness of our methods.

Dimension Reduction and Clustering Models for Single-Cell RNA Sequencing Data: A Comparative Study

With recent advances in single-cell RNA sequencing, enormous transcriptome datasets have been generated. These datasets have furthered our understanding of cellular heterogeneity and its underlying mechanisms in homogeneous populations. Single-cell RNA sequencing (scRNA-seq) data clustering can group cells belonging to the same cell type based on patterns embedded in gene expression. However, scRNA-seq data are high-dimensional, noisy, and sparse, owing to the limitation of existing scRNA-seq technologies. Traditional clustering methods are not effective and efficient for high-dimensional and sparse matrix computations. Therefore, several dimension reduction methods have been introduced. To validate a reliable and standard research routine, we conducted a comprehensive review and evaluation of four classical dimension reduction methods and five clustering models. Four experiments were progressively performed on two large scRNA-seq datasets using 20 models. Results showed that the feature selection method contributed positively to high-dimensional and sparse scRNA-seq data. Moreover, feature-extraction methods were able to promote clustering performance, although this was not eternally immutable. Independent component analysis (ICA) performed well in those small compressed feature spaces, whereas principal component analysis was steadier than all the other feature-extraction methods. In addition, ICA was not ideal for fuzzy C-means clustering in scRNA-seq data analysis. K-means clustering was combined with feature-extraction methods to achieve good results.

Multiview Clustering Based on Non-Negative Matrix Factorization and Pairwise Measurements

As we all know, multiview clustering has become a hot topic in machine learning and pattern recognition. Non-negative matrix factorization (NMF) has been one popular tool in multiview clustering due to its competitiveness and interpretation. However, the existing multiview clustering methods based on NMF only consider the similarity of intra-view, while neglecting the similarity of inter-view. In this paper, we propose a novel multiview clustering algorithm, named multiview clustering based on NMF and pairwise measurements, which incorporates pairwise co-regularization and manifold regularization with NMF. In the proposed algorithm, we consider the similarity of the inter-view via pairwise co-regularization to obtain the more compact representation of multiview data space. We can also obtain the part-based representation by NMF and preserve the locally geometrical structure of the data space by utilizing the manifold regularization. Furthermore, we give the theoretical proof that the objective function of the proposed algorithm is convergent for multiview clustering. Experimental results show that the proposed algorithm outperforms the state-of-the-arts for multiview clustering.

Simultaneous Dimensionality Reduction and Classification via Dual Embedding Regularized Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is a well-known paradigm for data representation. Traditional NMF-based classification methods first perform NMF or one of its variants on input data samples to obtain their low-dimensional representations, which are successively classified by means of a typical classifier [e.g., k -nearest neighbors (KNN) and support vector machine (SVM)]. Such a stepwise manner may overlook the dependency between the two processes, resulting in the compromise of the classification accuracy. In this paper, we elegantly unify the two processes by formulating a novel constrained optimization model, namely dual embedding regularized NMF (DENMF), which is semi-supervised. Our DENMF solution simultaneously finds the low-dimensional representations and assignment matrix via joint optimization for better classification. Specifically, input data samples are projected onto a couple of low-dimensional spaces (i.e., feature and label spaces), and locally linear embedding is employed to preserve the identical local geometric structure in different spaces. Moreover, we propose an alternating iteration algorithm to solve the resulting DENMF, whose convergence is theoretically proven. Experimental results over five benchmark datasets demonstrate that DENMF can achieve higher classification accuracy than state-of-the-art algorithms.

Improvement of Face Recognition with Gabor, PCA, and SVM Under Complex Illumination Conditions

Complex illumination condition is one of the most critical challenging problems for practical face recognition. However, numerous studies have had no effective solutions reported for full illumination variation of face images in the facial recognition research field. In order to effectively solve full illumination variation problem, we propose a novel approach for illumination normalization for facial images based on the enhanced contrast method of histogram equalization (HE) and fusion of illumination estimations (FOIE). Then, feature extraction is applied with consideration of both Gabor wavelet and principal component analysis methods to process illumination normalization. Next, a support vector machine classifier (SVM) is used for face classification. Experimental results show that superior performance can be obtained in the developed approach by comparisons with some state-of-the-arts.

Robust Face Recognition Based on a New Supervised Kernel Subspace Learning Method

Face recognition is one of the most popular techniques to achieve the goal of figuring out the identity of a person. This study has been conducted to develop a new non-linear subspace learning method named “supervised kernel locality-based discriminant neighborhood embedding,” which performs data classification by learning an optimum embedded subspace from a principal high dimensional space. In this approach, not only nonlinear and complex variation of face images is effectively represented using nonlinear kernel mapping, but local structure information of data from the same class and discriminant information from distinct classes are also simultaneously preserved to further improve final classification performance. Moreover, in order to evaluate the robustness of the proposed method, it was compared with several well-known pattern recognition methods through comprehensive experiments with six publicly accessible datasets. Experiment results reveal that our method consistently outperforms its competitors, which demonstrates strong potential to be implemented in many real-world systems.

Non-Negative Tensor Factorization for Human Behavioral Pattern Mining in Online Games

Multiplayer online battle arena is a genre of online games that has become extremely popular. Due to their success, these games also drew the attention of our research community, because they provide a wealth of information about human online interactions and behaviors. A crucial problem is the extraction of activity patterns that characterize this type of data, in an interpretable way. Here, we leverage the Non-negative Tensor Factorization to detect hidden correlated behaviors of playing in a well-known game: League of Legends. To this aim, we collect the entire gaming history of a group of about 1000 players, which accounts for roughly 100K matches. By applying our framework we are able to separate players into different groups. We show that each group exhibits similar features and playing strategies, as well as similar temporal trajectories, i.e., behavioral progressions over the course of their gaming history. We surprisingly discover that playing strategies are stable over time and we provide an explanation for this observation.

Fast Non-Negative Matrix Factorizations for Face Recognition

Non-negative Matrix Factorization (NMF), as a promising image-data representation approach, encounters the problems of slow convergence and weak classification ability. To overcome these limitations, this paper, based on different error measurements, proposes two kinds of NMF algorithms with fast gradient descent and high discriminant performance. It is shown that the proposed Fast NMF (FNMF) methods have larger step sizes than those of traditional NMFs. Moreover, the traditional NMFs are the special cases of our methods. To further enhance the discriminative power of non-negative features, we exploit our previous block NMF technique and obtain Block FNMF (BFNMF) algorithms, which are supervised decomposition approaches with some good properties, such as the highly sparse features and orthogonal features from different classes. In experiments, both convergence on non-negative decomposition and performance on face recognition (FR) are considered for evaluations. Compared with traditional NMF algorithms and some state-of-the-art methods, experimental results indicate the effective and superior performance of the proposed NMF methods.

On the Performance of Manhattan Nonnegative Matrix Factorization

Extracting low-rank and sparse structures from matrices has been extensively studied in machine learning, compressed sensing, and conventional signal processing, and has been widely applied to recommendation systems, image reconstruction, visual analytics, and brain signal processing. Manhattan nonnegative matrix factorization (MahNMF) is an extension of the conventional NMF, which models the heavy-tailed Laplacian noise by minimizing the Manhattan distance between a nonnegative matrix X and the product of two nonnegative low-rank factor matrices. Fast algorithms have been developed to restore the low-rank and sparse structures of X in the MahNMF. In this paper, we study the statistical performance of the MahNMF in the frame of the statistical learning theory. We decompose the expected reconstruction error of the MahNMF into the estimation error and the approximation error. The estimation error is bounded by the generalization error bounds of the MahNMF, while the approximation error is analyzed using the asymptotic results of the minimum distortion of vector quantization. The generalization error bound is valuable for determining the size of the training sample needed to guarantee a desirable upper bound for the defect between the expected and empirical reconstruction errors. Statistical performance analysis shows how the reduced dimensionality affects the estimation and approximation errors. Our framework can also be used for analyzing the performance of the NMF.

A New System of Face Recognition: Using Fuzziness and Sparsity

In this article, a new human face recognition scheme is proposed. The proposed system is based on the sparsity and fuzziness, and utilizes independent component analysis (ICA). The scheme includes four parts: a fuzzy comprehensive judgment model for estimating whether the information carried by training samples is enough or not, a proper edge extraction operator to discover more hidden information for single image, ICA feature extractor, and a sparse representation model for correlation coefficient calculation to classify testing samples. In view of the intrinsic patterns of gray information distribution of face images, a weighted fuzzy distance for judgement model and cluster analysis is proposed. The new proposed method is tested on ORL, FERET and UMIST databases. The experiment results demonstrate and illustrate the feasibility of the proposed method and the effective performances on recognition rate.

Robust Nonnegative Patch Alignment for Dimensionality Reduction

Dimensionality reduction is an important method to analyze high-dimensional data and has many applications in pattern recognition and computer vision. In this paper, we propose a robust nonnegative patch alignment for dimensionality reduction, which includes a reconstruction error term and a whole alignment term. We use correntropy-induced metric to measure the reconstruction error, in which the weight is learned adaptively for each entry. For the whole alignment, we propose locality-preserving robust nonnegative patch alignment (LP-RNA) and sparsity-preserviing robust nonnegative patch alignment (SP-RNA), which are unsupervised and supervised, respectively. In the LP-RNA, we propose a locally sparse graph to encode the local geometric structure of the manifold embedded in high-dimensional space. In particular, we select large p -nearest neighbors for each sample, then obtain the sparse representation with respect to these neighbors. The sparse representation is used to build a graph, which simultaneously enjoys locality, sparseness, and robustness. In the SP-RNA, we simultaneously use local geometric structure and discriminative information, in which the sparse reconstruction coefficient is used to characterize the local geometric structure and weighted distance is used to measure the separability of different classes. For the induced nonconvex objective function, we formulate it into a weighted nonnegative matrix factorization based on half-quadratic optimization. We propose a multiplicative update rule to solve this function and show that the objective function converges to a local optimum. Several experimental results on synthetic and real data sets demonstrate that the learned representation is more discriminative and robust than most existing dimensionality reduction methods.

Robust Hyperspectral Unmixing With Correntropy-Based Metric

Hyperspectral unmixing is one of the crucial steps for many hyperspectral applications. The problem of hyperspectral unmixing has proved to be a difficult task in unsupervised work settings where the endmembers and abundances are both unknown. In addition, this task becomes more challenging in the case that the spectral bands are degraded by noise. This paper presents a robust model for unsupervised hyperspectral unmixing. Specifically, our model is developed with the correntropy-based metric where the nonnegative constraints on both endmembers and abundances are imposed to keep physical significance. Besides, a sparsity prior is explicitly formulated to constrain the distribution of the abundances of each endmember. To solve our model, a half-quadratic optimization technique is developed to convert the original complex optimization problem into an iteratively reweighted nonnegative matrix factorization with sparsity constraints. As a result, the optimization of our model can adaptively assign small weights to noisy bands and put more emphasis on noise-free bands. In addition, with sparsity constraints, our model can naturally generate sparse abundances. Experiments on synthetic and real data demonstrate the effectiveness of our model in comparison to the related state-of-the-art unmixing models.

Projected gradients for subclass discriminant nonnegative subspace learning

Current discriminant nonnegative matrix factorization (NMF) methods either do not guarantee convergence to a stationary limit point or assume a compact data distribution inside classes, thus ignoring intra class variance in extracting discriminant data samples representations. To address both limitations, we regard that data inside each class has a multimodal distribution, forming various subclasses and perform optimization using a projected gradients framework to ensure limit point stationarity. The proposed method combines appropriate clustering-based discriminant criteria in the NMF decomposition cost function, in order to find discriminant projections that enhance class separability in the reduced dimensional projection space, thus improving classification performance. The developed algorithms have been applied to facial expression, face and object recognition, and experimental results verified that they successfully identified discriminant parts, thus enhancing recognition performance.

Manifold regularized discriminative nonnegative matrix factorization with fast gradient descent

Nonnegative matrix factorization (NMF) has become a popular data-representation method and has been widely used in image processing and pattern-recognition problems. This is because the learned bases can be interpreted as a natural parts-based representation of data and this interpretation is consistent with the psychological intuition of combining parts to form a whole. For practical classification tasks, however, NMF ignores both the local geometry of data and the discriminative information of different classes. In addition, existing research results show that the learned basis is unnecessarily parts-based because there is neither explicit nor implicit constraint to ensure the representation parts-based. In this paper, we introduce the manifold regularization and the margin maximization to NMF and obtain the manifold regularized discriminative NMF (MD-NMF) to overcome the aforementioned problems. The multiplicative update rule (MUR) can be applied to optimizing MD-NMF, but it converges slowly. In this paper, we propose a fast gradient descent (FGD) to optimize MD-NMF. FGD contains a Newton method that searches the optimal step length, and thus, FGD converges much faster than MUR. In addition, FGD includes MUR as a special case and can be applied to optimizing NMF and its variants. For a problem with 165 samples in R(1600), FGD converges in 28 s, while MUR requires 282 s. We also apply FGD in a variant of MD-NMF and experimental results confirm its efficiency. Experimental results on several face image datasets suggest the effectiveness of MD-NMF.

Face recognition using nearest feature space embedding

Face recognition algorithms often have to solve problems such as facial pose, illumination, and expression (PIE). To reduce the impacts, many researchers have been trying to find the best discriminant transformation in eigenspaces, either linear or nonlinear, to obtain better recognition results. Various researchers have also designed novel matching algorithms to reduce the PIE effects. In this study, a nearest feature space embedding (called NFS embedding) algorithm is proposed for face recognition. The distance between a point and the nearest feature line (NFL) or the NFS is embedded in the transformation through the discriminant analysis. Three factors, including class separability, neighborhood structure preservation, and NFS measurement, were considered to find the most effective and discriminating transformation in eigenspaces. The proposed method was evaluated by several benchmark databases and compared with several state-of-the-art algorithms. According to the compared results, the proposed method outperformed the other algorithms.

DISCRIMINATIVE LOCAL LEARNING PROJECTION FOR FACE RECOGNITION

Subspace learning method has commonly been used as a popular way to understand high dimensional data such as face images. In this paper, a novel subspace learning method called Discriminative Local Learning Projection (DLLP) is proposed for face recognition. By characterizing the local structures and dissimilarities between the supervised data manifolds, a linear transformation that can maximize the dissimilarities between all manifolds and simultaneously minimize the local estimation error can be computed. Thus the proposed algorithm embeds the discriminative information as well as the local geometry of samples into the objective function. And the abilities of preserving the local structure in each manifold and classification are both combined into the algorithm. Extensive experiments on face databases demonstrate the effectiveness of DLLP.

Face recognition under varying illumination using gradientfaces

In this correspondence, we propose a novel method to extract illumination insensitive features for face recognition under varying lighting called the Gradientfaces. Theoretical analysis shows Gradientfaces is an illumination insensitive measure, and robust to different illumination, including uncontrolled, natural lighting. In addition, Gradientfaces is derived from the image gradient domain such that it can discover underlying inherent structure of face images since the gradient domain explicitly considers the relationships between neighboring pixel points. Therefore, Gradientfaces has more discriminating power than the illumination insensitive measure extracted from the pixel domain. Recognition rates of 99.83% achieved on PIE database of 68 subjects, 98.96% achieved on Yale B of ten subjects, and 95.61% achieved on Outdoor database of 132 subjects under uncontrolled natural lighting conditions show that Gradientfaces is an effective method for face recognition under varying illumination. Furthermore, the experimental results on Yale database validate that Gradientfaces is also insensitive to image noise and object artifacts (such as facial expressions).

Discriminant nonnegative tensor factorization algorithms

Nonnegative matrix factorization (NMF) has proven to be very successful for image analysis, especially for object representation and recognition. NMF requires the object tensor (with valence more than one) to be vectorized. This procedure may result in information loss since the local object structure is lost due to vectorization. Recently, in order to remedy this disadvantage of NMF methods, nonnegative tensor factorizations (NTF) algorithms that can be applied directly to the tensor representation of object collections have been introduced. In this paper, we propose a series of unsupervised and supervised NTF methods. That is, we extend several NMF methods using arbitrary valence tensors. Moreover, by incorporating discriminant constraints inside the NTF decompositions, we present a series of discriminant NTF methods. The proposed approaches are tested for face verification and facial expression recognition, where it is shown that they outperform other popular subspace approaches.