Scatter balance: an angle-based supervised dimensionality reduction

Joint Sparse Locality-Aware Regression for Robust Discriminative Learning

With the dramatic increase of dimensions in the data representation, extracting latent low-dimensional features becomes of the utmost importance for efficient classification. Aiming at the problems of weakly discriminating marginal representation and difficulty in revealing the data manifold structure in most of the existing linear discriminant methods, we propose a more powerful discriminant feature extraction framework, namely, joint sparse locality-aware regression (JSLAR). In our model, we formulate a new strategy induced by the nonsquared L₂ norm for enhancing the local intraclass compactness of the data manifold, which can achieve the joint learning of the locality-aware graph structure and the desirable projection matrix. Besides, we formulate a weighted retargeted regression to perform the marginal representation learning adaptively instead of using the general average interclass margin. To alleviate the disturbance of outliers and prevent overfitting, we measure the regression term and locality-aware term together with the regularization term by forcing the row sparsity with the joint L_2,1 norms. Then, we derive an effective iterative algorithm for solving the proposed model. The experimental results over a range of benchmark databases demonstrate that the proposed JSLAR outperforms some state-of-the-art approaches.

Multidimensional Balance-Based Cluster Boundary Detection for High-Dimensional Data

The balance of neighborhood space around a central point is an important concept in cluster analysis. It can be used to effectively detect cluster boundary objects. The existing neighborhood analysis methods focus on the distribution of data, i.e., analyzing the characteristic of the neighborhood space from a single perspective, and could not obtain rich data characteristics. In this paper, we analyze the high-dimensional neighborhood space from multiple perspectives. By simulating each dimension of a data point's k nearest neighbors space ( k NNs) as a lever, we apply the lever principle to compute the balance fulcrum of each dimension after proving its inevitability and uniqueness. Then, we model the distance between the projected coordinate of the data point and the balance fulcrum on each dimension and construct the DHBlan coefficient to measure the balance of the neighborhood space. Based on this theoretical model, we propose a simple yet effective cluster boundary detection algorithm called Lever. Experiments on both low- and high-dimensional data sets validate the effectiveness and efficiency of our proposed algorithm.

Norm Discriminant Eigenspace Transform for Pattern Classification

Most of the supervised dimensionality reduction (DR) methods design interclass scatter as the separability between the class means, which may force to assume unimodal Gaussian likelihoods and their projection space trends toward the class means. This paper presents a novel DR approach, norm discriminant eigenspace transform (NDET), in which average norms ( l₂ ) of classes have been utilized to characterize the interclass separability and the within-class distance characterizes the intraclass compactness. NDET is intended to accommodate data distributions that may be multimodal and non-Gaussian. We derive an upper bound for NDET, and a specific solution space to attain this bound. Existence of the specific solution is very unwonted, thereby we have considered the solution space of upper bound to achieve better reduction of dimensionality and discrimination of classes. Also, a nonlinear version of NDET (kernel NDET) is developed to model nonlinear relationships between the features. We show, experimentally (on synthetic data) that NDET effectively overcomes the limitations, which arise due to unimodal and data distribution assumptions of the traditional algorithms. Extensive empirical studies are made; and the proposed method is compared with closely related state-of-the-art schemes on UCI machine learning repository and face recognition data sets, to establish its novelty.