Dimension Reduction for Non-Gaussian Data by Adaptive Discriminative Analysis

Numerically stable locality-preserving partial least squares discriminant analysis for efficient dimensionality reduction and classification of high-dimensional data

Dimensionality reduction plays a pivotal role in preparing high-dimensional data for classification and discrimination tasks by eliminating redundant features and enhancing the efficiency of classifiers. The effectiveness of a dimensionality reduction algorithm hinges on its numerical stability. When data projections are numerically stable, they lead to enhanced class separability in the lower-dimensional embedding, consequently yielding higher classification accuracy. This paper investigates the numerical attributes of dimensionality reduction and discriminant subspace learning, with a specific focus on Locality-Preserving Partial Least Squares Discriminant Analysis (LPPLS-DA). High-dimensional data frequently introduce singularity in the scatter matrices, posing a significant challenge. To tackle this issue, the paper explores two robust implementations of LPPLS-DA. These approaches not only optimize data projections but also capture more discriminative features, resulting in a marked improvement in classification accuracy. Empirical evidence supports these findings through numerical experiments conducted on synthetic and spectral datasets. The results demonstrate the superior performance of the proposed methods when compared to several state-of-the-art dimensionality reduction techniques in terms of both classification accuracy and dimension reduction.

Multiscale Laplacian Learning

Machine learning has greatly influenced many fields, including science. However, despite of the tremendous accomplishments of machine learning, one of the key limitations of most existing machine learning approaches is their reliance on large labeled sets, and thus, data with limited labeled samples remains a challenge. Moreover, the performance of machine learning methods often severely hindered in case of diverse data, usually associated with smaller data sets or data associated with areas of study where the size of the data sets is constrained by high experimental cost and/or ethics. These challenges call for innovative strategies for dealing with these types of data. In this work, the aforementioned challenges are addressed by integrating graph-based frameworks, semi-supervised techniques, multiscale structures, and modified and adapted optimization procedures. This results in two innovative multiscale Laplacian learning (MLL) approaches for machine learning tasks, such as data classification, and for tackling data with limited samples, diverse data, and small data sets. The first approach, multikernel manifold learning (MML), integrates manifold learning with multikernel information and incorporates a warped kernel regularizer using multiscale graph Laplacians. The second approach, the multiscale MBO (MMBO) method, introduces multiscale Laplacians to the modification of the famous classical Merriman-Bence-Osher (MBO) scheme, and makes use of fast solvers. We demonstrate the performance of our algorithms experimentally on a variety of benchmark data sets, and compare them favorably to the state-of-art approaches.

Joint Sparse Locality-Aware Regression for Robust Discriminative Learning

With the dramatic increase of dimensions in the data representation, extracting latent low-dimensional features becomes of the utmost importance for efficient classification. Aiming at the problems of weakly discriminating marginal representation and difficulty in revealing the data manifold structure in most of the existing linear discriminant methods, we propose a more powerful discriminant feature extraction framework, namely, joint sparse locality-aware regression (JSLAR). In our model, we formulate a new strategy induced by the nonsquared L₂ norm for enhancing the local intraclass compactness of the data manifold, which can achieve the joint learning of the locality-aware graph structure and the desirable projection matrix. Besides, we formulate a weighted retargeted regression to perform the marginal representation learning adaptively instead of using the general average interclass margin. To alleviate the disturbance of outliers and prevent overfitting, we measure the regression term and locality-aware term together with the regularization term by forcing the row sparsity with the joint L_2,1 norms. Then, we derive an effective iterative algorithm for solving the proposed model. The experimental results over a range of benchmark databases demonstrate that the proposed JSLAR outperforms some state-of-the-art approaches.

Locality Adaptive Discriminant Analysis Framework

Linear discriminant analysis (LDA) is a well-known technique for supervised dimensionality reduction and has been extensively applied in many real-world applications. LDA assumes that the samples are Gaussian distributed, and the local data distribution is consistent with the global distribution. However, real-world data seldom satisfy this assumption. To handle the data with complex distributions, some methods emphasize the local geometrical structure and perform discriminant analysis between neighbors. But the neighboring relationship tends to be affected by the noise in the input space. In this research, we propose a new supervised dimensionality reduction method, namely, locality adaptive discriminant analysis (LADA). In order to directly process the data with matrix representation, such as images, the 2-D LADA (2DLADA) is also developed. The proposed methods have the following salient properties: 1) they find the principle projection directions without imposing any assumption on the data distribution; 2) they explore the data relationship in the desired subspace, which contains less noise; and 3) they find the local data relationship automatically without the efforts for tuning parameters. The performance of dimensionality reduction shows the superiorities of the proposed methods over the state of the art.

An efficient fuzzy deep learning approach to recognize 2D faces using FADF and ResNet-164 architecture

Deep learning using fuzzy is highly modular and more accurate. Adaptive Fuzzy Anisotropy diffusion filter (FADF) is used to remove noise from the image while preserving edges, lines and improve smoothing effects. By detecting edge and noise information through pre-edge detection using fuzzy contrast enhancement, post-edge detection using fuzzy morphological gradient filter and noise detection technique. Convolution Neural Network (CNN) ResNet-164 architecture is used for automatic feature extraction. The resultant feature vectors are classified using ANFIS deep learning. Top-1 error rate is reduced from 21.43% to 18.8%. Top-5 error rate is reduced to 2.68%. The proposed work results in high accuracy rate with low computation cost. The recognition rate of 99.18% and accuracy of 98.24% is achieved on standard dataset. Compared to the existing techniques the proposed work outperforms in all aspects. Experimental results provide better result than the existing techniques on FACES 94, Feret, Yale-B, CMU-PIE, JAFFE dataset and other state-of-art dataset.

Generalized two-dimensional linear discriminant analysis with regularization

Recent advances show that two-dimensional linear discriminant analysis (2DLDA) is a successful matrix based dimensionality reduction method. However, 2DLDA may encounter the singularity issue theoretically, and also is sensitive to outliers. In this paper, a generalized Lp-norm 2DLDA framework with regularization for an arbitrary p>0 is proposed, named G2DLDA. There are mainly two contributions of G2DLDA: one is G2DLDA model uses an arbitrary Lp-norm to measure the between-class and within-class scatter, and hence a proper p can be selected to achieve robustness. The other one is that the introduced regularization term makes G2DLDA enjoy better generalization performance and avoid singularity. In addition, an effective learning algorithm is designed for G2LDA, which can be solved through a series of convex problems with closed-form solutions. Its convergence can be guaranteed theoretically when 1≤p≤2. Preliminary experimental results on three contaminated human face databases show the effectiveness of the proposed G2DLDA.

Outdoor Shadow Estimating Using Multiclass Geometric Decomposition Based on BLS

Illumination is a significant component of an image, and illumination estimation of an outdoor scene from given images is still challenging yet it has wide applications. Most of the traditional illumination estimating methods require prior knowledge or fixed objects within the scene, which makes them often limited by the scene of a given image. We propose an optimization approach that integrates the multiclass cues of the image(s) [a main input image and optional auxiliary input image(s)]. First, Sun visibility is estimated by the efficient broad learning system. And then for the scene with visible Sun, we classify the information in the image by the proposed classification algorithm, which combines the geometric information and shadow information to make the most of the information. And we apply a respective algorithm for every class to estimate the illumination parameters. Finally, our approach integrates all of the estimating results by the Markov random field. We make full use of the cues in the given image instead of an extra requirement for the scene, and the qualitative results are presented and show that our approach outperformed other methods with similar conditions.