Low-rank analysis-synthesis dictionary learning with adaptively ordinal locality

Randomized algorithms for large-scale dictionary learning

Dictionary learning is an important sparse representation algorithm which has been widely used in machine learning and artificial intelligence. However, for massive data in the big data era, classical dictionary learning algorithms are computationally expensive and even can be infeasible. To overcome this difficulty, we propose new dictionary learning methods based on randomized algorithms. The contributions of this work are as follows. First, we find that dictionary matrix is often numerically low-rank. Based on this property, we apply randomized singular value decomposition (RSVD) to the dictionary matrix, and propose a randomized algorithm for linear dictionary learning. Compared with the classical K-SVD algorithm, an advantage is that one can update all the elements of the dictionary matrix simultaneously. Second, to the best of our knowledge, there are few theoretical results on why one can solve the involved matrix computation problems inexactly in dictionary learning. To fill-in this gap, we show the rationality of this randomized algorithm with inexact solving, from a matrix perturbation analysis point of view. Third, based on the numerically low-rank property and Nyström approximation of the kernel matrix, we propose a randomized kernel dictionary learning algorithm, and establish the distance between the exact solution and the computed solution, to show the effectiveness of the proposed randomized kernel dictionary learning algorithm. Fourth, we propose an efficient scheme for the testing stage in kernel dictionary learning. By using this strategy, there is no need to form nor store kernel matrices explicitly both in the training and the testing stages. Comprehensive numerical experiments are performed on some real-world data sets. Numerical results demonstrate the rationality of our strategies, and show that the proposed algorithms are much efficient than some state-of-the-art dictionary learning algorithms. The MATLAB codes of the proposed algorithms are publicly available from https://github.com/Jiali-yang/RALDL_RAKDL.

Structure-aware contrastive hashing for unsupervised cross-modal retrieval

Cross-modal hashing has attracted a lot of attention and achieved remarkable success in large-scale cross-media similarity retrieval applications because of its superior computational efficiency and low storage overhead. However, constructing similarity relationship among samples in cross-modal unsupervised hashing is challenging because of the lack of manual annotation. Most existing unsupervised methods directly use the representations extracted from the backbone of their respective modality to construct instance similarity matrices, leading to inaccurate similarity matrices and resulting in suboptimal hash codes. To address this issue, a novel unsupervised hashing model, named Structure-aware Contrastive Hashing for Unsupervised Cross-modal Retrieval (SACH), is proposed in this paper. Specifically, we concurrently employ both high-dimensional representations and discriminative representations learned by the network to construct a more informative semantic correlative matrix across modalities. Moreover, we design a multimodal structure-aware alignment network to minimize heterogeneous gap in the high-order semantic space of each modality, effectively reducing disparities within heterogeneous data sources and enhancing the consistency of semantic information across modalities. Extensive experimental results on two widely utilized datasets demonstrate the superiority of our proposed SACH method in cross-modal retrieval tasks over existing state-of-the-art methods.

Discriminative analysis dictionary learning with adaptively ordinal locality preserving

Dictionary learning has found broad applications in signal and image processing. By adding constraints to the traditional dictionary learning model, dictionaries with discriminative capability can be obtained which can deal with the task of image classification. The Discriminative Convolutional Analysis Dictionary Learning (DCADL) algorithm proposed recently has achieved promising results with low computational complexity. However, DCADL is still limited in classification performance because of the lack of constraints on dictionary structures. To solve this problem, this study introduces an adaptively ordinal locality preserving (AOLP) term to the original model of DCADL to further improve the classification performance. With the AOLP term, the distance ranking in the neighborhood of each atom can be preserved, which can improve the discrimination of coding coefficients. In addition, a linear classifier for the classification of coding coefficients is trained along with the dictionary. A new method is designed specifically to solve the optimization problem corresponding to the proposed model. Experiments are performed on several commonly used datasets to show the promising results of the proposed algorithm in classification performance and computational efficiency.

Relaxed Block-Diagonal Dictionary Pair Learning With Locality Constraint for Image Recognition

We propose a novel structured analysis-synthesis dictionary pair learning method for efficient representation and image classification, referred to as relaxed block-diagonal dictionary pair learning with a locality constraint (RBD-DPL). RBD-DPL aims to learn relaxed block-diagonal representations of the input data to enhance the discriminability of both analysis and synthesis dictionaries by dynamically optimizing the block-diagonal components of representation, while the off-block-diagonal counterparts are set to zero. In this way, the learned synthesis subdictionary is allowed to be more flexible in reconstructing the samples from the same class, and the analysis dictionary effectively transforms the original samples into a relaxed coefficient subspace, which is closely associated with the label information. Besides, we incorporate a locality-constraint term as a complement of the relaxation learning to enhance the locality of the analytical encoding so that the learned representation exhibits high intraclass similarity. A linear classifier is trained in the learned relaxed representation space for consistent classification. RBD-DPL is computationally efficient because it avoids both the use of class-specific complementary data matrices to learn discriminative analysis dictionary, as well as the time-consuming l₁/l₀ -norm sparse reconstruction process. The experimental results demonstrate that our RBD-DPL achieves at least comparable or better recognition performance than the state-of-the-art algorithms. Moreover, both the training and testing time are significantly reduced, which verifies the efficiency of our method. The MATLAB code of the proposed RBD-DPL is available at https://github.com/chenzhe207/RBD-DPL.