Fast discrete cross-modal hashing with semantic consistency

Structure-aware contrastive hashing for unsupervised cross-modal retrieval

Cross-modal hashing has attracted a lot of attention and achieved remarkable success in large-scale cross-media similarity retrieval applications because of its superior computational efficiency and low storage overhead. However, constructing similarity relationship among samples in cross-modal unsupervised hashing is challenging because of the lack of manual annotation. Most existing unsupervised methods directly use the representations extracted from the backbone of their respective modality to construct instance similarity matrices, leading to inaccurate similarity matrices and resulting in suboptimal hash codes. To address this issue, a novel unsupervised hashing model, named Structure-aware Contrastive Hashing for Unsupervised Cross-modal Retrieval (SACH), is proposed in this paper. Specifically, we concurrently employ both high-dimensional representations and discriminative representations learned by the network to construct a more informative semantic correlative matrix across modalities. Moreover, we design a multimodal structure-aware alignment network to minimize heterogeneous gap in the high-order semantic space of each modality, effectively reducing disparities within heterogeneous data sources and enhancing the consistency of semantic information across modalities. Extensive experimental results on two widely utilized datasets demonstrate the superiority of our proposed SACH method in cross-modal retrieval tasks over existing state-of-the-art methods.

Large-Scale Cross-Modal Hashing with Unified Learning and Multi-Object Regional Correlation Reasoning

To explore the rich information contained in multi-modal data and take into account efficiency, deep cross-modal hash retrieval (DCMHR) is a wise solution. But currently, most DCMHR methods have two key limitations, one is that the recommended classification of DCMHR models is conditioned only on the objects in different regions, respectively. Another flaw is that these methods either do not learn the unified hash codes in training or cannot design an efficient training process. To solve these two problems, this paper designs Large-Scale Cross-Modal Hashing with Unified Learning and Multi-Object Regional Correlation Reasoning (HUMOR). For the proposed related labels classified by ImgNet, HUMOR uses Multiple Instance Learning (MIL) to reason the correlation of these labels. When regional correlation reasoning is low, these labels will be through "reduce-add" to rectification from max-to-min (global precedence) or min-to-max (regional precedence). Then, HUMOR conducts unified learning on hash loss and classification loss, adopts the four-step iterative algorithm to optimize the unified hash codes, and reduces bias in the model. Experiments on two baseline datasets show that the average performance of this method is higher than most of the DCMHR methods. The results demonstrate the effectiveness and innovation of our method.

Joint Specifics and Consistency Hash Learning for Large-Scale Cross-Modal Retrieval

With the dramatic increase in the amount of multimedia data, cross-modal similarity retrieval has become one of the most popular yet challenging problems. Hashing offers a promising solution for large-scale cross-modal data searching by embedding the high-dimensional data into the low-dimensional similarity preserving Hamming space. However, most existing cross-modal hashing usually seeks a semantic representation shared by multiple modalities, which cannot fully preserve and fuse the discriminative modal-specific features and heterogeneous similarity for cross-modal similarity searching. In this paper, we propose a joint specifics and consistency hash learning method for cross-modal retrieval. Specifically, we introduce an asymmetric learning framework to fully exploit the label information for discriminative hash code learning, where 1) each individual modality can be better converted into a meaningful subspace with specific information, 2) multiple subspaces are semantically connected to capture consistent information, and 3) the integration complexity of different subspaces is overcome so that the learned collaborative binary codes can merge the specifics with consistency. Then, we introduce an alternatively iterative optimization to tackle the specifics and consistency hashing learning problem, making it scalable for large-scale cross-modal retrieval. Extensive experiments on five widely used benchmark databases clearly demonstrate the effectiveness and efficiency of our proposed method on both one-cross-one and one-cross-two retrieval tasks.