Large Scale Spectral Clustering Via Landmark-Based Sparse Representation

Efficient and Effective One-Step Multiview Clustering

Multiview clustering algorithms have attracted intensive attention and achieved superior performance in various fields recently. Despite the great success of multiview clustering methods in realistic applications, we observe that most of them are difficult to apply to large-scale datasets due to their cubic complexity. Moreover, they usually use a two-stage scheme to obtain the discrete clustering labels, which inevitably causes a suboptimal solution. In light of this, an efficient and effective one-step multiview clustering (E2OMVC) method is proposed to directly obtain clustering indicators with a small-time burden. Specifically, according to the anchor graphs, the smaller similarity graph of each view is constructed, from which the low-dimensional latent features are generated to form the latent partition representation. By introducing a label discretization mechanism, the binary indicator matrix can be directly obtained from the unified partition representation which is formed by fusing all latent partition representations from different views. In addition, by coupling the fusion of all latent information and the clustering task into a joint framework, the two processes can help each other and obtain a better clustering result. Extensive experimental results demonstrate that the proposed method can achieve comparable or better performance than the state-of-the-art methods. The demo code of this work is publicly available at https://github.com/WangJun2023/EEOMVC.

Fast Multiview Clustering by Optimal Graph Mining

Multiview clustering (MVC) aims to exploit heterogeneous information from different sources and was extensively investigated in the past decade. However, far less attention has been paid to handling large-scale multiview data. In this brief, we fill this gap and propose a fast multiview clustering by an optimal graph mining model to handle large-scale data. We mine a consistent clustering structure from landmark-based graphs of different views, from which the optimal graph based on the one-hot encoding of cluster labels is recovered. Our model is parameter-free, so intractable hyperparameter tuning is avoided. An efficient algorithm of linear complexity to the number of samples is developed to solve the optimization problems. Extensive experiments on real-world datasets of various scales demonstrate the superiority of our proposal.

Deep Spectral Clustering With Constrained Laplacian Rank

Spectral clustering (SC) is a well-performed and prevalent technique for data processing and analysis, which has attracted significant attention in the field of clustering. While the scalability and generalization ability of this method make it prohibitive for the large-scale dataset and the out-of-sample-extension problem. In this work, we propose a new efficient deep clustering architecture based on SC, named deep SC (DSC) with constrained Laplacian rank (DSCCLR). DSCCLR develops a self-adaptive affinity matrix with a clustering-friendly structure by constraining the Laplacian rank, which greatly mines the intrinsic relationships. Meanwhile, by introducing a simple fully connected network with an orthogonality constraint on the last layer, DSCCLR learns discriminative representations in a short training time. The proposed method has the following salient properties: 1) it overcomes limited generalization ability and scalability of the existing DSC methods; 2) it explores the intrinsic relationship between samples in the affinity matrix, which maintains the latent manifold of data as much as possible; and 3) it alleviates the complexity of eigendecomposition via a simple but effective fully connected network. The extensive empirical results demonstrate the superiorities of DSCCLR over other 17 clustering methods.

Discrete and Balanced Spectral Clustering With Scalability

Spectral Clustering (SC) has been the main subject of intensive research due to its remarkable clustering performance. Despite its successes, most existing SC methods suffer from several critical issues. First, they typically involve two independent stages, i.e., learning the continuous relaxation matrix followed by the discretization of the cluster indicator matrix. This two-stage approach can result in suboptimal solutions that negatively impact the clustering performance. Second, these methods are hard to maintain the balance property of clusters inherent in many real-world data, which restricts their practical applicability. Finally, these methods are computationally expensive and hence unable to handle large-scale datasets. In light of these limitations, we present a novel Discrete and Balanced Spectral Clustering with Scalability (DBSC) model that integrates the learning the continuous relaxation matrix and the discrete cluster indicator matrix into a single step. Moreover, the proposed model also maintains the size of each cluster approximately equal, thereby achieving soft-balanced clustering. What's more, the DBSC model incorporates an anchor-based strategy to improve its scalability to large-scale datasets. The experimental results demonstrate that our proposed model outperforms existing methods in terms of both clustering performance and balance performance. Specifically, the clustering accuracy of DBSC on CMUPIE data achieved a 17.93% improvement compared with that of the SOTA methods (LABIN, EBSC, etc.).

ECCA: Efficient Correntropy-Based Clustering Algorithm With Orthogonal Concept Factorization

One of the hottest topics in unsupervised learning is how to efficiently and effectively cluster large amounts of unlabeled data. To address this issue, we propose an orthogonal conceptual factorization (OCF) model to increase clustering effectiveness by restricting the degree of freedom of matrix factorization. In addition, for the OCF model, a fast optimization algorithm containing only a few low-dimensional matrix operations is given to improve clustering efficiency, as opposed to the traditional CF optimization algorithm, which involves dense matrix multiplications. To further improve the clustering efficiency while suppressing the influence of the noises and outliers distributed in real-world data, an efficient correntropy-based clustering algorithm (ECCA) is proposed in this article. Compared with OCF, an anchor graph is constructed and then OCF is performed on the anchor graph instead of directly performing OCF on the original data, which can not only further improve the clustering efficiency but also inherit the advantages of the high performance of spectral clustering. In particular, the introduction of the anchor graph makes ECCA less sensitive to changes in data dimensions and still maintains high efficiency at higher data dimensions. Meanwhile, for various complex noises and outliers in real-world data, correntropy is introduced into ECCA to measure the similarity between the matrix before and after decomposition, which can greatly improve the clustering effectiveness and robustness. Subsequently, a novel and efficient half-quadratic optimization algorithm was proposed to quickly optimize the ECCA model. Finally, extensive experiments on different real-world datasets and noisy datasets show that ECCA can archive promising effectiveness and robustness while achieving tens to thousands of times the efficiency compared with other state-of-the-art baselines.

Spectral Clustering With Adaptive Neighbors for Deep Learning

Spectral clustering is a well-known clustering algorithm for unsupervised learning, and its improved algorithms have been successfully adapted for many real-world applications. However, traditional spectral clustering algorithms are still facing many challenges to the task of unsupervised learning for large-scale datasets because of the complexity and cost of affinity matrix construction and the eigen-decomposition of the Laplacian matrix. From this perspective, we are looking forward to finding a more efficient and effective way by adaptive neighbor assignments for affinity matrix construction to address the above limitation of spectral clustering. It tries to learn an affinity matrix from the view of global data distribution. Meanwhile, we propose a deep learning framework with fully connected layers to learn a mapping function for the purpose of replacing the traditional eigen-decomposition of the Laplacian matrix. Extensive experimental results have illustrated the competitiveness of the proposed algorithm. It is significantly superior to the existing clustering algorithms in the experiments of both toy datasets and real-world datasets.

Tensorized Bipartite Graph Learning for Multi-View Clustering

Despite the impressive clustering performance and efficiency in characterizing both the relationship between the data and cluster structure, most existing graph-based multi-view clustering methods still have the following drawbacks. They suffer from the expensive time burden due to both the construction of graphs and eigen-decomposition of Laplacian matrix. Moreover, none of them simultaneously considers the similarity of inter-view and similarity of intra-view. In this article, we propose a variance-based de-correlation anchor selection strategy for bipartite construction. The selected anchors not only cover the whole classes but also characterize the intrinsic structure of data. Following that, we present a tensorized bipartite graph learning for multi-view clustering (TBGL). Specifically, TBGL exploits the similarity of inter-view by minimizing the tensor Schatten p-norm, which well exploits both the spatial structure and complementary information embedded in the bipartite graphs of views. We exploit the similarity of intra-view by using the [Formula: see text]-norm minimization regularization and connectivity constraint on each bipartite graph. So the learned graph not only well encodes discriminative information but also has the exact connected components which directly indicates the clusters of data. Moreover, we solve TBGL by an efficient algorithm which is time-economical and has good convergence. Extensive experimental results demonstrate that TBGL is superior to the state-of-the-art methods. Codes and datasets are available: https://github.com/xdweixia/TBGL-MVC.

Self-Constrained Spectral Clustering

As a leading graph clustering technique, spectral clustering is one of the most widely used clustering methods to capture complex clusters in data. Some additional prior information can help it to further reduce the difference between its clustering results and users' expectations. However, it is hard to get the prior information under unsupervised scene to guide the clustering process. To solve this problem, we propose a self-constrained spectral clustering algorithm. In this algorithm, we extend the objective function of spectral clustering by adding pairwise and label self-constrained terms to it. We provide the theoretical analysis to show the roles of the self-constrained terms and the extensibility of the proposed algorithm. Based on the new objective function, we build an optimization model for self-constrained spectral clustering so that we can simultaneously learn the clustering results and constraints. Furthermore, we propose an iterative method to solve the new optimization problem. Compared to other existing versions of spectral clustering algorithms, the new algorithm can discover a high-quality cluster structure of a data set without prior information. Extensive experiments on benchmark data sets illustrate the effectiveness of the proposed algorithm.

Multiple Kernel Clustering With Compressed Subspace Alignment

Multiple kernel clustering (MKC) has recently achieved remarkable progress in fusing multisource information to boost the clustering performance. However, the O(n²) memory consumption and O(n³) computational complexity prohibit these methods from being applied into median- or large-scale applications, where n denotes the number of samples. To address these issues, we carefully redesign the formulation of subspace segmentation-based MKC, which reduces the memory and computational complexity to O(n) and O(n²) , respectively. The proposed algorithm adopts a novel sampling strategy to enhance the performance and accelerate the speed of MKC. Specifically, we first mathematically model the sampling process and then learn it simultaneously during the procedure of information fusion. By this way, the generated anchor point set can better serve data reconstruction across different views, leading to improved discriminative capability of the reconstruction matrix and boosted clustering performance. Although the integrated sampling process makes the proposed algorithm less efficient than the linear complexity algorithms, the elaborate formulation makes our algorithm straightforward for parallelization. Through the acceleration of GPU and multicore techniques, our algorithm achieves superior performance against the compared state-of-the-art methods on six datasets with comparable time cost to the linear complexity algorithms.

Secuer: Ultrafast, scalable and accurate clustering of single-cell RNA-seq data

Identifying cell clusters is a critical step for single-cell transcriptomics study. Despite the numerous clustering tools developed recently, the rapid growth of scRNA-seq volumes prompts for a more (computationally) efficient clustering method. Here, we introduce Secuer, a Scalable and Efficient speCtral clUstERing algorithm for scRNA-seq data. By employing an anchor-based bipartite graph representation algorithm, Secuer enjoys reduced runtime and memory usage over one order of magnitude for datasets with more than 1 million cells. Meanwhile, Secuer also achieves better or comparable accuracy than competing methods in small and moderate benchmark datasets. Furthermore, we showcase that Secuer can also serve as a building block for a new consensus clustering method, Secuer-consensus, which again improves the runtime and scalability of state-of-the-art consensus clustering methods while also maintaining the accuracy. Overall, Secuer is a versatile, accurate, and scalable clustering framework suitable for small to ultra-large single-cell clustering tasks.

LFSC: A linear fast semi-supervised clustering algorithm that integrates reference-bulk and single-cell transcriptomes

The identification of cell types in complex tissues is an important step in research into cellular heterogeneity in disease. We present a linear fast semi-supervised clustering (LFSC) algorithm that utilizes reference samples generated from bulk RNA sequencing data to identify cell types from single-cell transcriptomes. An anchor graph is constructed to depict the relationship between reference samples and cells. By applying a connectivity constraint to the learned graph, LFSC enables the preservation of the underlying cluster structure. Moreover, the overall complexity of LFSC is linear to the size of the data, which greatly improves effectiveness and efficiency. By applying LFSC to real single-cell RNA sequencing datasets, we discovered that it has superior performance over existing baseline methods in clustering accuracy and robustness. An application using infiltrating T cells in liver cancer demonstrates that LFSC can successfully find new cell types, discover differently expressed genes, and explore new cancer-associated biomarkers.

Multiview Subspace Clustering by an Enhanced Tensor Nuclear Norm

Despite the promising preliminary results, tensor-singular value decomposition (t-SVD)-based multiview subspace is incapable of dealing with real problems, such as noise and illumination changes. The major reason is that tensor-nuclear norm minimization (TNNM) used in t-SVD regularizes each singular value equally, which does not make sense in matrix completion and coefficient matrix learning. In this case, the singular values represent different perspectives and should be treated differently. To well exploit the significant difference between singular values, we study the weighted tensor Schatten p -norm based on t-SVD and develop an efficient algorithm to solve the weighted tensor Schatten p -norm minimization (WTSNM) problem. After that, applying WTSNM to learn the coefficient matrix in multiview subspace clustering, we present a novel multiview clustering method by integrating coefficient matrix learning and spectral clustering into a unified framework. The learned coefficient matrix well exploits both the cluster structure and high-order information embedded in multiview views. The extensive experiments indicate the efficiency of our method in six metrics.

Large-Scale Subspace Clustering by Independent Distributed and Parallel Coding

Subspace clustering is a popular method to discover underlying low-dimensional structures of high-dimensional multimedia data (e.g., images, videos, and texts). In this article, we consider a large-scale subspace clustering (LS²C) problem, that is, partitioning million data points with a millon dimensions. To address this, we explore an independent distributed and parallel framework by dividing big data/variable matrices and regularization by both columns and rows. Specifically, LS²C is independently decomposed into many subproblems by distributing those matrices into different machines by columns since the regularization of the code matrix is equal to a sum of that of its submatrices (e.g., square-of-Frobenius/ l₁ -norm). Consensus optimization is designed to solve these subproblems in a parallel way for saving communication costs. Moreover, we provide theoretical guarantees that LS²C can recover consensus subspace representations of high-dimensional data points under broad conditions. Compared with the state-of-the-art LS²C methods, our approach achieves better clustering results in public datasets, including a million images and videos.

Hierarchical Topology-Based Cluster Representation for Scalable Evolutionary Multiobjective Clustering

Evolutionary multiobjective clustering (MOC) algorithms have shown promising potential to outperform conventional single-objective clustering algorithms, especially when the number of clusters k is not set before clustering. However, the computational burden becomes a tricky problem due to the extensive search space and fitness computational time of the evolving population, especially when the data size is large. This article proposes a new, hierarchical, topology-based cluster representation for scalable MOC, which can simplify the search procedure and decrease computational overhead. A coarse-to-fine-trained topological structure that fits the spatial distribution of the data is utilized to identify a set of seed points/nodes, then a tree-based graph is built to represent clusters. During optimization, a bipartite graph partitioning strategy incorporated with the graph nodes helps in performing a cluster ensemble operation to generate offspring solutions more effectively. For the determination of the final result, which is underexplored in the existing methods, the usage of a cluster ensemble strategy is also presented, whether k is provided or not. Comparison experiments are conducted on a series of different data distributions, revealing the superiority of the proposed algorithm in terms of both clustering performance and computing efficiency.

Fast Self-Supervised Clustering With Anchor Graph

Benefit from avoiding the utilization of labeled samples, which are usually insufficient in the real world, unsupervised learning has been regarded as a speedy and powerful strategy on clustering tasks. However, clustering directly from primal data sets leads to high computational cost, which limits its application on large-scale and high-dimensional problems. Recently, anchor-based theories are proposed to partly mitigate this problem and field naturally sparse affinity matrix, while it is still a challenge to get excellent performance along with high efficiency. To dispose of this issue, we first presented a fast semisupervised framework (FSSF) combined with a balanced K -means-based hierarchical K -means (BKHK) method and the bipartite graph theory. Thereafter, we proposed a fast self-supervised clustering method involved in this crucial semisupervised framework, in which all labels are inferred from a constructed bipartite graph with exactly k connected components. The proposed method remarkably accelerates the general semisupervised learning through the anchor and consists of four significant parts: 1) obtaining the anchor set as interim through BKHK algorithm; 2) constructing the bipartite graph; 3) solving the self-supervised problem to construct a typical probability model with FSSF; and 4) selecting the most representative points regarding anchors from BKHK as an interim and conducting label propagation. The experimental results on toy examples and benchmark data sets have demonstrated that the proposed method outperforms other approaches.

A Hybrid Forecast Model for Household Electric Power by Fusing Landmark-Based Spectral Clustering and Deep Learning

Household power load forecasting plays an important role in the operation and planning of power grids. To address the prediction issue of household power consumption in power grids, this paper chooses a time series of historical power consumption as the feature variables and uses landmark-based spectral clustering (LSC) and a deep learning model to cluster and predict the power consumption dataset, respectively. Firstly, the investigated data are reshaped into a matrix and all missing entries are recovered by matrix completion. Secondly, the data samples are divided into three clusters by the LSC method according to the periodicity and regularity of power consumption. Then, all samples in each cluster are expanded via bootstrap aggregating technique. Subsequently, a combination of a convolutional neural network (CNN) and a long short-term memory (LSTM) is employed to predict power consumption. The goal of CNN is to extract the features from input data in sequence learning, and LSTM aims to train and predict the power consumption. Finally, the forecasting performance of the LSC–CNN–LSTM is compared with several other deep learning models to verify its reliability and effectiveness in the field of household power load. The experimental results show that the proposed hybrid method is superior to other state-of-the-art deep learning techniques in forecasting performance.

Achieving deep clustering through the use of variational autoencoders and similarity-based loss

Clustering is an important and challenging research topic in many fields. Although various clustering algorithms have been developed in the past, traditional shallow clustering algorithms cannot mine the underlying structural information of the data. Recent advances have shown that deep clustering can achieve excellent performance on clustering tasks. In this work, a novel variational autoencoder-based deep clustering algorithm is proposed. It treats the Gaussian mixture model as the prior latent space and uses an additional classifier to distinguish different clusters in the latent space accurately. A similarity-based loss function is proposed consisting specifically of the cross-entropy of the predicted transition probabilities of clusters and the Wasserstein distance of the predicted posterior distributions. The new loss encourages the model to learn meaningful cluster-oriented representations to facilitate clustering tasks. The experimental results show that our method consistently achieves competitive results on various data sets.

Consensus clustering for Bayesian mixture models

BACKGROUND

Cluster analysis is an integral part of precision medicine and systems biology, used to define groups of patients or biomolecules. Consensus clustering is an ensemble approach that is widely used in these areas, which combines the output from multiple runs of a non-deterministic clustering algorithm. Here we consider the application of consensus clustering to a broad class of heuristic clustering algorithms that can be derived from Bayesian mixture models (and extensions thereof) by adopting an early stopping criterion when performing sampling-based inference for these models. While the resulting approach is non-Bayesian, it inherits the usual benefits of consensus clustering, particularly in terms of computational scalability and providing assessments of clustering stability/robustness.

RESULTS

In simulation studies, we show that our approach can successfully uncover the target clustering structure, while also exploring different plausible clusterings of the data. We show that, when a parallel computation environment is available, our approach offers significant reductions in runtime compared to performing sampling-based Bayesian inference for the underlying model, while retaining many of the practical benefits of the Bayesian approach, such as exploring different numbers of clusters. We propose a heuristic to decide upon ensemble size and the early stopping criterion, and then apply consensus clustering to a clustering algorithm derived from a Bayesian integrative clustering method. We use the resulting approach to perform an integrative analysis of three 'omics datasets for budding yeast and find clusters of co-expressed genes with shared regulatory proteins. We validate these clusters using data external to the analysis.

CONCLUSTIONS

Our approach can be used as a wrapper for essentially any existing sampling-based Bayesian clustering implementation, and enables meaningful clustering analyses to be performed using such implementations, even when computational Bayesian inference is not feasible, e.g. due to poor exploration of the target density (often as a result of increasing numbers of features) or a limited computational budget that does not along sufficient samples to drawn from a single chain. This enables researchers to straightforwardly extend the applicability of existing software to much larger datasets, including implementations of sophisticated models such as those that jointly model multiple datasets.

Multiview Spectral Clustering With Bipartite Graph

Multi-view spectral clustering has become appealing due to its good performance in capturing the correlations among all views. However, on one hand, many existing methods usually require a quadratic or cubic complexity for graph construction or eigenvalue decomposition of Laplacian matrix; on the other hand, they are inefficient and unbearable burden to be applied to large scale data sets, which can be easily obtained in the era of big data. Moreover, the existing methods cannot encode the complementary information between adjacency matrices, i.e., similarity graphs of views and the low-rank spatial structure of adjacency matrix of each view. To address these limitations, we develop a novel multi-view spectral clustering model. Our model well encodes the complementary information by Schatten p -norm regularization on the third tensor whose lateral slices are composed of the adjacency matrices of the corresponding views. To further improve the computational efficiency, we leverage anchor graphs of views instead of full adjacency matrices of the corresponding views, and then present a fast model that encodes the complementary information embedded in anchor graphs of views by Schatten p -norm regularization on the tensor bipartite graph. Finally, an efficient alternating algorithm is derived to optimize our model. The constructed sequence was proved to converge to the stationary KKT point. Extensive experimental results indicate that our method has good performance.

A Survey on Multi-View Clustering

Clustering is a machine learning paradigm of dividing sample subjects into a number of groups such that subjects in the same groups are more similar to those in other groups. With advances in information acquisition technologies, samples can frequently be viewed from different angles or in different modalities, generating multi-view data. Multi-view clustering, that clusters subjects into subgroups using multi-view data, has attracted more and more attentions. Although MVC methods have been developed rapidly, there has not been enough survey to summarize and analyze the current progress. Therefore, we propose a novel taxonomy of the MVC approaches. Similar to other machine learning methods, we categorize them into generative and discriminative classes. In discriminative class, based on the way of view integration, we split it further into five groups: Common Eigenvector Matrix, Common Coefficient Matrix, Common Indicator Matrix, Direct Combination and Combination After Projection. Furthermore, we relate MVC to other topics: multi-view representation, ensemble clustering, multi-task clustering, multi-view supervised and semi-supervised learning. Several representative real-world applications are elaborated for practitioners. Some benchmark multi-view datasets are introduced and representative MVC algorithms from each group are empirically evaluated to analyze how they perform on benchmark datasets. To promote future development of MVC approaches, we point out several open problems that may require further investigation and thorough examination.

Learnable Subspace Clustering

This article studies the large-scale subspace clustering (LS²C) problem with millions of data points. Many popular subspace clustering methods cannot directly handle the LS²C problem although they have been considered to be state-of-the-art methods for small-scale data points. A simple reason is that these methods often choose all data points as a large dictionary to build huge coding models, which results in high time and space complexity. In this article, we develop a learnable subspace clustering paradigm to efficiently solve the LS²C problem. The key concept is to learn a parametric function to partition the high-dimensional subspaces into their underlying low-dimensional subspaces instead of the computationally demanding classical coding models. Moreover, we propose a unified, robust, predictive coding machine (RPCM) to learn the parametric function, which can be solved by an alternating minimization algorithm. Besides, we provide a bounded contraction analysis of the parametric function. To the best of our knowledge, this article is the first work to efficiently cluster millions of data points among the subspace clustering methods. Experiments on million-scale data sets verify that our paradigm outperforms the related state-of-the-art methods in both efficiency and effectiveness.

Multiview Clustering: A Scalable and Parameter-Free Bipartite Graph Fusion Method

Multiview clustering partitions data into different groups according to their heterogeneous features. Most existing methods degenerate the applicability of models due to their intractable hyper-parameters triggered by various regularization terms. Moreover, traditional spectral based methods always encounter the expensive time overheads and fail in exploring the explicit clusters from graphs. In this paper, we present a scalable and parameter-free graph fusion framework for multiview clustering, seeking for a joint graph compatible across multiple views in a self-supervised weighting manner. Our formulation coalesces multiple view-wise graphs straightforward and learns the weights as well as the joint graph interactively, which could actively release the model from any weight-related hyper-parameters. Meanwhile, we manipulate the joint graph by a connectivity constraint such that the connected components indicate clusters directly. The designed algorithm is initialization-independent and time-economical which obtains the stable performance and scales well with the data size. Substantial experiments on toy data as well as real datasets are conducted that verify the superiority of the proposed method compared to the state-of-the-arts over the clustering performance and time expenditure.

Large Graph Clustering With Simultaneous Spectral Embedding and Discretization

Spectral clustering methods are gaining more and more interests and successfully applied in many fields because of their superior performance. However, there still exist two main problems to be solved: 1) spectral clustering methods consist of two successive optimization stages-spectral embedding and spectral rotation, which may not lead to globally optimal solutions, 2) and it is known that spectral methods are time-consuming with very high computational complexity. There are methods proposed to reduce the complexity for data vectors but not for graphs that only have information about similarity matrices. In this paper, we propose a new method to solve these two challenging problems for graph clustering. In the new method, a new framework is established to perform spectral embedding and spectral rotation simultaneously. The newly designed objective function consists of both terms of embedding and rotation, and we use an improved spectral rotation method to make it mathematically rigorous for the optimization. To further accelerate the algorithm, we derive a low-dimensional representation matrix from a graph by using label propagation, with which, in return, we can reconstruct a double-stochastic and positive semidefinite similarity matrix. Experimental results demonstrate that our method has excellent performance in time cost and accuracy.

Fast Manifold Ranking With Local Bipartite Graph

During the past decades, manifold ranking has been widely applied to content-based image retrieval and shown excellent performance. However, manifold ranking is computationally expensive in both graph construction and ranking learning. Much effort has been devoted to improve its performance by introducing approximating techniques. In this paper, we propose a fast manifold ranking method, namely Local Bipartite Manifold Ranking (LBMR). Given a set of images, we first extract multiple regions from each image to form a large image descriptor matrix, and then use the anchor-based strategy to construct a local bipartite graph in which a regional k -means (RKM) is proposed to obtain high quality anchors. We propose an iterative method to directly solve the manifold ranking problem from the local bipartite graph, which monotonically decreases the objective function value in each iteration until the algorithm converges. Experimental results on several real-world image datasets demonstrate the effectiveness and efficiency of our proposed method.

Linear-time cluster ensembles of large-scale single-cell RNA-seq and multimodal data

A fundamental task in single-cell RNA-seq (scRNA-seq) analysis is the identification of transcriptionally distinct groups of cells. Numerous methods have been proposed for this problem, with a recent focus on methods for the cluster analysis of ultralarge scRNA-seq data sets produced by droplet-based sequencing technologies. Most existing methods rely on a sampling step to bridge the gap between algorithm scalability and volume of the data. Ignoring large parts of the data, however, often yields inaccurate groupings of cells and risks overlooking rare cell types. We propose method Specter that adopts and extends recent algorithmic advances in (fast) spectral clustering. In contrast to methods that cluster a (random) subsample of the data, we adopt the idea of landmarks that are used to create a sparse representation of the full data from which a spectral embedding can then be computed in linear time. We exploit Specter's speed in a cluster ensemble scheme that achieves a substantial improvement in accuracy over existing methods and identifies rare cell types with high sensitivity. Its linear-time complexity allows Specter to scale to millions of cells and leads to fast computation times in practice. Furthermore, on CITE-seq data that simultaneously measures gene and protein marker expression, we show that Specter is able to use multimodal omics measurements to resolve subtle transcriptomic differences between subpopulations of cells.

Deep Clustering With Sample-Assignment Invariance Prior

Most popular clustering methods map raw image data into a projection space in which the clustering assignment is obtained with the vanilla k-means approach. In this article, we discovered a novel prior, namely, there exists a common invariance when assigning an image sample to clusters using different metrics. In short, different distance metrics will lead to similar soft clustering assignments on the manifold. Based on such a novel prior, we propose a novel clustering method by minimizing the discrepancy between pairwise sample assignments for each data point. To the best of our knowledge, this could be the first work to reveal the sample-assignment invariance prior based on the idea of treating labels as ideal representations. Furthermore, the proposed method is one of the first end-to-end clustering approaches, which jointly learns clustering assignment and representation. Extensive experimental results show that the proposed method is remarkably superior to 16 state-of-the-art clustering methods on five image data sets in terms of four evaluation metrics.

Discrete Optimal Graph Clustering

Graph-based clustering is one of the major clustering methods. Most of it works in three separate steps: 1) similarity graph construction; 2) clustering label relaxing; and 3) label discretization with k -means (KM). Such common practice has three disadvantages: 1) the predefined similarity graph is often fixed and may not be optimal for the subsequent clustering; 2) the relaxing process of cluster labels may cause significant information loss; and 3) label discretization may deviate from the real clustering result since KM is sensitive to the initialization of cluster centroids. To tackle these problems, in this paper, we propose an effective discrete optimal graph clustering framework. A structured similarity graph that is theoretically optimal for clustering performance is adaptively learned with a guidance of reasonable rank constraints. Besides, to avoid the information loss, we explicitly enforce a discrete transformation on the intermediate continuous label, which derives a tractable optimization problem with a discrete solution. Furthermore, to compensate for the unreliability of the learned labels and enhance the clustering accuracy, we design an adaptive robust module that learns the prediction function for the unseen data based on the learned discrete cluster labels. Finally, an iterative optimization strategy guaranteed with convergence is developed to directly solve the clustering results. Extensive experiments conducted on both real and synthetic datasets demonstrate the superiority of our proposed methods compared with several state-of-the-art clustering approaches.

LABIN: Balanced Min Cut for Large-Scale Data

Although many spectral clustering algorithms have been proposed during the past decades, they are not scalable to large-scale data due to their high computational complexities. In this paper, we propose a novel spectral clustering method for large-scale data, namely, large-scale balanced min cut (LABIN). A new model is proposed to extend the self-balanced min-cut (SBMC) model with the anchor-based strategy and a fast spectral rotation with linear time complexity is proposed to solve the new model. Extensive experimental results show the superior performance of our proposed method in comparison with the state-of-the-art methods including SBMC.

Fast Semisupervised Learning With Bipartite Graph for Large-Scale Data

As the captured information in our real word is very scare and labeling sample is time cost and expensive, semisupervised learning (SSL) has an important application in computer vision and machine learning. Among SSL approaches, a graph-based SSL (GSSL) model has recently attracted much attention for high accuracy. However, for most traditional GSSL methods, the large-scale data bring higher computational complexity, which acquires a better computing platform. In order to dispose of these issues, we propose a novel approach, bipartite GSSL normalized (BGSSL-normalized) method, in this paper. This method consists of three parts. First, the bipartite graph between the original data and the anchor points is constructed, which is parameter-insensitive, scale-invariant, naturally sparse, and simple operation. Then, the label of the original data and anchors can be inferred through the graph. Besides, we extend our algorithm to handle out-of-sample for large-scale data by the inferred label of anchors, which not only retains good classification result but also saves a large amount of time. The computational complexity of BGSSL-normalized can be reduced to O(ndm+nm²) , which is a significant improvement compared with traditional GSSL methods that need O(n²d+n³) , where n , d , and m are the number of samples, features, and anchors, respectively. The experimental results on several publicly available data sets demonstrate that our approaches can achieve better classification accuracy with less time costs.

Developmental Resonance Network

Adaptive resonance theory (ART) networks deal with normalized input data only, which means that they need the normalization process for the raw input data, under the assumption that the upper and lower bounds of the input data are known in advance. Without such an assumption, ART networks cannot be utilized. To solve this problem and improve the learning performance, inspired by the ART networks, we propose a developmental resonance network (DRN) by employing new techniques of a global weight and node connection and grouping processes. The proposed DRN learns the global weight converging to the unknown range of the input data and properly clusters by grouping similar nodes into one. These techniques enable DRN to learn the raw input data without the normalization process while retaining the stability, plasticity, and memory usage efficiency without node proliferation. Simulation results verify that our DRN, applied to the unsupervised clustering problem, can cluster raw data properly without a prior normalization process.

Fast Large-Scale Spectral Clustering via Explicit Feature Mapping

We propose an efficient spectral clustering method for large-scale data. The main idea in our method consists of employing random Fourier features to explicitly represent data in kernel space. The complexity of spectral clustering thus is shown lower than existing Nyström approximations on large-scale data. With m training points from a total of n data points, Nyström method requires O(nmd+m³+nm²) operations, where d is the input dimension. In contrast, our proposed method requires O(nDd+D³+n'D²) , where n' is the number of data points needed until convergence and D is the kernel mapped dimension. In large-scale datasets where n' << n hold true, our explicitly mapping method can significantly speed up eigenvector approximation and benefit prediction speed in spectral clustering. For instance, on MNIST (60 000 data points), the proposed method is similar in clustering accuracy to Nyström methods while its speed is twice as fast as Nyström.

Multiview Spectral Clustering via Structured Low-Rank Matrix Factorization

Multiview data clustering attracts more attention than their single-view counterparts due to the fact that leveraging multiple independent and complementary information from multiview feature spaces outperforms the single one. Multiview spectral clustering aims at yielding the data partition agreement over their local manifold structures by seeking eigenvalue-eigenvector decompositions. Among all the methods, low-rank representation (LRR) is effective, by exploring the multiview consensus structures beyond the low rankness to boost the clustering performance. However, as we observed, such classical paradigm still suffers from the following stand-out limitations for multiview spectral clustering of overlooking the flexible local manifold structure, caused by aggressively enforcing the low-rank data correlation agreement among all views, and such a strategy, therefore, cannot achieve the satisfied between-views agreement; worse still, LRR is not intuitively flexible to capture the latent data clustering structures. In this paper, first, we present the structured LRR by factorizing into the latent low-dimensional data-cluster representations, which characterize the data clustering structure for each view. Upon such representation, second, the Laplacian regularizer is imposed to be capable of preserving the flexible local manifold structure for each view. Third, we present an iterative multiview agreement strategy by minimizing the divergence objective among all factorized latent data-cluster representations during each iteration of optimization process, where such latent representation from each view serves to regulate those from other views, and such an intuitive process iteratively coordinates all views to be agreeable. Fourth, we remark that such data-cluster representation can flexibly encode the data clustering structure from any view with an adaptive input cluster number. To this end, finally, a novel nonconvex objective function is proposed via the efficient alternating minimization strategy. The complexity analysis is also presented. The extensive experiments conducted against the real-world multiview data sets demonstrate the superiority over the state of the arts.

Approximate sparse spectral clustering based on local information maintenance for hyperspectral image classification

Sparse spectral clustering (SSC) has become one of the most popular clustering approaches in recent years. However, its high computational complexity prevents its application to large-scale datasets such as hyperspectral images (HSIs). In this paper, we propose two efficient approximate sparse spectral clustering methods for HSIs clustering in which clustering performance is improved by utilizing local information among the data. Firstly, we construct a smaller representative dataset on which sparse spectral clustering is performed. Then the labels of ground object are extending to whole dataset based on the local information according to two extending strategies. The first one is that the local interpolation is utilized to improve the extension of the clustering result. The other one is that the label extension is turned to a problem of subspace embedding, and is fulfilled by locally linear embedding (LLE). Several experiments on HSIs demonstrated that the proposed algorithms are effective for HSIs clustering.