1
|
Two-stage semi-supervised clustering ensemble framework based on constraint weight. INT J MACH LEARN CYB 2023. [DOI: 10.1007/s13042-022-01651-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
|
2
|
Robust semi-supervised data representation and imputation by correntropy based constraint nonnegative matrix factorization. APPL INTELL 2022. [DOI: 10.1007/s10489-022-03884-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
3
|
Zhang Y, Zhang Z, Wang Y, Zhang Z, Zhang L, Yan S, Wang M. Dual-Constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior. Int J Comput Vis 2021. [DOI: 10.1007/s11263-021-01524-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
|
4
|
Gao Y, Luo S, Pan J, Wang Z, Gao P. Kernel alignment unsupervised discriminative dimensionality reduction. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2021.03.127] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
5
|
Bhalla V, Chaudhury S. Integrated Semi-Supervised Model for Learning and Classification. PROCEEDINGS OF 3RD INTERNATIONAL CONFERENCE ON COMPUTER VISION AND IMAGE PROCESSING 2020:183-195. [DOI: 10.1007/978-981-32-9088-4_16] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/19/2023]
|
6
|
|
7
|
Yu Z, Zhang Y, You J, Chen CLP, Wong HS, Han G, Zhang J. Adaptive Semi-Supervised Classifier Ensemble for High Dimensional Data Classification. IEEE TRANSACTIONS ON CYBERNETICS 2019; 49:366-379. [PMID: 29989979 DOI: 10.1109/tcyb.2017.2761908] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
High dimensional data classification with very limited labeled training data is a challenging task in the area of data mining. In order to tackle this task, we first propose a feature selection-based semi-supervised classifier ensemble framework (FSCE) to perform high dimensional data classification. Then, we design an adaptive semi-supervised classifier ensemble framework (ASCE) to improve the performance of FSCE. When compared with FSCE, ASCE is characterized by an adaptive feature selection process, an adaptive weighting process (AWP), and an auxiliary training set generation process (ATSGP). The adaptive feature selection process generates a set of compact subspaces based on the selected attributes obtained by the feature selection algorithms, while the AWP associates each basic semi-supervised classifier in the ensemble with a weight value. The ATSGP enlarges the training set with unlabeled samples. In addition, a set of nonparametric tests are adopted to compare multiple semi-supervised classifier ensemble (SSCE)approaches over different datasets. The experiments on 20 high dimensional real-world datasets show that: 1) the two adaptive processes in ASCE are useful for improving the performance of the SSCE approach and 2) ASCE works well on high dimensional datasets with very limited labeled training data, and outperforms most state-of-the-art SSCE approaches.
Collapse
|
8
|
Zhi X, Yan H, Fan J, Zheng S. Efficient discriminative clustering via QR decomposition-based Linear Discriminant Analysis. Knowl Based Syst 2018. [DOI: 10.1016/j.knosys.2018.04.031] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
|
9
|
Liu CL, Hsaio WH, Lin CY. Bayesian exploratory clustering with entropy Chinese restaurant process. INTELL DATA ANAL 2018. [DOI: 10.3233/ida-163332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Affiliation(s)
- Chien-Liang Liu
- Department of Industrial Engineering and Management, NCTU, Hsinchu, Taiwan
| | | | - Che-Yuan Lin
- Department of Computer Science, NCTU, Hsinchu, Taiwan
| |
Collapse
|
10
|
Yousefnezhad M, Huang SJ, Zhang D. WoCE: A framework for Clustering Ensemble by Exploiting the Wisdom of Crowds Theory. IEEE TRANSACTIONS ON CYBERNETICS 2018; 48:486-499. [PMID: 28060718 DOI: 10.1109/tcyb.2016.2642999] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
The wisdom of crowds (WOCs), as a theory in the social science, gets a new paradigm in computer science. The WOC theory explains that the aggregate decision made by a group is often better than those of its individual members if specific conditions are satisfied. This paper presents a novel framework for unsupervised and semisupervised cluster ensemble by exploiting the WOC theory. We employ four conditions in the WOC theory, i.e., diversity, independency, decentralization, and aggregation, to guide both constructing of individual clustering results and final combination for clustering ensemble. First, independency criterion, as a novel mapping system on the raw data set, removes the correlation between features on our proposed method. Then, decentralization as a novel mechanism generates high quality individual clustering results. Next, uniformity as a new diversity metric evaluates the generated clustering results. Further, weighted evidence accumulation clustering method is proposed for the final aggregation without using thresholding procedure. Experimental study on varied data sets demonstrates that the proposed approach achieves superior performance to state-of-the-art methods.
Collapse
|
11
|
Yu Z, Lu Y, Zhang J, You J, Wong HS, Wang Y, Han G. Progressive Semisupervised Learning of Multiple Classifiers. IEEE TRANSACTIONS ON CYBERNETICS 2018; 48:689-702. [PMID: 28113355 DOI: 10.1109/tcyb.2017.2651114] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
Semisupervised learning methods are often adopted to handle datasets with very small number of labeled samples. However, conventional semisupervised ensemble learning approaches have two limitations: 1) most of them cannot obtain satisfactory results on high dimensional datasets with limited labels and 2) they usually do not consider how to use an optimization process to enlarge the training set. In this paper, we propose the progressive semisupervised ensemble learning approach (PSEMISEL) to address the above limitations and handle datasets with very small number of labeled samples. When compared with traditional semisupervised ensemble learning approaches, PSEMISEL is characterized by two properties: 1) it adopts the random subspace technique to investigate the structure of the dataset in the subspaces and 2) a progressive training set generation process and a self evolutionary sample selection process are proposed to enlarge the training set. We also use a set of nonparametric tests to compare different semisupervised ensemble learning methods over multiple datasets. The experimental results on 18 real-world datasets from the University of California, Irvine machine learning repository show that PSEMISEL works well on most of the real-world datasets, and outperforms other state-of-the-art approaches on 10 out of 18 datasets.
Collapse
|
12
|
Sun S, Xie X, Yang M. Multiview Uncorrelated Discriminant Analysis. IEEE TRANSACTIONS ON CYBERNETICS 2016; 46:3272-3284. [PMID: 26662351 DOI: 10.1109/tcyb.2015.2502248] [Citation(s) in RCA: 31] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Multiview learning is more robust than single-view learning in many real applications. Canonical correlation analysis (CCA) is a popular technique to utilize information stemming from multiple feature sets. However, it does not exploit label information effectively. Later multiview linear discriminant analysis (MLDA) was proposed through combining CCA and linear discriminant analysis (LDA). Due to the successful application of uncorrelated LDA (ULDA), which seeks optimal discriminant features with minimum redundancy, we propose a new supervised learning method called multiview ULDA (MULDA) in this paper. This method combines the theory of ULDA with CCA. Then we adapt discriminant CCA (DCCA) instead of the CCA in MLDA and MULDA, and discuss about the effect of this modification. Furthermore, we generalize these methods to the nonlinear case by kernel-based learning techniques. The new method is called kernel multiview uncorrelated discriminant analysis (KMUDA). Then we modify kernel multiview discriminant analysis and KMUDA by replacing Kernel CCA with Kernel DCCA. Our methods are tested on different real datasets and compared with other state-of-the-art methods. Experimental results validate the effectiveness of our methods.
Collapse
|
13
|
Huang S, Wang H, Li T, Yang Y, Li T. Constraint Co-Projections for Semi-Supervised Co-Clustering. IEEE TRANSACTIONS ON CYBERNETICS 2016; 46:3047-3058. [PMID: 26584506 DOI: 10.1109/tcyb.2015.2496174] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Co-clustering aims to simultaneously cluster the objects and features to explore intercorrelated patterns. However, it is usually difficult to obtain good co-clustering results by just analyzing the object-feature correlation data due to the sparsity of the data and the noise. Meanwhile, most co-clustering algorithms cannot take the prior information into consideration and may produce unmeaningful results. Semi-supervised co-clustering aims to incorporate the known prior knowledge into the co-clustering algorithm. In this paper, a new technique named constraint co-projections for semi-supervised co-clustering (CPSSCC) is presented. Constraint co-projections can not only make use of two popular techniques including pairwise constraints and constraint projections, but also simultaneously perform the object constraint projections and feature constraint projections. The two popular techniques are illustrated for semi-supervised co-clustering when some objects and features are believed to be in the same cluster a priori. Furthermore, we also prove that the co-clustering problem can be formulated as a typical eigen-problem and can be efficiently solved with the selected eigenvectors. To the best of our knowledge, constraint co-projections is first stated in this paper and this is the first work on using CPSSCC. Extensive experiments on benchmark data sets demonstrate the effectiveness of the proposed method. This paper also shows that CPSSCC has some favorable features compared with previous related co-clustering algorithms.
Collapse
|
14
|
Liu CL, Hsaio WH, Lee CH, Chang TH, Kuo TH. Semi-Supervised Text Classification With Universum Learning. IEEE TRANSACTIONS ON CYBERNETICS 2016; 46:462-473. [PMID: 25730839 DOI: 10.1109/tcyb.2015.2403573] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
Universum, a collection of nonexamples that do not belong to any class of interest, has become a new research topic in machine learning. This paper devises a semi-supervised learning with Universum algorithm based on boosting technique, and focuses on situations where only a few labeled examples are available. We also show that the training error of AdaBoost with Universum is bounded by the product of normalization factor, and the training error drops exponentially fast when each weak classifier is slightly better than random guessing. Finally, the experiments use four data sets with several combinations. Experimental results indicate that the proposed algorithm can benefit from Universum examples and outperform several alternative methods, particularly when insufficient labeled examples are available. When the number of labeled examples is insufficient to estimate the parameters of classification functions, the Universum can be used to approximate the prior distribution of the classification functions. The experimental results can be explained using the concept of Universum introduced by Vapnik, that is, Universum examples implicitly specify a prior distribution on the set of classification functions.
Collapse
|
15
|
Dornaika F, El Traboulsi Y. Learning Flexible Graph-Based Semi-Supervised Embedding. IEEE TRANSACTIONS ON CYBERNETICS 2016; 46:206-218. [PMID: 25730836 DOI: 10.1109/tcyb.2015.2399456] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
This paper introduces a graph-based semi-supervised embedding method as well as its kernelized version for generic classification and recognition tasks. The aim is to combine the merits of flexible manifold embedding and nonlinear graph-based embedding for semi-supervised learning. The proposed linear method will be flexible since it estimates a nonlinear manifold that is the closest one to a linear embedding. The proposed kernelized method will also be flexible since it estimates a kernel-based embedding that is the closest to a nonlinear manifold. In both proposed methods, the nonlinear manifold and the mapping (linear transform for the linear method and the kernel multipliers for the kernelized method) are simultaneously estimated, which overcomes the shortcomings of a cascaded estimation. The dimension of the final embedding obtained by the two proposed methods is not limited to the number of classes. They can be used by any kind of classifiers once the data are embedded into the new subspaces. Unlike nonlinear dimensionality reduction approaches, which suffer from out-of-sample problem, our proposed methods have an obvious advantage that the learnt subspace has a direct out-of-sample extension to novel samples, and are thus easily generalized to the entire high-dimensional input space. We provide extensive experiments on seven public databases in order to study the performance of the proposed methods. These experiments demonstrate much improvement over the state-of-the-art algorithms that are based on label propagation or graph-based semi-supervised embedding.
Collapse
|
16
|
Yang Y, Ma Z, Yang Y, Nie F, Shen HT. Multitask spectral clustering by exploring intertask correlation. IEEE TRANSACTIONS ON CYBERNETICS 2015; 45:1069-1080. [PMID: 25252288 DOI: 10.1109/tcyb.2014.2344015] [Citation(s) in RCA: 35] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
Clustering, as one of the most classical research problems in pattern recognition and data mining, has been widely explored and applied to various applications. Due to the rapid evolution of data on the Web, more emerging challenges have been posed on traditional clustering techniques: 1) correlations among related clustering tasks and/or within individual task are not well captured; 2) the problem of clustering out-of-sample data is seldom considered; and 3) the discriminative property of cluster label matrix is not well explored. In this paper, we propose a novel clustering model, namely multitask spectral clustering (MTSC), to cope with the above challenges. Specifically, two types of correlations are well considered: 1) intertask clustering correlation, which refers the relations among different clustering tasks and 2) intratask learning correlation, which enables the processes of learning cluster labels and learning mapping function to reinforce each other. We incorporate a novel l2,p -norm regularizer to control the coherence of all the tasks based on an assumption that related tasks should share a common low-dimensional representation. Moreover, for each individual task, an explicit mapping function is simultaneously learnt for predicting cluster labels by mapping features to the cluster label matrix. Meanwhile, we show that the learning process can naturally incorporate discriminative information to further improve clustering performance. We explore and discuss the relationships between our proposed model and several representative clustering techniques, including spectral clustering, k -means and discriminative k -means. Extensive experiments on various real-world datasets illustrate the advantage of the proposed MTSC model compared to state-of-the-art clustering approaches.
Collapse
|