1
|
Ahmed MH, Tiun S, Omar N, Sani NS. A multi-view representation technique based on principal component analysis for enhanced short text clustering. PLoS One 2024; 19:e0309206. [PMID: 39178180 PMCID: PMC11343383 DOI: 10.1371/journal.pone.0309206] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2024] [Accepted: 08/08/2024] [Indexed: 08/25/2024] Open
Abstract
Clustering texts together is an essential task in data mining and information retrieval, whose aim is to group unlabeled texts into meaningful clusters that facilitate extracting and understanding useful information from large volumes of textual data. However, clustering short texts (STC) is complex because they typically contain sparse, ambiguous, noisy, and lacking information. One of the challenges for STC is finding a proper representation for short text documents to generate cohesive clusters. However, typically, STC considers only a single-view representation to do clustering. The single-view representation is inefficient for representing text due to its inability to represent different aspects of the target text. In this paper, we propose the most suitable multi-view representation (MVR) (by finding the best combination of different single-view representations) to enhance STC. Our work will explore different types of MVR based on different sets of single-view representation combinations. The combination of the single-view representations is done by a fixed length concatenation via Principal Component analysis (PCA) technique. Three standard datasets (Twitter, Google News, and StackOverflow) are used to evaluate the performances of various sets of MVRs on STC. Based on experimental results, the best combination of single-view representation as an effective for STC was the 5-views MVR (a combination of BERT, GPT, TF-IDF, FastText, and GloVe). Based on that, we can conclude that MVR improves the performance of STC; however, the design for MVR requires selective single-view representations.
Collapse
Affiliation(s)
- Majid Hameed Ahmed
- CAIT, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Bangi, Selangor, Malaysia
- Ministry of Higher Education and Scientific Research, Baghdad, Iraq
| | - Sabrina Tiun
- CAIT, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Bangi, Selangor, Malaysia
| | - Nazlia Omar
- CAIT, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Bangi, Selangor, Malaysia
| | - Nor Samsiah Sani
- CAIT, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Bangi, Selangor, Malaysia
| |
Collapse
|
2
|
Yang L, Fan W, Bouguila N. Deep Clustering Analysis via Dual Variational Autoencoder With Spherical Latent Embeddings. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2023; 34:6303-6312. [PMID: 34941534 DOI: 10.1109/tnnls.2021.3135460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
In recent years, clustering methods based on deep generative models have received great attention in various unsupervised applications, due to their capabilities for learning promising latent embeddings from original data. This article proposes a novel clustering method based on variational autoencoder (VAE) with spherical latent embeddings. The merits of our clustering method can be summarized as follows. First, instead of considering the Gaussian mixture model (GMM) as the prior over latent space as in a variety of existing VAE-based deep clustering methods, the von Mises-Fisher mixture model prior is deployed in our method, leading to spherical latent embeddings that can explicitly control the balance between the capacity of decoder and the utilization of latent embedding in a principled way. Second, a dual VAE structure is leveraged to impose the reconstruction constraint for the latent embedding and its corresponding noise counterpart, which embeds the input data into a hyperspherical latent space for clustering. Third, an augmented loss function is proposed to enhance the robustness of our model, which results in a self-supervised manner through the mutual guidance between the original data and the augmented ones. The effectiveness of the proposed deep generative clustering method is validated through comparisons with state-of-the-art deep clustering methods on benchmark datasets. The source code of the proposed model is available at https://github.com/fwt-team/DSVAE.
Collapse
|
3
|
Exploration and analysis of On-Surface and In-Air handwriting attributes to improve dysgraphia disorder diagnosis in children based on machine learning methods. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104715] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/22/2023]
|
4
|
Ren X, Jia L, Zhao Z, Qiang Y, Wu W, Han P, Zhao J, Sun J. Weakly supervised label propagation algorithm classifies lung cancer imaging subtypes. Sci Rep 2023; 13:5167. [PMID: 36997586 PMCID: PMC10063585 DOI: 10.1038/s41598-023-32301-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2022] [Accepted: 03/25/2023] [Indexed: 04/01/2023] Open
Abstract
Aiming at the problems of long time, high cost, invasive sampling damage, and easy emergence of drug resistance in lung cancer gene detection, a reliable and non-invasive prognostic method is proposed. Under the guidance of weakly supervised learning, deep metric learning and graph clustering methods are used to learn higher-level abstract features in CT imaging features. The unlabeled data is dynamically updated through the k-nearest label update strategy, and the unlabeled data is transformed into weak label data and continue to update the process of strong label data to optimize the clustering results and establish a classification model for predicting new subtypes of lung cancer imaging. Five imaging subtypes are confirmed on the lung cancer dataset containing CT, clinical and genetic information downloaded from the TCIA lung cancer database. The successful establishment of the new model has a significant accuracy rate for subtype classification (ACC = 0.9793), and the use of CT sequence images, gene expression, DNA methylation and gene mutation data from the cooperative hospital in Shanxi Province proves the biomedical value of this method. The proposed method also can comprehensively evaluate intratumoral heterogeneity based on the correlation between the final lung CT imaging features and specific molecular subtypes.
Collapse
Affiliation(s)
- Xueting Ren
- College of Information and Computer, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Liye Jia
- College of Information and Computer, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Zijuan Zhao
- College of Information and Computer, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Yan Qiang
- College of Information and Computer, Taiyuan University of Technology, Taiyuan, Shanxi, China
| | - Wei Wu
- Department of Clinical Laboratory, Affiliated People's Hospital of Shanxi Medical University, Shanxi Provincial People's Hospital, Taiyuan, Shanxi, China
| | - Peng Han
- North Automatic Control Technology Institute, Taiyuan, Shanxi, China
| | - Juanjuan Zhao
- College of Information and Computer, Taiyuan University of Technology, Taiyuan, Shanxi, China.
| | - Jingyu Sun
- College of Information and Computer, Taiyuan University of Technology, Taiyuan, Shanxi, China
| |
Collapse
|
5
|
Astrid M, Zaheer MZ, Lee SI. PseudoBound: Limiting the Anomaly Reconstruction Capability of One-Class Classifiers Using Pseudo Anomalies. Neurocomputing 2023. [DOI: 10.1016/j.neucom.2023.03.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/11/2023]
|
6
|
Rong Z, Liu Z, Song J, Cao L, Yu Y, Qiu M, Hou Y. MCluster-VAEs: An end-to-end variational deep learning-based clustering method for subtype discovery using multi-omics data. Comput Biol Med 2022; 150:106085. [PMID: 36162197 DOI: 10.1016/j.compbiomed.2022.106085] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Revised: 07/30/2022] [Accepted: 09/03/2022] [Indexed: 11/03/2022]
Abstract
The discovery of cancer subtypes based on unsupervised clustering helps in providing a precise diagnosis, guide treatment, and improve patients' prognoses. Instead of single-omics data, multi-omics data can improve the clustering performance because it obtains a comprehensive landscape for understanding biological systems and mechanisms. However, heterogeneous data from multiple sources raises high complexity and different kinds of noise, which are detrimental to the extraction of clustering information. We propose an end-to-end deep learning based method, called Multi-omics Clustering Variational Autoencoders (MCluster-VAEs), that can extract cluster-friendly representations on multi-omics data. First, a unified network architecture with an attention mechanism was developed for accurately modeling multi-omics data. Then, using a novel objective function built from the Variational Bayes technique, the model was trained to effectively obtain the posterior estimation of the clustering assignments. Compared with 12 other state-of-the-art multi-omics clustering methods, MCluster-VAEs achieved an outstanding performance on benchmark datasets from the TCGA database. On the Pan Cancer dataset, MCluster-VAEs achieved an adjusted Rand index of approximately 0.78 for cancer category recognition, an increase of more than 18% compared with other methods. Furthermore, a survival analysis and clinical parameter enrichment tests conducted on 10 cancer datasets demonstrated that MCluster-VAEs provides comparable and even better results than many common integrative approaches. These results demonstrate that MCluster-VAEs are a powerful new tool for dissecting complex multi-omics relationships and providing new insights for cancer subtype discovery.
Collapse
Affiliation(s)
- Zhiwei Rong
- Department of Biostatistics Beijing, Peking University School of Public Health, No. 38 Xueyuan Road, Haidian District, Beijing, 100000, China
| | - Zhilin Liu
- Department of Biostatistics Beijing, Peking University School of Public Health, No. 38 Xueyuan Road, Haidian District, Beijing, 100000, China
| | - Jiali Song
- Department of Biostatistics Beijing, Peking University School of Public Health, No. 38 Xueyuan Road, Haidian District, Beijing, 100000, China
| | - Lei Cao
- Department of Epidemiology and Biostatistics Harbin, Harbin Medical University School of Public Health, Harbin, 150000, Heilongjiang, China
| | - Yipe Yu
- Department of Biostatistics Beijing, Peking University School of Public Health, No. 38 Xueyuan Road, Haidian District, Beijing, 100000, China
| | - Mantang Qiu
- Department of Thoracic Surgery Beijing, Peking University People's Hospital, Beijing, 100000, China.
| | - Yan Hou
- Department of Biostatistics Beijing, Peking University School of Public Health, No. 38 Xueyuan Road, Haidian District, Beijing, 100000, China; Peking University Clinical Research Center, No. 38 Xueyuan Road, Haidian District, Beijing, 100000, China.
| |
Collapse
|
7
|
Time-series classification with SAFE: Simple and fast segmented word embedding-based neural time series classifier. Inf Process Manag 2022. [DOI: 10.1016/j.ipm.2022.103044] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
8
|
Tissera D, Vithanage K, Wijesinghe R, Xavier A, Jayasena S, Fernando S, Rodrigo R. Neural mixture models with expectation-maximization for end-to-end deep clustering. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.07.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
|
9
|
Ali F, Kumar H, Patil S, Ahmad A, Babour A, Daud A. Deep-GHBP: Improving prediction of Growth Hormone-binding proteins using deep learning model. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103856] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
10
|
A biologically-inspired hybrid deep learning approach for brain tumor classification from magnetic resonance imaging using improved gabor wavelet transform and Elmann-BiLSTM network. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103949] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
11
|
Zhou Q, Li B, Tao P, Xu Z, Zhou C, Wu Y, Hu H. Residual-recursive autoencoder for accelerated evolution in savonius wind turbines optimization. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.04.103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
12
|
Si L, Qiang W, Li J, Xu F, Sun F. Multi-view representation learning from local consistency and global alignment. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.06.069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
|
13
|
EDCWRN: efficient deep clustering with the weight of representations and the help of neighbors. APPL INTELL 2022. [DOI: 10.1007/s10489-022-03895-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
14
|
Hazratgholizadeh R, Balafar MA, Derakhshi MRF. Active constrained deep embedded clustering with dual source. APPL INTELL 2022. [DOI: 10.1007/s10489-022-03752-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
15
|
Optimal DeepMRSeg based tumor segmentation with GAN for brain tumor classification. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103537] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/19/2023]
|
16
|
Kumar S, Gupta S, Arora S. A comparative simulation of normalization methods for machine learning-based intrusion detection systems using KDD Cup’99 dataset. JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2022. [DOI: 10.3233/jifs-211191] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Network Intrusion detection systems (NIDS) detect malicious and intrusive information in computer networks. Presently, commercial NIDS is based on machine learning approaches that have complex algorithms and increase intrusion detection efficiency and efficacy. These machine learning-based NIDS use high dimensional network traffic data from which intrusive information is to be detected. This high-dimensional network traffic data in NIDS needs to be preprocessed and normalized to make it suitable for machine learning tools. A machine learning approach with appropriate normalization and prepossessing increases NIDS performance. This paper presents an empirical study on various normalization methods implemented on a benchmark network traffic dataset, KDD Cup’99, that has been used to evaluate the NIDS model. The present study shows decimal normalization has a better prediction performance than non-normalized traffic data categorized into ‘normal’ or ‘intrusive’ classes.
Collapse
Affiliation(s)
- Satish Kumar
- School of Computer Science and Engineering, Shri Mata Vaishno Devi University, Kakryal, Katra, Jammu and Kashmir, India
| | - Sunanda Gupta
- School of Computer Science and Engineering, Shri Mata Vaishno Devi University, Kakryal, Katra, Jammu and Kashmir, India
| | - Sakshi Arora
- School of Computer Science and Engineering, Shri Mata Vaishno Devi University, Kakryal, Katra, Jammu and Kashmir, India
| |
Collapse
|