Cluster validity index for irregular clustering results

A clustering effectiveness measurement model based on merging similar clusters

This article presents a clustering effectiveness measurement model based on merging similar clusters to address the problems experienced by the affinity propagation (AP) algorithm in the clustering process, such as excessive local clustering, low accuracy, and invalid clustering evaluation results that occur due to the lack of variety in some internal evaluation indices when the proportion of clusters is very high. First, depending upon the "rough clustering" process of the AP clustering algorithm, similar clusters are merged according to the relationship between the similarity between any two clusters and the average inter-cluster similarity in the entire sample set to decrease the maximum number of clusters Kmax. Then, a new scheme is proposed to calculate intra-cluster compactness, inter-cluster relative density, and inter-cluster overlap coefficient. On the basis of this new method, several internal evaluation indices based on intra-cluster cohesion and inter-cluster dispersion are designed. Results of experiments show that the proposed model can perform clustering and classification correctly and provide accurate ranges for clustering using public UCI and NSL-KDD datasets, and it is significantly superior to the three improved clustering algorithms compared with it in terms of intrusion detection indices such as detection rate and false positive rate (FPR).

Mapping of structural arrangement of cells and collective calcium transients: an integrated framework combining live cell imaging using confocal microscopy and UMAP-assisted HDBSCAN-based approach

Live cell calcium (Ca2+) imaging is one of the important tools to record cellular activity during in vitro and in vivo preclinical studies. Specially, high-resolution microscopy can provide valuable dynamic information at the single cell level. One of the major challenges in the implementation of such imaging schemes is to extract quantitative information in the presence of significant heterogeneity in Ca2+ responses attained due to variation in structural arrangement and drug distribution. To fill this gap, we propose time-lapse imaging using spinning disk confocal microscopy and machine learning-enabled framework for automated grouping of Ca2+ spiking patterns. Time series analysis is performed to correlate the drug induced cellular responses to self-assembly pattern present in multicellular systems. The framework is designed to reduce the large-scale dynamic responses using uniform manifold approximation and projection (UMAP). In particular, we propose the suitability of hierarchical DBSCAN (HDBSCAN) in view of reduced number of hyperparameters. We find UMAP-assisted HDBSCAN outperforms existing approaches in terms of clustering accuracy in segregation of Ca2+ spiking patterns. One of the novelties includes the application of non-linear dimension reduction in segregation of the Ca2+ transients with statistical similarity. The proposed pipeline for automation was also proved to be a reproducible and fast method with minimal user input. The algorithm was used to quantify the effect of cellular arrangement and stimulus level on collective Ca2+ responses induced by GPCR targeting drug. The analysis revealed a significant increase in subpopulation containing sustained oscillation corresponding to higher packing density. In contrast to traditional measurement of rise time and decay ratio from Ca2+ transients, the proposed pipeline was used to classify the complex patterns with longer duration and cluster-wise model fitting. The two-step process has a potential implication in deciphering biophysical mechanisms underlying the Ca2+ oscillations in context of structural arrangement between cells.

Adaptive encoding-based evolutionary approach for Chinese document clustering

Document clustering has long been an important research direction in intelligent system. When being applied to process Chinese documents, new challenges were posted since it is infeasible to directly split the Chinese documents using the whitespace character. Moreover, many Chinese document clustering algorithms require prior knowledge of the cluster number, which is impractical to know in real-world applications. Considering these problems, we propose a general Chinese document clustering framework, where the main clustering task is fulfilled with an adaptive encoding-based evolutionary approach. Specifically, the adaptive encoding scheme is proposed to automatically learn the cluster number, and novel crossover and mutation operators are designed to fit this scheme. In addition, a single step of K-means is incorporated to conduct a joint global and local search, enhancing the overall exploitation ability. The experiments on benchmark datasets demonstrate the superiority of the proposed method in both the efficiency and the clustering precision.

Advanced Clustering Techniques for Emotional Grouping in Learning Environments Using an AR-Sandbox

Recently, it has been proven that the emotional aspect directly influences the learning process, so that, based on data mining techniques, this behavior has been sought to be characterized. This has made clustering techniques become one of the most used techniques for this purpose. However, studies where emotional data obtained from a person’s brain activity are used, are rare. For this reason, the present study aims to implement and compare advanced clustering techniques based on emotional metrics obtained through Brain-Computer Interfaces, captured in an AR-Sandbox, which fulfills the role of a learning environment. The evaluation of these techniques is carried out using internal criteria such as silhouette coefficient, Composed Density Between and within, Calinski-Harabasz and other statistical measures. When carrying out this study, it was obtained as a result that, the Density-Based Spatial Clustering of Application with Noise and Density-Based Hierarchical Spatial Clustering of Noisy Applications algorithms as the Density-based clustering methods, presented a better level of well-separation, cohesion and compaction, in comparison to the rest of the techniques implemented.

Quantitative Confocal Microscopy for Grouping of Dose-Response Data: Deciphering Calcium Sequestration and Subsequent Cell Death in the Presence of Excess Norepinephrine

Fluorescent calcium (Ca²⁺) imaging is one of the preferred methods to record cellular activity during in vitro preclinical studies, high-content drug screening, and toxicity analysis. Visualization and analysis for dose-response data obtained using high-resolution imaging remain challenging, due to the inherent heterogeneity present in the Ca²⁺ spiking. To address this challenge, we propose measurement of cytosolic Ca²⁺ ions using spinning-disk confocal microscopy and machine learning-based analytics that is scalable. First, we implemented uniform manifold approximation and projection (UMAP) for visualizing the multivariate time-series dataset in the two-dimensional (2D) plane using Python. The dataset was obtained through live imaging experiments with norepinephrine-induced Ca²⁺ oscillation in HeLa cells for a large range of doses. Second, we demonstrate that the proposed framework can be used to depict the grouping of the spiking pattern for lower and higher drug doses. To the best of our knowledge, this is the first attempt at UMAP visualization of the time-series dose response and identification of the Ca²⁺ signature during lytic death. Such quantitative microscopy can be used as a component of a high-throughput data analysis workflow for toxicity analysis.

A new validity function of FCM clustering algorithm based on intra-class compactness and inter-class separation

Fuzzy C-means (FCM) clustering algorithm is a widely used method in data mining. However, there is a big limitation that the predefined number of clustering must be given. So it is very important to find an optimal number of clusters. Therefore, a new validity function of FCM clustering algorithm is proposed to verify the validity of the clustering results. This function is defined based on the intra-class compactness and inter-class separation from the fuzzy membership matrix, the data similarity between classes and the geometric structure of the data set, whose minimum value represents the optimal clustering partition result. The proposed clustering validity function and seven traditional clustering validity functions are experimentally verified on four artificial data sets and six UCI data sets. The simulation results show that the proposed validity function can obtain the optimal clustering number of the data set more accurately, and can still find the more accurate clustering number under the condition of changing the fuzzy weighted index, which has strong adaptability and robustness.