Count data modeling and classification using finite mixtures of distributions

Bayesian Estimation of Inverted Beta Mixture Models With Extended Stochastic Variational Inference for Positive Vector Classification

The finite inverted beta mixture model (IBMM) has been proven to be efficient in modeling positive vectors. Under the traditional variational inference framework, the critical challenge in Bayesian estimation of the IBMM is that the computational cost of performing inference with large datasets is prohibitively expensive, which often limits the use of Bayesian approaches to small datasets. An efficient alternative provided by the recently proposed stochastic variational inference (SVI) framework allows for efficient inference on large datasets. Nevertheless, when using the SVI framework to address the non-Gaussian statistical models, the evidence lower bound (ELBO) cannot be explicitly calculated due to the intractable moment computation. Therefore, the algorithm under the SVI framework cannot directly use stochastic optimization to optimize the ELBO, and an analytically tractable solution cannot be derived. To address this problem, we propose an extended version of the SVI framework with more flexibility, namely, the extended SVI (ESVI) framework. This framework can be used in many non-Gaussian statistical models. First, some approximation strategies are applied to further lower the ELBO to avoid intractable moment calculations. Then, stochastic optimization with noisy natural gradients is used to optimize the lower bound. The excellent performance and effectiveness of the proposed method are verified in real data evaluation.

Brain tumour microstructure is associated with post-surgical cognition

Brain tumour microstructure is potentially predictive of changes following treatment to cognitive functions subserved by the functional networks in which they are embedded. To test this hypothesis, intra-tumoural microstructure was quantified from diffusion-weighted MRI to identify which tumour subregions (if any) had a greater impact on participants' cognitive recovery after surgical resection. Additionally, we studied the role of tumour microstructure in the functional interaction between the tumour and the rest of the brain. Sixteen patients (22-56 years, 7 females) with brain tumours located in or near speech-eloquent areas of the brain were included in the analyses. Two different approaches were adopted for tumour segmentation from a multishell diffusion MRI acquisition: the first used a two-dimensional four group partition of feature space, whilst the second used data-driven clustering with Gaussian mixture modelling. For each approach, we assessed the capability of tumour microstructure to predict participants' cognitive outcomes after surgery and the strength of association between the BOLD signal of individual tumour subregions and the global BOLD signal. With both methodologies, the volumes of partially overlapped subregions within the tumour significantly predicted cognitive decline in verbal skills after surgery. We also found that these particular subregions were among those that showed greater functional interaction with the unaffected cortex. Our results indicate that tumour microstructure measured by MRI multishell diffusion is associated with cognitive recovery after surgery.

Cross-collection latent Beta-Liouville allocation model training with privacy protection and applications

Cross-collection topic models extend previous single-collection topic models, such as Latent Dirichlet Allocation (LDA), to multiple collections. The purpose of cross-collection topic modeling is to model document-topic representations and reveal similarities between each topic and differences among groups. However, the restriction of Dirichlet prior and the significant privacy risk have hampered those models' performance and utility. Training those cross-collection topic models may, in particular, leak sensitive information from the training dataset. To address the two issues mentioned above, we propose a novel model, cross-collection latent Beta-Liouville allocation (ccLBLA), which operates a more powerful prior, Beta-Liouville distribution with a more general covariance structure that enhances topic correlation analysis. To provide privacy protection for the ccLBLA model, we leverage the inherent differential privacy guarantee of the Collapsed Gibbs Sampling (CGS) inference scheme and then propose a hybrid privacy protection algorithm for the ccLBLA model (HPP-ccLBLA) that prevents inferring data from intermediate statistics during the CGS training process without sacrificing its utility. More crucially, our technique is the first attempt to use the cross-collection topic model in image classification applications and investigate the cross-collection topic model's capabilities beyond text analysis. The experimental results for comparative text mining and image classification will show the merits of our proposed approach.

Dirichlet Process Mixture of Generalized Inverted Dirichlet Distributions for Positive Vector Data With Extended Variational Inference

A Bayesian nonparametric approach for estimation of a Dirichlet process (DP) mixture of generalized inverted Dirichlet distributions [i.e., an infinite generalized inverted Dirichlet mixture model (InGIDMM)] has been proposed. The generalized inverted Dirichlet distribution has been proven to be efficient in modeling the vectors that contain only positive elements. Under the classical variational inference (VI) framework, the key challenge in the Bayesian estimation of InGIDMM is that the expectation of the joint distribution of data and variables cannot be explicitly calculated. Therefore, numerical methods are usually applied to simulate the optimal posterior distributions. With the recently proposed extended VI (EVI) framework, we introduce lower bound approximations to the original variational objective function in the VI framework such that an analytically tractable solution can be derived. Hence, the problem in numerical simulation has been overcome. By applying the DP mixture technique, an InGIDMM can automatically determine the number of mixture components from the observed data. Moreover, the DP mixture model with an infinite number of mixture components also avoids the problems of underfitting and overfitting. The performance of the proposed approach is demonstrated with both synthesized data and real-life data applications.

Automated Nuclear Lamina Network Recognition and Quantitative Analysis in Structured Illumination Super-Resolution Microscope Images Using a Gaussian Mixture Model and Morphological Processing

Studying the architecture of nuclear lamina networks is significantly important in biomedicine owing not only to their influence on the genome, but also because they are associated with several diseases. To save labor and time, an automated method for nuclear lamina network recognition and quantitative analysis is proposed for use with lattice structured illumination super-resolution microscope images in this study. This method is based on a Gaussian mixture model and morphological processing. It includes steps for target region generation, bias field correction, image segmentation, network connection, meshwork generation, and meshwork analysis. The effectiveness of the proposed method was confirmed by recognizing and quantitatively analyzing nuclear lamina networks in five images that are presented to show the method’s performance. The experimental results show that our algorithm achieved high accuracy in nuclear lamina network recognition and quantitative analysis, and the median face areas size of lamina networks from U2OS osteosarcoma cells are 0.3184 μm2.

Insights Into Multiple/Single Lower Bound Approximation for Extended Variational Inference in Non-Gaussian Structured Data Modeling

For most of the non-Gaussian statistical models, the data being modeled represent strongly structured properties, such as scalar data with bounded support (e.g., beta distribution), vector data with unit length (e.g., Dirichlet distribution), and vector data with positive elements (e.g., generalized inverted Dirichlet distribution). In practical implementations of non-Gaussian statistical models, it is infeasible to find an analytically tractable solution to estimating the posterior distributions of the parameters. Variational inference (VI) is a widely used framework in Bayesian estimation. Recently, an improved framework, namely, the extended VI (EVI), has been introduced and applied successfully to a number of non-Gaussian statistical models. EVI derives analytically tractable solutions by introducing lower bound approximations to the variational objective function. In this paper, we compare two approximation strategies, namely, the multiple lower bounds (MLBs) approximation and the single lower bound (SLB) approximation, which can be applied to carry out the EVI. For implementation, two different conditions, the weak and the strong conditions, are discussed. Convergence of the EVI depends on the selection of the lower bound, regardless of the choice of weak or strong condition. We also discuss the convergence properties to clarify the differences between MLB and SLB. Extensive comparisons are made based on some EVI-based non-Gaussian statistical models. Theoretical analysis is conducted to demonstrate the differences between the weak and strong conditions. Experimental results based on real data show advantages of the SLB approximation over the MLB approximation.

Correlation and association analyses in microbiome study integrating multiomics in health and disease

Correlation and association analyses are one of the most widely used statistical methods in research fields, including microbiome and integrative multiomics studies. Correlation and association have two implications: dependence and co-occurrence. Microbiome data are structured as phylogenetic tree and have several unique characteristics, including high dimensionality, compositionality, sparsity with excess zeros, and heterogeneity. These unique characteristics cause several statistical issues when analyzing microbiome data and integrating multiomics data, such as large p and small n, dependency, overdispersion, and zero-inflation. In microbiome research, on the one hand, classic correlation and association methods are still applied in real studies and used for the development of new methods; on the other hand, new methods have been developed to target statistical issues arising from unique characteristics of microbiome data. Here, we first provide a comprehensive view of classic and newly developed univariate correlation and association-based methods. We discuss the appropriateness and limitations of using classic methods and demonstrate how the newly developed methods mitigate the issues of microbiome data. Second, we emphasize that concepts of correlation and association analyses have been shifted by introducing network analysis, microbe-metabolite interactions, functional analysis, etc. Third, we introduce multivariate correlation and association-based methods, which are organized by the categories of exploratory, interpretive, and discriminatory analyses and classification methods. Fourth, we focus on the hypothesis testing of univariate and multivariate regression-based association methods, including alpha and beta diversities-based, count-based, and relative abundance (or compositional)-based association analyses. We demonstrate the characteristics and limitations of each approaches. Fifth, we introduce two specific microbiome-based methods: phylogenetic tree-based association analysis and testing for survival outcomes. Sixth, we provide an overall view of longitudinal methods in analysis of microbiome and omics data, which cover standard, static, regression-based time series methods, principal trend analysis, and newly developed univariate overdispersed and zero-inflated as well as multivariate distance/kernel-based longitudinal models. Finally, we comment on current association analysis and future direction of association analysis in microbiome and multiomics studies.

Axially Symmetric Data Clustering Through Dirichlet Process Mixture Models of Watson Distributions

This paper proposes a Bayesian nonparametric framework for clustering axially symmetric data. Our approach is based on a Dirichlet processes mixture model with Watson distributions, which can also be considered as the infinite Watson mixture model. In this paper, first, we extend the finite Watson mixture model into its infinite counterpart based on the framework of truncated Dirichlet process mixture model with a stick-breaking representation. Second, we propose a coordinate ascent mean-field variational inference algorithm that can effectively learn the parameters of our model with closed-form solutions; Third, to cope with a massive data set, we develop a stochastic variational inference algorithm to learn the proposed model through the method of stochastic gradient ascent; Finally, the proposed nonparametric Bayesian model is evaluated through simulated axially symmetric data sets and a real-world application, namely, gene expression data clustering.

Improving on Deterministic Approximate Bayesian Inferences for Mixture Distributions

This paper presents the branching approach, which can improve deterministic implementation methods (such as variational Bayesian inference and expectation propagation method) for Bayesian mixture distributions. This proposed approach utilizes a set of artificial conditions defined by using the latent variables of the mixture distribution. This condition set is updated iteratively by branching based on a condition selected from the previous condition set. The target approximate Bayesian inference is obtained by merging the approximate conditional inferences under each condition in the condition set. The proposed approach is compared with several standard implementation methods by using a numerical example and a real-world example.

Capturing Spatial Interdependence in Image Features: The Counting Grid, an Epitomic Representation for Bags of Features

In recent scene recognition research images or large image regions are often represented as disorganized "bags" of features which can then be analyzed using models originally developed to capture co-variation of word counts in text. However, image feature counts are likely to be constrained in different ways than word counts in text. For example, as a camera pans upwards from a building entrance over its first few floors and then further up into the sky Fig. 1 Fig. 1. Feature counts change slightly as the field of view moves. For example, the abundance of the "car" features is reduced, but the counts of the features found on building facades are increased. The counting grid model accounts for such changes naturally, and it can also account for images of different scenes.

Online learning of a Dirichlet process mixture of Beta-Liouville distributions via variational inference

A large class of problems can be formulated in terms of the clustering process. Mixture models are an increasingly important tool in statistical pattern recognition and for analyzing and clustering complex data. Two challenging aspects that should be addressed when considering mixture models are how to choose between a set of plausible models and how to estimate the model's parameters. In this paper, we address both problems simultaneously within a unified online nonparametric Bayesian framework that we develop to learn a Dirichlet process mixture of Beta-Liouville distributions (i.e., an infinite Beta-Liouville mixture model). The proposed infinite model is used for the online modeling and clustering of proportional data for which the Beta-Liouville mixture has been shown to be effective. We propose a principled approach for approximating the intractable model's posterior distribution by a tractable one-which we develop-such that all the involved mixture's parameters can be estimated simultaneously and effectively in a closed form. This is done through variational inference that enjoys important advantages, such as handling of unobserved attributes and preventing under or overfitting; we explain that in detail. The effectiveness of the proposed work is evaluated on three challenging real applications, namely facial expression recognition, behavior modeling and recognition, and dynamic textures clustering.

Incorporating mean template into finite mixture model for image segmentation

The well-known finite mixture model (FMM) has been regarded as a useful tool for image segmentation application. However, the pixels in FMM are considered independent of each other and the spatial relationship between neighboring pixels is not taken into account. These limitations make the FMM more sensitive to noise. In this brief, we propose a simple and effective method to make the traditional FMM more robust to noise with the help of a mean template. FMM can be considered a linear combination of prior and conditional probability from the expression of its mathematical formula. We calculate these probabilities with two mean templates: a weighted arithmetic mean template and a weighted geometric mean template. Thus, in our model, the prior probability (or conditional probability) of an image pixel is influenced by the probabilities of pixels in its immediate neighborhood to incorporate the local spatial and intensity information for eliminating the noise. Finally, our algorithm is general enough and can be extended to any other FMM-based models to achieve super performance. Experimental results demonstrate the improved robustness and effectiveness of our approach.

Dirichlet multinomial mixtures: generative models for microbial metagenomics

We introduce Dirichlet multinomial mixtures (DMM) for the probabilistic modelling of microbial metagenomics data. This data can be represented as a frequency matrix giving the number of times each taxa is observed in each sample. The samples have different size, and the matrix is sparse, as communities are diverse and skewed to rare taxa. Most methods used previously to classify or cluster samples have ignored these features. We describe each community by a vector of taxa probabilities. These vectors are generated from one of a finite number of Dirichlet mixture components each with different hyperparameters. Observed samples are generated through multinomial sampling. The mixture components cluster communities into distinct 'metacommunities', and, hence, determine envirotypes or enterotypes, groups of communities with a similar composition. The model can also deduce the impact of a treatment and be used for classification. We wrote software for the fitting of DMM models using the 'evidence framework' (http://code.google.com/p/microbedmm/). This includes the Laplace approximation of the model evidence. We applied the DMM model to human gut microbe genera frequencies from Obese and Lean twins. From the model evidence four clusters fit this data best. Two clusters were dominated by Bacteroides and were homogenous; two had a more variable community composition. We could not find a significant impact of body mass on community structure. However, Obese twins were more likely to derive from the high variance clusters. We propose that obesity is not associated with a distinct microbiota but increases the chance that an individual derives from a disturbed enterotype. This is an example of the 'Anna Karenina principle (AKP)' applied to microbial communities: disturbed states having many more configurations than undisturbed. We verify this by showing that in a study of inflammatory bowel disease (IBD) phenotypes, ileal Crohn's disease (ICD) is associated with a more variable community.

Real-time vector quantization and clustering based on ordinary differential equations

This brief presents a dynamical system approach to vector quantization or clustering based on ordinary differential equations with the potential for real-time implementation. Two examples of different pattern clusters demonstrate that the model can successfully quantize different types of input patterns. Furthermore, we analyze and study the stability of our dynamical system. By discovering the equilibrium points for certain input patterns and analyzing their stability, we have shown the quantizing behavior of the system with respect to its vigilance parameter. The proposed system is applied to two real-world problems, providing comparable results to the best reported findings. This validates the effectiveness of our proposed approach.