Emergent communication of multimodal deep generative models based on Metropolis-Hastings naming game

Deep generative models (DGM) are increasingly employed in emergent communication systems. However, their application in multimodal data contexts is limited. This study proposes a novel model that combines multimodal DGM with the Metropolis-Hastings (MH) naming game, enabling two agents to focus jointly on a shared subject and develop common vocabularies. The model proves that it can handle multimodal data, even in cases of missing modalities. Integrating the MH naming game with multimodal variational autoencoders (VAE) allows agents to form perceptual categories and exchange signs within multimodal contexts. Moreover, fine-tuning the weight ratio to favor a modality that the model could learn and categorize more readily improved communication. Our evaluation of three multimodal approaches - mixture-of-experts (MoE), product-of-experts (PoE), and mixture-of-product-of-experts (MoPoE)-suggests an impact on the creation of latent spaces, the internal representations of agents. Our results from experiments with the MNIST + SVHN and Multimodal165 datasets indicate that combining the Gaussian mixture model (GMM), PoE multimodal VAE, and MH naming game substantially improved information sharing, knowledge formation, and data reconstruction.

Understanding the transmission of African swine fever in wild boars of South Korea: A simulation study for parameter estimation

Since the introduction of African swine fever (ASF) to Georgia in 2007, the disease has spread to many other countries including South Korea. Initial detection of ASF from wild boars (WB) in South Korea was reported in early October 2019. Since then, more than a thousand WB samples collected from the northern part of the country have been confirmed as ASF positive (2.9% of ASF positivity among WB samples collected until June 2020), indicating that the disease is endemic in the WB populations. To control the disease, multiple layers of fence-lines have been erected. Nevertheless, outbreaks continuously occurred across the fence, requiring a better understanding of the spatial transmission mechanism of ASF in WBs. Hence, we developed a novel ASF transmission model to estimate ecological and epidemiological parameters related to the spread of the disease in the WB population of South Korea. The results showed that roads and rivers were effective to prevent the spread, reducing the transmission rate to approximately 37% on average. Only a limited level of reduction was indicated via fence-lines, implying erection of fences might be considered as a temporary measure to delay the spread. This study also revealed that the probability of ASF transmission to adjacent habitats considerably decreased with increasing distance, supporting the slow spatial transmission speed reported from other European countries. Considering that elucidation of ASF dynamics in WB is crucial to mitigate the impact of the disease, we believe this study provides some useful ecological and epidemiological implications to control the disease in future.

Efficient estimation of bounded gradient-drift diffusion models for affect on CPU and GPU

Computational modeling plays an important role in a gamut of research fields. In affect research, continuous-time stochastic models are becoming increasingly popular. Recently, a non-linear, continuous-time, stochastic model has been introduced for affect dynamics, called the Affective Ising Model (AIM). The drawback of non-linear models like the AIM is that they generally come with serious computational challenges for parameter estimation and related statistical analyses. The likelihood function of the AIM does not have a closed form expression. Consequently, simulation based or numerical methods have to be considered in order to evaluate the likelihood function. Additionally, the likelihood function can have multiple local minima. Consequently, a global optimization heuristic is required and such heuristics generally require a large number of likelihood function evaluations. In this paper, a Julia software package is introduced that is dedicated to fitting the AIM. The package includes an implementation of a numeric algorithm for fast computations of the likelihood function, which can be run both on graphics processing units (GPU) and central processing units (CPU). The numerical method introduced in this paper is compared to the more traditional Euler-Maruyama method for solving stochastic differential equations. Furthermore, the estimation software is tested by means of a recovery study and estimation times are reported for benchmarks that were run on several computing devices (two different GPUs and three different CPUs). According to these results, a single parameter estimation can be obtained in less than thirty seconds using a mainstream NVIDIA GPU.

Bayesian Computational Methods for Sampling from the Posterior Distribution of a Bivariate Survival Model, Based on AMH Copula in the Presence of Right-Censored Data

In this paper, we study the performance of Bayesian computational methods to estimate the parameters of a bivariate survival model based on the Ali-Mikhail-Haq copula with marginal distributions given by Weibull distributions. The estimation procedure was based on Monte Carlo Markov Chain (MCMC) algorithms. We present three version of the Metropolis-Hastings algorithm: Independent Metropolis-Hastings (IMH), Random Walk Metropolis (RWM) and Metropolis-Hastings with a natural-candidate generating density (MH). Since the creation of a good candidate generating density in IMH and RWM may be difficult, we also describe how to update a parameter of interest using the slice sampling (SS) method. A simulation study was carried out to compare the performances of the IMH, RWM and SS. A comparison was made using the sample root mean square error as an indicator of performance. Results obtained from the simulations show that the SS algorithm is an effective alternative to the IMH and RWM methods when simulating values from the posterior distribution, especially for small sample sizes. We also applied these methods to a real data set.

Bayesian inference in time-varying additive hazards models with applications to disease mapping

Environmental health and disease mapping studies are often concerned with the evaluation of the combined effect of various socio-demographic and behavioral factors, and environmental exposures on time-to-events of interest, such as death of individuals, organisms or plants. In such studies, estimation of the hazard function is often of interest. In addition to known explanatory variables, the hazard function maybe subject to spatial/geographical variations, such that proximally located regions may experience similar hazards than regions that are distantly located. A popular approach for handling this type of spatially-correlated time-to-event data is the Cox's Proportional Hazards (PH) regression model with spatial frailties. However, the PH assumption poses a major practical challenge, as it entails that the effects of the various explanatory variables remain constant over time. This assumption is often unrealistic, for instance, in studies with long follow-ups where the effects of some exposures on the hazard may vary drastically over time. Our goal in this paper is to offer a flexible semiparametric additive hazards model (AH) with spatial frailties. Our proposed model allows both the frailties as well as the regression coefficients to be time-varying, thus relaxing the proportionality assumption. Our estimation framework is Bayesian, powered by carefully tailored posterior sampling strategies via Markov chain Monte Carlo techniques. We apply the model to a dataset on prostate cancer survival from the US state of Louisiana to illustrate its advantages.

Improving the generation and selection of virtual populations in quantitative systems pharmacology models

Quantitative systems pharmacology (QSP) models aim to describe mechanistically the pathophysiology of disease and predict the effects of therapies on that disease. For most drug development applications, it is important to predict not only the mean response to an intervention but also the distribution of responses, due to inter-patient variability. Given the necessary complexity of QSP models, and the sparsity of relevant human data, the parameters of QSP models are often not well determined. One approach to overcome these limitations is to develop alternative virtual patients (VPs) and virtual populations (Vpops), which allow for the exploration of parametric uncertainty and reproduce inter-patient variability in response to perturbation. Here we evaluated approaches to improve the efficiency of generating Vpops. We aimed to generate Vpops without sacrificing diversity of the VPs' pathophysiologies and phenotypes. To do this, we built upon a previously published approach (Allen et al., 2016) by (a) incorporating alternative optimization algorithms (genetic algorithm and Metropolis-Hastings) or alternatively (b) augmenting the optimized objective function. Each method improved the baseline algorithm by requiring significantly fewer plausible patients (precursors to VPs) to create a reasonable Vpop.

Improving model prediction of soil N2O emissions through Bayesian calibration

The biogeochemical processes that lead to the production of N₂O in arable soils are controlled by temporally and spatially varying drivers. The need for prediction of soil N₂O emissions across scales means that agroecosystem biogeochemistry models are widely used to simulate N₂O emissions. Due to the parameter-dense nature of agroecosystem models their parameters have to be calibrated according to the soil and climatic conditions of the intended area of application. Bayesian calibration is considered one of the most advanced ways to complete this task. In this study, we calibrate nine parameters of the Landscape-DNDC process-based agroecosystem model, which are key to its N₂O prediction. The Metropolis-Hastings algorithm is used at four separate implementations in order to estimate parameter posterior distributions at four arable sites in the UK. The results of this process are visualised, summarised and assessed against measured N₂O data from ten independent arable sites. The study shows that, in many cases, soil N₂O emission peaks that were not predicted with the default model parameters were predicted after calibration. Overall, the prediction of soil N₂O fluxes across all the sites that were considered was improved by 33% when using the calibrated parameters.

Simultaneous estimation of population receptive field and hemodynamic parameters from single point BOLD responses using Metropolis-Hastings sampling

We introduce a new approach to Bayesian pRF model estimation using Markov Chain Monte Carlo (MCMC) sampling for simultaneous estimation of pRF and hemodynamic parameters. To obtain high performance on commonly accessible hardware we present a novel heuristic consisting of interpolation between precomputed responses for predetermined stimuli and a large cross-section of receptive field parameters. We investigate the validity of the proposed approach with respect to MCMC convergence, tuning and biases. We compare different combinations of pRF - Compressive Spatial Summation (CSS), Dumoulin-Wandell (DW) and hemodynamic (5-parameter and 3-parameter Balloon-Windkessel) models within our framework with and without the usage of the new heuristic. We evaluate estimation consistency and log probability across models. We perform as well a comparison of one model with and without lookup table within the RStan framework using its No-U-Turn Sampler. We present accelerated computation of whole-ROI parameters for one subject. Finally, we discuss risks and limitations associated with the usage of the new heuristic as well as the means of resolving them. We found that the new algorithm is a valid sampling approach to joint pRF/hemodynamic parameter estimation and that it exhibits very high performance.

Bayesian variable selection for a semi-competing risks model with three hazard functions

A variable selection procedure is developed for a semi-competing risks regression model with three hazard functions that uses spike-and-slab priors and stochastic search variable selection algorithms for posterior inference. A rule is devised for choosing the threshold on the marginal posterior probability of variable inclusion based on the Deviance Information Criterion (DIC) that is examined in a simulation study. The method is applied to data from esophageal cancer patients from the MD Anderson Cancer Center, Houston, TX, where the most important covariates are selected in each of the hazards of effusion, death before effusion, and death after effusion. The DIC procedure that is proposed leads to similar selected models regardless of the choices of some of the hyperparameters. The application results show that patients with intensity-modulated radiation therapy have significantly reduced risks of pericardial effusion, pleural effusion, and death before either effusion type.

VMCMC: a graphical and statistical analysis tool for Markov chain Monte Carlo traces

Background

MCMC-based methods are important for Bayesian inference of phylogeny and related parameters. Although being computationally expensive, MCMC yields estimates of posterior distributions that are useful for estimating parameter values and are easy to use in subsequent analysis. There are, however, sometimes practical difficulties with MCMC, relating to convergence assessment and determining burn-in, especially in large-scale analyses. Currently, multiple software are required to perform, e.g., convergence, mixing and interactive exploration of both continuous and tree parameters.

Results

We have written a software called VMCMC to simplify post-processing of MCMC traces with, for example, automatic burn-in estimation. VMCMC can also be used both as a GUI-based application, supporting interactive exploration, and as a command-line tool suitable for automated pipelines.

Conclusions

VMCMC is a free software available under the New BSD License. Executable jar files, tutorial manual and source code can be downloaded from https://bitbucket.org/rhali/visualmcmc/.

Electronic supplementary material

The online version of this article (doi:10.1186/s12859-017-1505-3) contains supplementary material, which is available to authorized users.

A Bootstrap Metropolis-Hastings Algorithm for Bayesian Analysis of Big Data

Markov chain Monte Carlo (MCMC) methods have proven to be a very powerful tool for analyzing data of complex structures. However, their computer-intensive nature, which typically require a large number of iterations and a complete scan of the full dataset for each iteration, precludes their use for big data analysis. In this paper, we propose the so-called bootstrap Metropolis-Hastings (BMH) algorithm, which provides a general framework for how to tame powerful MCMC methods to be used for big data analysis; that is to replace the full data log-likelihood by a Monte Carlo average of the log-likelihoods that are calculated in parallel from multiple bootstrap samples. The BMH algorithm possesses an embarrassingly parallel structure and avoids repeated scans of the full dataset in iterations, and is thus feasible for big data problems. Compared to the popular divide-and-combine method, BMH can be generally more efficient as it can asymptotically integrate the whole data information into a single simulation run. The BMH algorithm is very flexible. Like the Metropolis-Hastings algorithm, it can serve as a basic building block for developing advanced MCMC algorithms that are feasible for big data problems. This is illustrated in the paper by the tempering BMH algorithm, which can be viewed as a combination of parallel tempering and the BMH algorithm. BMH can also be used for model selection and optimization by combining with reversible jump MCMC and simulated annealing, respectively.