Bayesian model selection for complex dynamic systems

Neural superstatistics for Bayesian estimation of dynamic cognitive models

Mathematical models of cognition are often memoryless and ignore potential fluctuations of their parameters. However, human cognition is inherently dynamic. Thus, we propose to augment mechanistic cognitive models with a temporal dimension and estimate the resulting dynamics from a superstatistics perspective. Such a model entails a hierarchy between a low-level observation model and a high-level transition model. The observation model describes the local behavior of a system, and the transition model specifies how the parameters of the observation model evolve over time. To overcome the estimation challenges resulting from the complexity of superstatistical models, we develop and validate a simulation-based deep learning method for Bayesian inference, which can recover both time-varying and time-invariant parameters. We first benchmark our method against two existing frameworks capable of estimating time-varying parameters. We then apply our method to fit a dynamic version of the diffusion decision model to long time series of human response times data. Our results show that the deep learning approach is very efficient in capturing the temporal dynamics of the model. Furthermore, we show that the erroneous assumption of static or homogeneous parameters will hide important temporal information.

A novel integrated experimental and computational approach to unravel fibroblast motility in response to chemical gradients in 3D collagen matrices

Fibroblasts play an essential role in tissue repair and regeneration as they migrate to wounded areas to secrete and remodel the extracellular matrix. Fibroblasts recognize chemical substances such as growth factors, which enhance their motility towards the wounded tissues through chemotaxis. Although several studies have characterized single-cell fibroblast motility before, the migration patterns of fibroblasts in response to external factors have not been fully explored in 3D environments. We present a study that combines experimental and computational efforts to characterize the effect of chemical stimuli on the invasion of 3D collagen matrices by fibroblasts. Experimentally, we used microfluidic devices to create chemical gradients using collagen matrices of distinct densities. We evaluated how cell migration patterns were affected by the presence of growth factors and the mechanical properties of the matrix. Based on these results, we present a discrete-based computational model to simulate cell motility, which we calibrated through the quantitative comparison of experimental and computational data via Bayesian optimization. By combining these approaches, we predict that fibroblasts respond to both the presence of chemical factors and their spatial location. Furthermore, our results show that the presence of these chemical gradients could be reproduced by our computational model through increases in the magnitude of cell-generated forces and enhanced cell directionality. Although these model predictions require further experimental validation, we propose that our framework can be applied as a tool that takes advantage of experimental data to guide the calibration of models and predict which mechanisms at the cellular level may justify the experimental findings. Consequently, these new insights may also guide the design of new experiments, tailored to validate the variables of interest identified by the model.

Classification at the accuracy limit: facing the problem of data ambiguity

Data classification, the process of analyzing data and organizing it into categories or clusters, is a fundamental computing task of natural and artificial information processing systems. Both supervised classification and unsupervised clustering work best when the input vectors are distributed over the data space in a highly non-uniform way. These tasks become however challenging in weakly structured data sets, where a significant fraction of data points is located in between the regions of high point density. We derive the theoretical limit for classification accuracy that arises from this overlap of data categories. By using a surrogate data generation model with adjustable statistical properties, we show that sufficiently powerful classifiers based on completely different principles, such as perceptrons and Bayesian models, all perform at this universal accuracy limit under ideal training conditions. Remarkably, the accuracy limit is not affected by certain non-linear transformations of the data, even if these transformations are non-reversible and drastically reduce the information content of the input data. We further compare the data embeddings that emerge by supervised and unsupervised training, using the MNIST data set and human EEG recordings during sleep. We find for MNIST that categories are significantly separated not only after supervised training with back-propagation, but also after unsupervised dimensionality reduction. A qualitatively similar cluster enhancement by unsupervised compression is observed for the EEG sleep data, but with a very small overall degree of cluster separation. We conclude that the handwritten letters in MNIST can be considered as 'natural kinds', whereas EEG sleep recordings are a relatively weakly structured data set, so that unsupervised clustering will not necessarily re-cover the human-defined sleep stages.

Unravelling cell migration: defining movement from the cell surface

Cell motility is essential for life and development. Unfortunately, cell migration is also linked to several pathological processes, such as cancer metastasis. Cells' ability to migrate relies on many actors. Cells change their migratory strategy based on their phenotype and the properties of the surrounding microenvironment. Cell migration is, therefore, an extremely complex phenomenon. Researchers have investigated cell motility for more than a century. Recent discoveries have uncovered some of the mysteries associated with the mechanisms involved in cell migration, such as intracellular signaling and cell mechanics. These findings involve different players, including transmembrane receptors, adhesive complexes, cytoskeletal components , the nucleus, and the extracellular matrix. This review aims to give a global overview of our current understanding of cell migration.

Sleep as a random walk: a super-statistical analysis of EEG data across sleep stages

In clinical practice, human sleep is classified into stages, each associated with different levels of muscular activity and marked by characteristic patterns in the EEG signals. It is however unclear whether this subdivision into discrete stages with sharply defined boundaries is truly reflecting the dynamics of human sleep. To address this question, we consider one-channel EEG signals as heterogeneous random walks: stochastic processes controlled by hyper-parameters that are themselves time-dependent. We first demonstrate the heterogeneity of the random process by showing that each sleep stage has a characteristic distribution and temporal correlation function of the raw EEG signals. Next, we perform a super-statistical analysis by computing hyper-parameters, such as the standard deviation, kurtosis, and skewness of the raw signal distributions, within subsequent 30-second epochs. It turns out that also the hyper-parameters have characteristic, sleep-stage-dependent distributions, which can be exploited for a simple Bayesian sleep stage detection. Moreover, we find that the hyper-parameters are not piece-wise constant, as the traditional hypnograms would suggest, but show rising or falling trends within and across sleep stages, pointing to an underlying continuous rather than sub-divided process that controls human sleep. Based on the hyper-parameters, we finally perform a pairwise similarity analysis between the different sleep stages, using a quantitative measure for the separability of data clusters in multi-dimensional spaces.

To improve our understand of how EEG activity reflects the dynamics of human sleep, Metzner et al. use human EEG data and superstatistical analysis to demonstrate that each sleep stage has a characteristic distribution and temporal correlation function of raw EEG signals. They also show that the hyper-parameters controlling the EEG signals have characteristic, sleep-stage-dependent distributions, which can be exploited for a simple Bayesian sleep stage detection.

Analysis of the early response to spinal cord injury identified a key role for mTORC1 signaling in the activation of neural stem progenitor cells

Xenopus laevis are able to regenerate the spinal cord during larvae stages through the activation of neural stem progenitor cells (NSPCs). Here we use high-resolution expression profiling to characterize the early transcriptome changes induced after spinal cord injury, aiming to identify the signals that trigger NSPC proliferation. The analysis delineates a pathway that starts with a rapid and transitory activation of immediate early genes, followed by migration processes and immune response genes, the pervasive increase of NSPC-specific ribosome biogenesis factors, and genes involved in stem cell proliferation. Western blot and immunofluorescence analysis showed that mTORC1 is rapidly and transiently activated after SCI, and its pharmacological inhibition impairs spinal cord regeneration and proliferation of NSPC through the downregulation of genes involved in the G1/S transition of cell cycle, with a strong effect on PCNA. We propose that the mTOR signaling pathway is a key player in the activation of NPSCs during the early steps of spinal cord regeneration.

Integrated in silico and 3D in vitro model of macrophage migration in response to physical and chemical factors in the tumor microenvironment

Macrophages are abundant in the tumor microenvironment (TME), serving as accomplices to cancer cells for their invasion. Studies have explored the biochemical mechanisms that drive pro-tumor macrophage functions; however the role of TME interstitial flow (IF) is often disregarded. Therefore, we developed a three-dimensional microfluidic-based model with tumor cells and macrophages to study how IF affects macrophage migration and its potential contribution to cancer invasion. The presence of either tumor cells or IF individually increased macrophage migration directedness and speed. Interestingly, there was no additive effect on macrophage migration directedness and speed under the simultaneous presence of tumor cells and IF. Further, we present an in silico model that couples chemokine-mediated signaling with mechanosensing networks to explain our in vitro observations. In our model design, we propose IL-8, CCL2, and β-integrin as key pathways that commonly regulate various Rho GTPases. In agreement, in vitro macrophage migration remained elevated when exposed to a saturating concentration of recombinant IL-8 or CCL2 or to the co-addition of a sub-saturating concentration of both cytokines. Moreover, antibody blockade against IL-8 and/or CCL2 inhibited migration that could be restored by IF, indicating cytokine-independent mechanisms of migration induction. Importantly, we demonstrate the utility of an integrated in silico and 3D in vitro approach to aid the design of tumor-associated macrophage-based immunotherapeutic strategies.

Integrating Quantitative Assays with Biologically Based Mathematical Modeling for Predictive Oncology

We provide an overview on the use of biological assays to calibrate and initialize mechanism-based models of cancer phenomena. Although artificial intelligence methods currently dominate the landscape in computational oncology, mathematical models that seek to explicitly incorporate biological mechanisms into their formalism are of increasing interest. These models can guide experimental design and provide insights into the underlying mechanisms of cancer progression. Historically, these models have included a myriad of parameters that have been difficult to quantify in biologically relevant systems, limiting their practical insights. Recently, however, there has been much interest calibrating biologically based models with the quantitative measurements available from (for example) RNA sequencing, time-resolved microscopy, and in vivo imaging. In this contribution, we summarize how a variety of experimental methods quantify tumor characteristics from the molecular to tissue scales and describe how such data can be directly integrated with mechanism-based models to improve predictions of tumor growth and treatment response.

On the efficiency of chemotactic pursuit - Comparing blind search with temporal and spatial gradient sensing

In chemotaxis, cells are modulating their migration patterns in response to concentration gradients of a guiding substance. Immune cells are believed to use such chemotactic sensing for remotely detecting and homing in on pathogens. Considering that immune cells may encounter a multitude of targets with vastly different migration properties, ranging from immobile to highly mobile, it is not clear which strategies of chemotactic pursuit are simultaneously efficient and versatile. We tackle this problem theoretically and define a tunable response function that maps temporal or spatial concentration gradients to migration behavior. The seven free parameters of this response function are optimized numerically with the objective of maximizing search efficiency against a wide spectrum of target cell properties. Finally, we reverse-engineer the best-performing parameter sets to uncover strategies of chemotactic pursuit that are efficient under different biologically realistic boundary conditions. Although strategies based on the temporal or spatial sensing of chemotactic gradients are significantly more efficient than unguided migration, such ‘blind search’ turns out to work surprisingly well, in particular if the immune cells are fast and directionally persistent. The resulting simulated data can be used for the design of chemotaxis experiments and for the development of algorithms that automatically detect and quantify goal oriented behavior in measured immune cell trajectories.

On the Use of Entropy to Improve Model Selection Criteria

The most widely used forms of model selection criteria, the Bayesian Information Criterion (BIC) and the Akaike Information Criterion (AIC), are expressed in terms of synthetic indicators of the residual distribution: the variance and the mean-squared error of the residuals respectively. In many applications in science, the noise affecting the data can be expected to have a Gaussian distribution. Therefore, at the same level of variance and mean-squared error, models, whose residuals are more uniformly distributed, should be favoured. The degree of uniformity of the residuals can be quantified by the Shannon entropy. Including the Shannon entropy in the BIC and AIC expressions improves significantly these criteria. The better performances have been demonstrated empirically with a series of simulations for various classes of functions and for different levels and statistics of the noise. In presence of outliers, a better treatment of the errors, using the Geodesic Distance, has proved essential.

Non-Gaussian, non-ergodic, and non-Fickian diffusion of tracers in mucin hydrogels

Native mucus is polymer-based soft-matter material of paramount biological importance. How non-Gaussian and non-ergodic is the diffusive spreading of pathogens in mucus? We study the passive, thermally driven motion of micron-sized tracers in hydrogels of mucins, the main polymeric component of mucus. We report the results of the Bayesian analysis for ranking several diffusion models for a set of tracer trajectories [C. E. Wagner et al., Biomacromolecules, 2017, 18, 3654]. The models with "diffusing diffusivity", fractional and standard Brownian motion are used. The likelihood functions and evidences of each model are computed, ranking the significance of each model for individual traces. We find that viscoelastic anomalous diffusion is often most probable, followed by Brownian motion, while the model with a diffusing diffusion coefficient is only realised rarely. Our analysis also clarifies the distribution of time-averaged displacements, correlations of scaling exponents and diffusion coefficients, and the degree of non-Gaussianity of displacements at varying pH levels. Weak ergodicity breaking is also quantified. We conclude that-consistent with the original study-diffusion of tracers in the mucin gels is most non-Gaussian and non-ergodic at low pH that corresponds to the most heterogeneous networks. Using the Bayesian approach with the nested-sampling algorithm, together with the quantitative analysis of multiple statistical measures, we report new insights into possible physical mechanisms of diffusion in mucin gels.

Bayesian analysis of single-particle tracking data using the nested-sampling algorithm: maximum-likelihood model selection applied to stochastic-diffusivity data

We employ Bayesian statistics using the nested-sampling algorithm to compare and rank multiple models of ergodic diffusion (including anomalous diffusion) as well as to assess their optimal parameters for in silico-generated and real time-series. We focus on the recently-introduced model of Brownian motion with "diffusing diffusivity"-giving rise to widely-observed non-Gaussian displacement statistics-and its comparison to Brownian and fractional Brownian motion, also for the time-series with some measurement noise. We conduct this model-assessment analysis using Bayesian statistics and the nested-sampling algorithm on the level of individual particle trajectories. We evaluate relative model probabilities and compute best-parameter sets for each diffusion model, comparing the estimated parameters to the true ones. We test the performance of the nested-sampling algorithm and its predictive power both for computer-generated (idealised) trajectories as well as for real single-particle-tracking trajectories. Our approach delivers new important insight into the objective selection of the most suitable stochastic model for a given time-series. We also present first model-ranking results in application to experimental data of tracer diffusion in polymer-based hydrogels.

Forecasting extreme atmospheric events with a recurrence-interval-analysis-based autoregressive conditional duration model

With most city dwellers in China subjected to air pollution, forecasting extreme air pollution spells is of paramount significance in both scheduling outdoor activities and ameliorating air pollution. In this paper, we integrate the autoregressive conditional duration model (ACD) with the recurrence interval analysis (RIA) and also extend the ACD model to a spatially autoregressive conditional duration (SACD) model by adding a spatially reviewed term to quantitatively explain and predict extreme air pollution recurrence intervals. Using the hourly data of six pollutants and the air quality index (AQI) during 2013–2016 collected from 12 national air quality monitoring stations in Beijing as our test samples, we attest that the spatially reviewed recurrence intervals have some general explanatory power over the recurrence intervals in the neighbouring air quality monitoring stations. We also conduct a one-step forecast using the RIA-ACD(1,1) and RIA-SACD(1,1,1) models and find that 90% of the predicted recurrence intervals are smaller than 72 hours, which justifies the predictive power of the proposed models. When applied to more time lags and neighbouring stations, the models are found to yield results that are consistent with reality, which evinces the feasibility of predicting extreme air pollution events through a recurrence-interval-analysis-based autoregressive conditional duration model. Moreover, the addition of a spatial term has proved effective in enhancing the predictive power.

Integration of in vitro and in silico Models Using Bayesian Optimization With an Application to Stochastic Modeling of Mesenchymal 3D Cell Migration

Cellular migration plays a crucial role in many aspects of life and development. In this paper, we propose a computational model of 3D migration that is solved by means of the tau-leaping algorithm and whose parameters have been calibrated using Bayesian optimization. Our main focus is two-fold: to optimize the numerical performance of the mechano-chemical model as well as to automate the calibration process of in silico models using Bayesian optimization. The presented mechano-chemical model allows us to simulate the stochastic behavior of our chemically reacting system in combination with mechanical constraints due to the surrounding collagen-based matrix. This numerical model has been used to simulate fibroblast migration. Moreover, we have performed in vitro analysis of migrating fibroblasts embedded in 3D collagen-based fibrous matrices (2 mg/ml). These in vitro experiments have been performed with the main objective of calibrating our model. Nine model parameters have been calibrated testing 300 different parametrizations using a completely automatic approach. Two competing evaluation metrics based on the Bhattacharyya coefficient have been defined in order to fit the model parameters. These metrics evaluate how accurately the in silico model is replicating in vitro measurements regarding the two main variables quantified in the experimental data (number of protrusions and the length of the longest protrusion). The selection of an optimal parametrization is based on the balance between the defined evaluation metrics. Results show how the calibrated model is able to predict the main features observed in the in vitro experiments.