Bayesian parameter estimation for nonlinear modelling of biological pathways

Systematic Bayesian posterior analysis guided by Kullback-Leibler divergence facilitates hypothesis formation

Bayesian inference produces a posterior distribution for the parameters of a mathematical model that can be used to guide the formation of hypotheses; specifically, the posterior may be searched for evidence of alternative model hypotheses, which serves as a starting point for hypothesis formation and model refinement. Previous approaches to search for this evidence are largely qualitative and unsystematic; further, demonstrations of these approaches typically stop at hypothesis formation, leaving the questions they raise unanswered. Here, we introduce a Kullback-Leibler (KL) divergence-based ranking to expedite Bayesian hypothesis formation and investigate the hypotheses it generates, ultimately generating novel, biologically significant insights. Our approach uses KL divergence to rank parameters by how much information they gain from experimental data. Subsequently, rather than searching all model parameters at random, we use this ranking to prioritize examining the posteriors of the parameters that gained the most information from the data for evidence of alternative model hypotheses. We test our approach with two examples, which showcase the ability of our approach to systematically uncover different types of alternative hypothesis evidence. First, we test our KL divergence ranking on an established example of Bayesian hypothesis formation. Our top-ranked parameter matches the one previously identified to produce alternative hypotheses. In the second example, we apply our ranking in a novel study of a computational model of prolactin-induced JAK2-STAT5 signaling, a pathway that mediates beta cell proliferation. Within the top 3 ranked parameters (out of 33), we find a bimodal posterior revealing two possible ranges for the prolactin receptor degradation rate. We go on to refine the model, incorporating new data and determining which degradation rate is most plausible. Overall, while the effectiveness of our approach depends on having a properly formulated prior and on the form of the posterior distribution, we demonstrate that our approach offers a novel and generalizable quantitative framework for Bayesian hypothesis formation and use it to produce a novel, biologically-significant insight into beta cell signaling.

Bayesian parameter estimation for dynamical models in systems biology

Dynamical systems modeling, particularly via systems of ordinary differential equations, has been used to effectively capture the temporal behavior of different biochemical components in signal transduction networks. Despite the recent advances in experimental measurements, including sensor development and '-omics' studies that have helped populate protein-protein interaction networks in great detail, modeling in systems biology lacks systematic methods to estimate kinetic parameters and quantify associated uncertainties. This is because of multiple reasons, including sparse and noisy experimental measurements, lack of detailed molecular mechanisms underlying the reactions, and missing biochemical interactions. Additionally, the inherent nonlinearities with respect to the states and parameters associated with the system of differential equations further compound the challenges of parameter estimation. In this study, we propose a comprehensive framework for Bayesian parameter estimation and complete quantification of the effects of uncertainties in the data and models. We apply these methods to a series of signaling models of increasing mathematical complexity. Systematic analysis of these dynamical systems showed that parameter estimation depends on data sparsity, noise level, and model structure, including the existence of multiple steady states. These results highlight how focused uncertainty quantification can enrich systems biology modeling and enable additional quantitative analyses for parameter estimation.

An optimal control approach for enhancing natural killer cells' secretion of cytolytic molecules

Natural killer (NK) cells are immune effector cells that can detect and lyse cancer cells. However, NK cell exhaustion, a phenotype characterized by reduced secretion of cytolytic models upon serial stimulation, limits the NK cell's ability to lyse cells. In this work, we investigated in silico strategies that counteract the NK cell's reduced secretion of cytolytic molecules. To accomplish this goal, we constructed a mathematical model that describes the dynamics of the cytolytic molecules granzyme B (GZMB) and perforin-1 (PRF1) and calibrated the model predictions to published experimental data using a Bayesian parameter estimation approach. We applied an information-theoretic approach to perform a global sensitivity analysis, from which we found that the suppression of phosphatase activity maximizes the secretion of GZMB and PRF1. However, simply reducing the phosphatase activity is shown to deplete the cell's intracellular pools of GZMB and PRF1. Thus, we added a synthetic Notch (synNotch) signaling circuit to our baseline model as a method for controlling the secretion of GZMB and PRF1 by inhibiting phosphatase activity and increasing production of GZMB and PRF1. We found that the optimal synNotch system depends on the frequency of NK cell stimulation. For only a few rounds of stimulation, the model predicts that inhibition of phosphatase activity leads to more secreted GZMB and PRF1; however, for many rounds of stimulation, the model reveals that increasing production of the cytolytic molecules is the optimal strategy. In total, we developed a mathematical framework that provides actionable insight into engineering robust NK cells for clinical applications.

Dynamic Regulation of JAK-STAT Signaling Through the Prolactin Receptor Predicted by Computational Modeling

Introduction

The expansion of insulin-producing beta cells during pregnancy is critical to maintain glucose homeostasis in the face of increasing insulin resistance. Prolactin receptor (PRLR) signaling is one of the primary mediators of beta cell expansion during pregnancy, and loss of PRLR signaling results in reduced beta cell mass and gestational diabetes. Harnessing the proliferative potential of prolactin signaling to expand beta cell mass outside of the context of pregnancy requires quantitative understanding of the signaling at the molecular level.

Methods

A mechanistic computational model was constructed to describe prolactin-mediated JAK-STAT signaling in pancreatic beta cells. The effect of different regulatory modules was explored through ensemble modeling. A Bayesian approach for likelihood estimation was used to fit the model to experimental data from the literature.

Results

Including receptor upregulation, with either inhibition by SOCS proteins, receptor internalization, or both, allowed the model to match experimental results for INS-1 cells treated with prolactin. The model predicts that faster dimerization and nuclear import rates of STAT5B compared to STAT5A can explain the higher STAT5B nuclear translocation. The model was used to predict the dose response of STAT5B translocation in rat primary beta cells treated with prolactin and reveal possible strategies to modulate STAT5 signaling.

Conclusions

JAK-STAT signaling must be tightly controlled to obtain the biphasic response in STAT5 activation seen experimentally. Receptor up-regulation, combined with SOCS inhibition, receptor internalization, or both is required to match experimental data. Modulating reactions upstream in the signaling can enhance STAT5 activation to increase beta cell survival.

Electronic supplementary material

The online version of this article (10.1007/s12195-020-00647-8) contains supplementary material, which is available to authorized users.

Enhancing network activation in natural killer cells: predictions from in silico modeling

Natural killer (NK) cells are part of the innate immune system and are capable of killing diseased cells. As a result, NK cells are being used for adoptive cell therapies for cancer patients. The activation of NK cell stimulatory receptors leads to a cascade of intracellular phosphorylation reactions, which activates key signaling species that facilitate the secretion of cytolytic molecules required for cell killing. Strategies that maximize the activation of such intracellular species can increase the likelihood of NK cell killing upon contact with a cancer cell and thereby improve efficacy of NK cell-based therapies. However, due to the complexity of intracellular signaling, it is difficult to deduce a priori which strategies can enhance species activation. Therefore, we constructed a mechanistic model of the CD16, 2B4 and NKG2D signaling pathways in NK cells to simulate strategies that enhance signaling. The model predictions were fit to published data and validated with a separate dataset. Model simulations demonstrate strong network activation when the CD16 pathway is stimulated. The magnitude of species activation is most sensitive to the receptor's initial concentration and the rate at which the receptor is activated. Co-stimulation of CD16 and NKG2D in silico required fewer ligands to achieve half-maximal activation than other combinations, suggesting co-stimulating these pathways is most effective in activating the species. We applied the model to predict the effects of perturbing the signaling network and found two strategies that can potently enhance network activation. When the availability of ligands is low, it is more influential to engineer NK cell receptors that are resistant to proteolytic cleavage. In contrast, for high ligand concentrations, inhibiting phosphatase activity leads to sustained species activation. The work presented here establishes a framework for understanding the complex, nonlinear aspects of NK cell signaling and provides detailed strategies for enhancing NK cell activation.

Modelling Growth and Decline in a Two-Species Model System: Pathogenic Escherichia coli O157:H7 and Psychrotrophic Spoilage Bacteria in Milk

Shiga toxin-producing Escherichia coli O157:H7 is a food-borne pathogen and the major cause of hemorrhagic colitis. Pseudomonas is the genus most frequent psychrotrophic spoilage microorganisms present in milk. Two-species bacterial systems with E. coli O157:H7, non-pathogenic E. coli, and P. fluorescens in skimmed milk at 7, 13, 19, or 25 °C were studied. Bacterial interactions were modelled after applying a Bayesian approach. No direct correlation between P. fluorescens's growth rate and its effect on the maximum population densities of E. coli species was found. The results show the complexity of the interactions between two species in a food model. The use of natural microbiota members to control foodborne pathogens could be useful to improve food safety during the processing and storage of refrigerated foods.

Bayesian inference for dynamical systems

Bayesian inference is a common method for conducting parameter estimation for dynamical systems. Despite the prevalent use of Bayesian inference for performing parameter estimation for dynamical systems, there is a need for a formalized and detailed methodology. This paper presents a comprehensive methodology for dynamical system parameter estimation using Bayesian inference and it covers utilizing different distributions, Markov Chain Monte Carlo (MCMC) sampling, obtaining credible intervals for parameters, and prediction intervals for solutions. A logistic growth example is given to illustrate the methodology.

Continuous in vivo Metabolism by NMR

Dense time-series metabolomics data are essential for unraveling the underlying dynamic properties of metabolism. Here we extend high-resolution-magic angle spinning (HR-MAS) to enable continuous in vivo monitoring of metabolism by NMR (CIVM-NMR) and provide analysis tools for these data. First, we reproduced a result in human chronic lymphoid leukemia cells by using isotope-edited CIVM-NMR to rapidly and unambiguously demonstrate unidirectional flux in branched-chain amino acid metabolism. We then collected untargeted CIVM-NMR datasets for Neurospora crassa, a classic multicellular model organism, and uncovered dynamics between central carbon metabolism, amino acid metabolism, energy storage molecules, and lipid and cell wall precursors. Virtually no sample preparation was required to yield a dynamic metabolic fingerprint over hours to days at ~4-min temporal resolution with little noise. CIVM-NMR is simple and readily adapted to different types of cells and microorganisms, offering an experimental complement to kinetic models of metabolism for diverse biological systems.

Dynamic modeling and network approaches for omics time course data: overview of computational approaches and applications

Inferring networks and dynamics of genes, proteins, cells and other biological entities from high-throughput biological omics data is a central and challenging issue in computational and systems biology. This is essential for understanding the complexity of human health, disease susceptibility and pathogenesis for Predictive, Preventive, Personalized and Participatory (P4) system and precision medicine. The delineation of the possible interactions of all genes/proteins in a genome/proteome is a task for which conventional experimental techniques are ill suited. Urgently needed are rapid and inexpensive computational and statistical methods that can identify interacting candidate disease genes or drug targets out of thousands that can be further investigated or validated by experimentations. Moreover, identifying biological dynamic systems, and simultaneously estimating the important kinetic structural and functional parameters, which may not be experimentally accessible could be important directions for drug-disease-gene network studies. In this article, we present an overview and comparison of recent developments of dynamic modeling and network approaches for time-course omics data, and their applications to various biological systems, health conditions and disease statuses. Moreover, various data reduction and analytical schemes ranging from mathematical to computational to statistical methods are compared including their merits, drawbacks and limitations. The most recent software, associated web resources and other potentials for the compared methods are also presented and discussed in detail.

Assessing the role of climate factors on malaria transmission dynamics in South Sudan

Malaria is endemic in South Sudan and it is one of the most severe diseases in the war-torn nation. There has been much concern about whether the severity of its transmission might depend upon climatic conditions that are related to the reproduction of the single-cell parasite attaching to female mosquitoes, especially in high altitude areas. The country experiences two different climatic conditions; namely one tropical and the other hot and semi-arid. In this study, we aim to assess the potential impact of climatic conditions on malaria prevalence in these two climatically distinct regions of South Sudan. We develop and analyze a host-mosquito disease-based model that includes temperature and rainfall. The model has also been parameterized in a Bayesian framework using Bayesian Markov Chain Monte Carlo (MCMC). The mathematical analysis for this study has included equilibria, stability and a sensitivity index on the basic reproduction number R₀. The threshold R₀ is also used to provide a numerical basis for further refinement and prediction of the impact of climate variability on malaria transmission intensity over the study region. The study highlights the impact of various temperature values on the population dynamics of the mosquito.

Modelling the effect of bednet coverage on malaria transmission in South Sudan

A campaign for malaria control, using Long Lasting Insecticide Nets (LLINs) was launched in South Sudan in 2009. The success of such a campaign often depends upon adequate available resources and reliable surveillance data which help officials understand existing infections. An optimal allocation of resources for malaria control at a sub-national scale is therefore paramount to the success of efforts to reduce malaria prevalence. In this paper, we extend an existing SIR mathematical model to capture the effect of LLINs on malaria transmission. Available data on malaria is utilized to determine realistic parameter values of this model using a Bayesian approach via Markov Chain Monte Carlo (MCMC) methods. Then, we explore the parasite prevalence on a continued rollout of LLINs in three different settings in order to create a sub-national projection of malaria. Further, we calculate the model’s basic reproductive number and study its sensitivity to LLINs’ coverage and its efficacy. From the numerical simulation results, we notice a basic reproduction number, R0, confirming a substantial increase of incidence cases if no form of intervention takes place in the community. This work indicates that an effective use of LLINs may reduce R0 and hence malaria transmission. We hope that this study will provide a basis for recommending a scaling-up of the entry point of LLINs’ distribution that targets households in areas at risk of malaria.

Parameter uncertainty quantification using surrogate models applied to a spatial model of yeast mating polarization

A common challenge in systems biology is quantifying the effects of unknown parameters and estimating parameter values from data. For many systems, this task is computationally intractable due to expensive model evaluations and large numbers of parameters. In this work, we investigate a new method for performing sensitivity analysis and parameter estimation of complex biological models using techniques from uncertainty quantification. The primary advance is a significant improvement in computational efficiency from the replacement of model simulation by evaluation of a polynomial surrogate model. We demonstrate the method on two models of mating in budding yeast: a smaller ODE model of the heterotrimeric G-protein cycle, and a larger spatial model of pheromone-induced cell polarization. A small number of model simulations are used to fit the polynomial surrogates, which are then used to calculate global parameter sensitivities. The surrogate models also allow rapid Bayesian inference of the parameters via Markov chain Monte Carlo (MCMC) by eliminating model simulations at each step. Application to the ODE model shows results consistent with published single-point estimates for the model and data, with the added benefit of calculating the correlations between pairs of parameters. On the larger PDE model, the surrogate models allowed convergence for the distribution of 15 parameters, which otherwise would have been computationally prohibitive using simulations at each MCMC step. We inferred parameter distributions that in certain cases peaked at values different from published values, and showed that a wide range of parameters would permit polarization in the model. Strikingly our results suggested different diffusion constants for active versus inactive Cdc42 to achieve good polarization, which is consistent with experimental observations in another yeast species S. pombe.

Bayesian state space models for dynamic genetic network construction across multiple tissues

Construction of gene-gene interaction networks and potential pathways is a challenging and important problem in genomic research for complex diseases while estimating the dynamic changes of the temporal correlations and non-stationarity are the keys in this process. In this paper, we develop dynamic state space models with hierarchical Bayesian settings to tackle this challenge for inferring the dynamic profiles and genetic networks associated with disease treatments. We treat both the stochastic transition matrix and the observation matrix time-variant and include temporal correlation structures in the covariance matrix estimations in the multivariate Bayesian state space models. The unevenly spaced short time courses with unseen time points are treated as hidden state variables. Hierarchical Bayesian approaches with various prior and hyper-prior models with Monte Carlo Markov Chain and Gibbs sampling algorithms are used to estimate the model parameters and the hidden state variables. We apply the proposed Hierarchical Bayesian state space models to multiple tissues (liver, skeletal muscle, and kidney) Affymetrix time course data sets following corticosteroid (CS) drug administration. Both simulation and real data analysis results show that the genomic changes over time and gene-gene interaction in response to CS treatment can be well captured by the proposed models. The proposed dynamic Hierarchical Bayesian state space modeling approaches could be expanded and applied to other large scale genomic data, such as next generation sequence (NGS) combined with real time and time varying electronic health record (EHR) for more comprehensive and robust systematic and network based analysis in order to transform big biomedical data into predictions and diagnostics for precision medicine and personalized healthcare with better decision making and patient outcomes.

Computational dynamic approaches for temporal omics data with applications to systems medicine

Modeling and predicting biological dynamic systems and simultaneously estimating the kinetic structural and functional parameters are extremely important in systems and computational biology. This is key for understanding the complexity of the human health, drug response, disease susceptibility and pathogenesis for systems medicine. Temporal omics data used to measure the dynamic biological systems are essentials to discover complex biological interactions and clinical mechanism and causations. However, the delineation of the possible associations and causalities of genes, proteins, metabolites, cells and other biological entities from high throughput time course omics data is challenging for which conventional experimental techniques are not suited in the big omics era. In this paper, we present various recently developed dynamic trajectory and causal network approaches for temporal omics data, which are extremely useful for those researchers who want to start working in this challenging research area. Moreover, applications to various biological systems, health conditions and disease status, and examples that summarize the state-of-the art performances depending on different specific mining tasks are presented. We critically discuss the merits, drawbacks and limitations of the approaches, and the associated main challenges for the years ahead. The most recent computing tools and software to analyze specific problem type, associated platform resources, and other potentials for the dynamic trajectory and interaction methods are also presented and discussed in detail.

Knowledge gaps to understanding cardiac macrophage polarization following myocardial infarction

Following myocardial infarction (MI), macrophages coordinate both pro-inflammatory and reparative responses of the left ventricle (LV) by reacting to and secreting cytokines, chemokines, and growth factors and by stimulating endothelial cells and fibroblasts to modulate neovascularization and scar formation. Healing of the infarcted LV can be divided into three distinct, but overlapping phases: inflammatory, proliferative, and maturation. Macrophages are involved in all phases. Despite macrophages being a major leukocyte cell type in the post-MI LV, how this cell type regulates LV remodeling over the post-MI time continuum is not completely understood. In this review, we summarize the current literature as a foundation to discuss the major knowledge gaps that remain. Defining the post-MI temporal macrophage phenotypes to establish a classification system is the first step in exploring how macrophage phenotypes are regulated, how temporal stimulation and secretion profiles evolve, and how best to modify stimuli to yield predictable cell responses. This article is part of a Special Issue entitled: The role of post-translational protein modifications on heart and vascular metabolism edited by Jason R.B. Dyck & Jan F.C. Glatz.

A mathematical modeling study of the HIV epidemics at two rural townships in the Liangshan Prefecture of the Sichuan Province of China

Background

As a response to a severe HIV epidemic in the Liangshan Prefecture, one of the worst in China, population based HIV interventions, including two population-wide HIV screening, have been carried out since 2005 at two townships in a remote mountainous region of Liangshan. The objective of our mathematical modeling study is to assess the temporal dynamics of the HIV epidemic in the two townships based on the data collected in the study area during the period 2005–2010.

Methods

A mathematical model was set up to describe the population dynamics of HIV transmission in study area. The model was calibrated by fitting it to the HIV testing and treatment data from 2005 to 2008. Validation of the model was done by comparing its predicted value of HIV prevalence in 2010 to the prevalence data obtained in the 2010 population wide HIV testing. The validated model was used to produce estimation of HIV incidence, prevalence and death.

Results

Our model estimations show that population-based HIV interventions have significantly slowed down the rise of the HIV epidemic in the two townships. Over the five-year period from 2005 to 2010, the year-over-year rate of increase in HIV incidence, prevalence, and death has declined by 91.5%, 28.7%, and 52.3%, respectively.

Conclusion

Mathematical models, when integrated with epidemiological and surveillance data, can be an effective tool for predicting the temporal dynamics of HIV and assessing the impacts of HIV interventions.

Using systems biology approaches to understand cardiac inflammation and extracellular matrix remodeling in the setting of myocardial infarction

Inflammation and extracellular matrix (ECM) remodeling are important components regulating the response of the left ventricle to myocardial infarction (MI). Significant cellular- and molecular-level contributors can be identified by analyzing data acquired through high-throughput genomic and proteomic technologies that provide expression levels for thousands of genes and proteins. Large-scale data provide both temporal and spatial information that need to be analyzed and interpreted using systems biology approaches in order to integrate this information into dynamic models that predict and explain mechanisms of cardiac healing post-MI. In this review, we summarize the systems biology approaches needed to computationally simulate post-MI remodeling, including data acquisition, data analysis for biomarker classification and identification, data integration to build dynamic models, and data interpretation for biological functions. An example for applying a systems biology approach to ECM remodeling is presented as a reference illustration.

From genes to networks: in systematic points of view

We present a report of the BIOCOMP'10 - The 2010 International Conference on Bioinformatics & Computational Biology and other related work in the area of systems biology.