A framework for parameter estimation and model selection from experimental data in systems biology using approximate Bayesian computation

TopoFilter: a MATLAB package for mechanistic model identification in systems biology

Background

To develop mechanistic dynamic models in systems biology, one often needs to identify all (or minimal) representations of the biological processes that are consistent with experimental data, out of a potentially large set of hypothetical mechanisms. However, a simple enumeration of all alternatives becomes quickly intractable when the number of model parameters grows. Selecting appropriate dynamic models out of a large ensemble of models, taking the uncertainty in our biological knowledge and in the experimental data into account, is therefore a key current problem in systems biology.

Results

The TopoFilter package addresses this problem in a heuristic and automated fashion by implementing the previously described topological filtering method for Bayesian model selection. It includes a core heuristic for searching the space of submodels of a parametrized model, coupled with a sampling-based exploration of the parameter space. Recent developments of the method allow to balance exhaustiveness and speed of the model space search, to efficiently re-sample parameters, to parallelize the search, and to use custom scoring functions. We use a theoretical example to motivate these features and then demonstrate TopoFilter’s applicability for a yeast signaling network with more than 250’000 possible model structures.

Conclusions

TopoFilter is a flexible software framework that makes Bayesian model selection and reduction efficient and scalable to network models of a complexity that represents contemporary problems in, for example, cell signaling. TopoFilter is open-source, available under the GPL-3.0 license at https://gitlab.com/csb.ethz/TopoFilter. It includes installation instructions, a quickstart guide, a description of all package options, and multiple examples.

Using a continuum model to decipher the mechanics of embryonic tissue spreading from time-lapse image sequences: An approximate Bayesian computation approach

Advanced imaging techniques generate large datasets capable of describing the structure and kinematics of tissue spreading in embryonic development, wound healing, and the progression of many diseases. These datasets can be integrated with mathematical models to infer biomechanical properties of the system, typically identifying an optimal set of parameters for an individual experiment. However, these methods offer little information on the robustness of the fit and are generally ill-suited for statistical tests of multiple experiments. To overcome this limitation and enable efficient use of large datasets in a rigorous experimental design, we use the approximate Bayesian computation rejection algorithm to construct probability density distributions that estimate model parameters for a defined theoretical model and set of experimental data. Here, we demonstrate this method with a 2D Eulerian continuum mechanical model of spreading embryonic tissue. The model is tightly integrated with quantitative image analysis of different sized embryonic tissue explants spreading on extracellular matrix (ECM) and is regulated by a small set of parameters including forces on the free edge, tissue stiffness, strength of cell-ECM adhesions, and active cell shape changes. We find statistically significant trends in key parameters that vary with initial size of the explant, e.g., for larger explants cell-ECM adhesion forces are weaker and free edge forces are stronger. Furthermore, we demonstrate that estimated parameters for one explant can be used to predict the behavior of other similarly sized explants. These predictive methods can be used to guide further experiments to better understand how collective cell migration is regulated during development.

Comprehensive Review of Models and Methods for Inferences in Bio-Chemical Reaction Networks

The key processes in biological and chemical systems are described by networks of chemical reactions. From molecular biology to biotechnology applications, computational models of reaction networks are used extensively to elucidate their non-linear dynamics. The model dynamics are crucially dependent on the parameter values which are often estimated from observations. Over the past decade, the interest in parameter and state estimation in models of (bio-) chemical reaction networks (BRNs) grew considerably. The related inference problems are also encountered in many other tasks including model calibration, discrimination, identifiability, and checking, and optimum experiment design, sensitivity analysis, and bifurcation analysis. The aim of this review paper is to examine the developments in literature to understand what BRN models are commonly used, and for what inference tasks and inference methods. The initial collection of about 700 documents concerning estimation problems in BRNs excluding books and textbooks in computational biology and chemistry were screened to select over 270 research papers and 20 graduate research theses. The paper selection was facilitated by text mining scripts to automate the search for relevant keywords and terms. The outcomes are presented in tables revealing the levels of interest in different inference tasks and methods for given models in the literature as well as the research trends are uncovered. Our findings indicate that many combinations of models, tasks and methods are still relatively unexplored, and there are many new research opportunities to explore combinations that have not been considered-perhaps for good reasons. The most common models of BRNs in literature involve differential equations, Markov processes, mass action kinetics, and state space representations whereas the most common tasks are the parameter inference and model identification. The most common methods in literature are Bayesian analysis, Monte Carlo sampling strategies, and model fitting to data using evolutionary algorithms. The new research problems which cannot be directly deduced from the text mining data are also discussed.

Systems biology: perspectives on multiscale modeling in research on endocrine-related cancers

Drawing on concepts from experimental biology, computer science, informatics, mathematics and statistics, systems biologists integrate data across diverse platforms and scales of time and space to create computational and mathematical models of the integrative, holistic functions of living systems. Endocrine-related cancers are well suited to study from a systems perspective because of the signaling complexities arising from the roles of growth factors, hormones and their receptors as critical regulators of cancer cell biology and from the interactions among cancer cells, normal cells and signaling molecules in the tumor microenvironment. Moreover, growth factors, hormones and their receptors are often effective targets for therapeutic intervention, such as estrogen biosynthesis, estrogen receptors or HER2 in breast cancer and androgen receptors in prostate cancer. Given the complexity underlying the molecular control networks in these cancers, a simple, intuitive understanding of how endocrine-related cancers respond to therapeutic protocols has proved incomplete and unsatisfactory. Systems biology offers an alternative paradigm for understanding these cancers and their treatment. To correctly interpret the results of systems-based studies requires some knowledge of how in silico models are built, and how they are used to describe a system and to predict the effects of perturbations on system function. In this review, we provide a general perspective on the field of cancer systems biology, and we explore some of the advantages, limitations and pitfalls associated with using predictive multiscale modeling to study endocrine-related cancers.

Adapting machine-learning algorithms to design gene circuits

BACKGROUND

Gene circuits are important in many aspects of biology, and perform a wide variety of different functions. For example, some circuits oscillate (e.g. the cell cycle), some are bistable (e.g. as cells differentiate), some respond sharply to environmental signals (e.g. ultrasensitivity), and some pattern multicellular tissues (e.g. Turing's model). Often, one starts from a given circuit, and using simulations, asks what functions it can perform. Here we want to do the opposite: starting from a prescribed function, can we find a circuit that executes this function? Whilst simple in principle, this task is challenging from a computational perspective, since gene circuit models are complex systems with many parameters. In this work, we adapted machine-learning algorithms to significantly accelerate gene circuit discovery.

RESULTS

We use gradient-descent optimization algorithms from machine learning to rapidly screen and design gene circuits. With this approach, we found that we could rapidly design circuits capable of executing a range of different functions, including those that: (1) recapitulate important in vivo phenomena, such as oscillators, and (2) perform complex tasks for synthetic biology, such as counting noisy biological events.

CONCLUSIONS

Our computational pipeline will facilitate the systematic study of natural circuits in a range of contexts, and allow the automatic design of circuits for synthetic biology. Our method can be readily applied to biological networks of any type and size, and is provided as an open-source and easy-to-use python module, GeneNet.

Defining informative priors for ensemble modeling in systems biology

Ensemble modeling in molecular systems biology requires the reproducible translation of kinetic parameter data into informative probability distributions (priors), as well as approaches that sample parameters from these distributions without violating the thermodynamic consistency of the overall model. Although a number of pioneering frameworks for ensemble modeling have been published, the issue of generating informative priors has not yet been addressed. Here, we present a protocol that aims to fill this gap. This protocol discusses the collection of parameter values from a diverse range of sources (literature, databases and experiments), assessment of their plausibility, and creation of log-normal probability distributions that can be used as informative priors in ensemble modeling. Furthermore, the protocol enables sampling from the generated distributions while maintaining thermodynamic consistency. Once all parameter values have been retrieved from literature and databases, the protocol can be implemented within ~5-10 min per parameter. The aim of this protocol is to facilitate the design and use of informative distributions for ensemble modeling, especially in fields such as synthetic biology and systems medicine.

Ranking network mechanisms by how they fit diverse experiments and deciding on E. coli's ammonium transport and assimilation network

The complex ammonium transport and assimilation network of E. coli involves the ammonium transporter AmtB, the regulatory proteins GlnK and GlnB, and the central N-assimilating enzymes together with their highly complex interactions. The engineering and modelling of such a complex network seem impossible because functioning depends critically on a gamut of data known at patchy accuracy. We developed a way out of this predicament, which employs: (i) a constrained optimization-based technology for the simultaneous fitting of models to heterogeneous experimental data sets gathered through diverse experimental set-ups, (ii) a 'rubber band method' to deal with different degrees of uncertainty, both in experimentally determined or estimated parameter values and in measured transient or steady-state variables (training data sets), (iii) integration of human expertise to decide on accuracies of both parameters and variables, (iv) massive computation employing a fast algorithm and a supercomputer, (v) an objective way of quantifying the plausibility of models, which makes it possible to decide which model is the best and how much better that model is than the others. We applied the new technology to the ammonium transport and assimilation network, integrating recent and older data of various accuracies, from different expert laboratories. The kinetic model objectively ranked best, has E. coli's AmtB as an active transporter of ammonia to be assimilated with GlnK minimizing the futile cycling that is an inevitable consequence of intracellular ammonium accumulation. It is 130 times better than a model with facilitated passive transport of ammonia.

Proteolytic dynamics of human 20S thymoproteasome

An efficient immunosurveillance of CD8⁺ T cells in the periphery depends on positive/negative selection of thymocytes and thus on the dynamics of antigen degradation and epitope production by thymoproteasome and immunoproteasome in the thymus. Although studies in mouse systems have shown how thymoproteasome activity differs from that of immunoproteasome and strongly impacts the T cell repertoire, the proteolytic dynamics and the regulation of human thymoproteasome are unknown. By combining biochemical and computational modeling approaches, we show here that human 20S thymoproteasome and immunoproteasome differ not only in the proteolytic activity of the catalytic sites but also in the peptide transport. These differences impinge upon the quantity of peptide products rather than where the substrates are cleaved. The comparison of the two human 20S proteasome isoforms depicts different processing of antigens that are associated to tumors and autoimmune diseases.

Optimization of reflectometry experiments using information theory

A framework based on Bayesian statistics and information theory is developed to optimize the design of surface-sensitive reflectometry experiments. The method applies to model-based reflectivity data analysis, uses simulated reflectivity data and is capable of optimizing experiments that probe a sample under more than one condition. After presentation of the underlying theory and its implementation, the framework is applied to exemplary test problems for which the information gain ΔH is determined. Reflectivity data are simulated for the current generation of neutron reflectometers at the NIST Center for Neutron Research. However, the simulation can be easily modified for X-ray or neutron instruments at any source. With application to structural biology in mind, this work explores the dependence of ΔH on the scattering length density of aqueous solutions in which the sample structure is bathed, on the counting time and on the maximum momentum transfer of the measurement. Finally, the impact of a buried magnetic reference layer on ΔH is investigated.

Inferring Gene Regulatory Networks from Multiple Datasets

Gaussian process dynamical systems (GPDS) represent Bayesian nonparametric approaches to inference of nonlinear dynamical systems, and provide a principled framework for the learning of biological networks from multiple perturbed time series measurements of gene or protein expression. Such approaches are able to capture the full richness of complex ODE models, and can be scaled for inference in moderately large systems containing hundreds of genes. Related hierarchical approaches allow for inference from multiple datasets in which the underlying generative networks are assumed to have been rewired, either by context-dependent changes in network structure, evolutionary processes, or synthetic manipulation. These approaches can also be used to leverage experimentally determined network structures from one species into another where the network structure is unknown. Collectively, these methods provide a comprehensive and flexible platform for inference from a diverse range of data, with applications in systems and synthetic biology, as well as spatiotemporal modelling of embryo development. In this chapter we provide an overview of GPDS approaches and highlight their applications in the biological sciences, with accompanying tutorials available as a Jupyter notebook from https://github.com/cap76/GPDS .

PEITH(Θ): perfecting experiments with information theory in Python with GPU support

Motivation

Different experiments provide differing levels of information about a biological system. This makes it difficult, a priori, to select one of them beyond mere speculation and/or belief, especially when resources are limited. With the increasing diversity of experimental approaches and general advances in quantitative systems biology, methods that inform us about the information content that a given experiment carries about the question we want to answer, become crucial.

Results

PEITH(Θ) is a general purpose, Python framework for experimental design in systems biology. PEITH(Θ) uses Bayesian inference and information theory in order to derive which experiments are most informative in order to estimate all model parameters and/or perform model predictions.

Availability and implementation

https://github.com/MichaelPHStumpf/Peitho

Simultaneous Model Selection and Model Calibration for the Proliferation of Tumor and Normal Cells During In Vitro Chemotherapy Experiments

In vitro experiments were conducted in this work to analyze the proliferation of tumor (DU-145) and normal (macrophage RAW 264.7) cells under the influence of a chemotherapeutic drug (doxorubicin). Approximate Bayesian Computation (ABC) was used to select among four competing models to represent the number of cells and to estimate the model parameters, based on the experimental data. For one case, the selected model was validated in a replicated experiment, through the solution of a state estimation problem with a particle filter algorithm, thus demonstrating the robustness of the ABC procedure used in this work.

Towards an Aspect-Oriented Design and Modelling Framework for Synthetic Biology

Work on synthetic biology has largely used a component-based metaphor for system construction. While this paradigm has been successful for the construction of numerous systems, the incorporation of contextual design issues-either compositional, host or environmental-will be key to realising more complex applications. Here, we present a design framework that radically steps away from a purely parts-based paradigm by using aspect-oriented software engineering concepts. We believe that the notion of concerns is a powerful and biologically credible way of thinking about system synthesis. By adopting this approach, we can separate core concerns, which represent modular aims of the design, from cross-cutting concerns, which represent system-wide attributes. The explicit handling of cross-cutting concerns allows for contextual information to enter the design process in a modular way. As a proof-of-principle, we implemented the aspect-oriented approach in the Python tool, SynBioWeaver, which enables the combination, or weaving, of core and cross-cutting concerns. The power and flexibility of this framework is demonstrated through a number of examples covering the inclusion of part context, combining circuit designs in a context dependent manner, and the generation of rule, logic and reaction models from synthetic circuit designs.

An Introduction to the Mathematical Modeling in the Study of Cancer Systems Biology

BACKGROUND

Frequently occurring in cancer are the aberrant alterations of regulatory onco-metabolites, various oncogenes/epigenetic stochasticity, and suppressor genes, as well as the deficient mismatch repair mechanism, chronic inflammation, or those deviations belonging to the other cancer characteristics. How these aberrations that evolve overtime determine the global phenotype of malignant tumors remains to be completely understood. Dynamic analysis may have potential to reveal the mechanism of carcinogenesis and can offer new therapeutic intervention.

AIMS

We introduce simplified mathematical tools to model serial quantitative data of cancer biomarkers. We also highlight an introductory overview of mathematical tools and models as they apply from the viewpoint of known cancer features.

METHODS

Mathematical modeling of potentially actionable genomic products and how they proceed overtime during tumorigenesis are explored. This report is intended to be instinctive without being overly technical.

RESULTS

To date, many mathematical models of the common features of cancer have been developed. However, the dynamic of integrated heterogeneous processes and their cross talks related to carcinogenesis remains to be resolved.

CONCLUSIONS

In cancer research, outlining mathematical modeling of experimentally obtained data snapshots of molecular species may provide insights into a better understanding of the multiple biochemical circuits. Recent discoveries have provided support for the existence of complex cancer progression in dynamics that span from a simple 1-dimensional deterministic system to a stochastic (ie, probabilistic) or to an oscillatory and multistable networks. Further research in mathematical modeling of cancer progression, based on the evolving molecular kinetics (time series), could inform a specific and a predictive behavior about the global systems biology of vulnerable tumor cells in their earlier stages of oncogenesis. On this footing, new preventive measures and anticancer therapy could then be constructed.

The impact of experimental design choices on parameter inference for models of growing cell colonies

To better understand development, repair and disease progression, it is useful to quantify the behaviour of proliferative and motile cell populations as they grow and expand to fill their local environment. Inferring parameters associated with mechanistic models of cell colony growth using quantitative data collected from carefully designed experiments provides a natural means to elucidate the relative contributions of various processes to the growth of the colony. In this work, we explore how experimental design impacts our ability to infer parameters for simple models of the growth of proliferative and motile cell populations. We adopt a Bayesian approach, which allows us to characterize the uncertainty associated with estimates of the model parameters. Our results suggest that experimental designs that incorporate initial spatial heterogeneities in cell positions facilitate parameter inference without the requirement of cell tracking, while designs that involve uniform initial placement of cells require cell tracking for accurate parameter inference. As cell tracking is an experimental bottleneck in many studies of this type, our recommendations for experimental design provide for significant potential time and cost savings in the analysis of cell colony growth.

Evaluation of Parallel Tempering to Accelerate Bayesian Parameter Estimation in Systems Biology

Models of biological systems often have many unknown parameters that must be determined in order for model behavior to match experimental observations. Commonly-used methods for parameter estimation that return point estimates of the best-fit parameters are insufficient when models are high dimensional and under-constrained. As a result, Bayesian methods, which treat model parameters as random variables and attempt to estimate their probability distributions given data, have become popular in systems biology. Bayesian parameter estimation often relies on Markov Chain Monte Carlo (MCMC) methods to sample model parameter distributions, but the slow convergence of MCMC sampling can be a major bottleneck. One approach to improving performance is parallel tempering (PT), a physics-based method that uses swapping between multiple Markov chains run in parallel at different temperatures to accelerate sampling. The temperature of a Markov chain determines the probability of accepting an unfavorable move, so swapping with higher temperatures chains enables the sampling chain to escape from local minima. In this work we compared the MCMC performance of PT and the commonly-used Metropolis-Hastings (MH) algorithm on six biological models of varying complexity. We found that for simpler models PT accelerated convergence and sampling, and that for more complex models, PT often converged in cases MH became trapped in non-optimal local minima. We also developed a freely-available MATLAB package for Bayesian parameter estimation called PTEMPEST (http://github.com/RuleWorld/ptempest), which is closely integrated with the popular BioNetGen software for rule-based modeling of biological systems.

Bayesian inference of agent-based models: a tool for studying kidney branching morphogenesis

The adult mammalian kidney has a complex, highly-branched collecting duct epithelium that arises as a ureteric bud sidebranch from an epithelial tube known as the nephric duct. Subsequent branching of the ureteric bud to form the collecting duct tree is regulated by subcellular interactions between the epithelium and a population of mesenchymal cells that surround the tips of outgrowing branches. The mesenchymal cells produce glial cell-line derived neurotrophic factor (GDNF), that binds with RET receptors on the surface of the epithelial cells to stimulate several subcellular pathways in the epithelium. Such interactions are known to be a prerequisite for normal branching development, although competing theories exist for their role in morphogenesis. Here we introduce the first agent-based model of ex vivo kidney uretic branching. Through comparison with experimental data, we show that growth factor-regulated growth mechanisms can explain early epithelial cell branching, but only if epithelial cell division depends in a switch-like way on the local growth factor concentration; cell division occurring only if the driving growth factor level exceeds a threshold. We also show how a recently-developed method, "Approximate Approximate Bayesian Computation", can be used to infer key model parameters, and reveal the dependency between the parameters controlling a growth factor-dependent growth switch. These results are consistent with a requirement for signals controlling proliferation and chemotaxis, both of which are previously identified roles for GDNF.

Parameter uncertainty quantification using surrogate models applied to a spatial model of yeast mating polarization

A common challenge in systems biology is quantifying the effects of unknown parameters and estimating parameter values from data. For many systems, this task is computationally intractable due to expensive model evaluations and large numbers of parameters. In this work, we investigate a new method for performing sensitivity analysis and parameter estimation of complex biological models using techniques from uncertainty quantification. The primary advance is a significant improvement in computational efficiency from the replacement of model simulation by evaluation of a polynomial surrogate model. We demonstrate the method on two models of mating in budding yeast: a smaller ODE model of the heterotrimeric G-protein cycle, and a larger spatial model of pheromone-induced cell polarization. A small number of model simulations are used to fit the polynomial surrogates, which are then used to calculate global parameter sensitivities. The surrogate models also allow rapid Bayesian inference of the parameters via Markov chain Monte Carlo (MCMC) by eliminating model simulations at each step. Application to the ODE model shows results consistent with published single-point estimates for the model and data, with the added benefit of calculating the correlations between pairs of parameters. On the larger PDE model, the surrogate models allowed convergence for the distribution of 15 parameters, which otherwise would have been computationally prohibitive using simulations at each MCMC step. We inferred parameter distributions that in certain cases peaked at values different from published values, and showed that a wide range of parameters would permit polarization in the model. Strikingly our results suggested different diffusion constants for active versus inactive Cdc42 to achieve good polarization, which is consistent with experimental observations in another yeast species S. pombe.

Improving model prediction of soil N2O emissions through Bayesian calibration

The biogeochemical processes that lead to the production of N₂O in arable soils are controlled by temporally and spatially varying drivers. The need for prediction of soil N₂O emissions across scales means that agroecosystem biogeochemistry models are widely used to simulate N₂O emissions. Due to the parameter-dense nature of agroecosystem models their parameters have to be calibrated according to the soil and climatic conditions of the intended area of application. Bayesian calibration is considered one of the most advanced ways to complete this task. In this study, we calibrate nine parameters of the Landscape-DNDC process-based agroecosystem model, which are key to its N₂O prediction. The Metropolis-Hastings algorithm is used at four separate implementations in order to estimate parameter posterior distributions at four arable sites in the UK. The results of this process are visualised, summarised and assessed against measured N₂O data from ten independent arable sites. The study shows that, in many cases, soil N₂O emission peaks that were not predicted with the default model parameters were predicted after calibration. Overall, the prediction of soil N₂O fluxes across all the sites that were considered was improved by 33% when using the calibrated parameters.

Combining a Toggle Switch and a Repressilator within the AC-DC Circuit Generates Distinct Dynamical Behaviors

Although the structure of a genetically encoded regulatory circuit is an important determinant of its function, the relationship between circuit topology and the dynamical behaviors it can exhibit is not well understood. Here, we explore the range of behaviors available to the AC-DC circuit. This circuit consists of three genes connected as a combination of a toggle switch and a repressilator. Using dynamical systems theory, we show that the AC-DC circuit exhibits both oscillations and bistability within the same region of parameter space; this generates emergent behaviors not available to either the toggle switch or the repressilator alone. The AC-DC circuit can switch on oscillations via two distinct mechanisms, one of which induces coherence into ensembles of oscillators. In addition, we show that in the presence of noise, the AC-DC circuit can behave as an excitable system capable of spatial signal propagation or coherence resonance. Together, these results demonstrate how combinations of simple motifs can exhibit multiple complex behaviors.

Computational Methods for Estimating Molecular System from Membrane Potential Recordings in Nerve Growth Cone

Biological cells express intracellular biomolecular information to the extracellular environment as various physical responses. We show a novel computational approach to estimate intracellular biomolecular pathways from growth cone electrophysiological responses. Previously, it was shown that cGMP signaling regulates membrane potential (MP) shifts that control the growth cone turning direction during neuronal development. We present here an integrated deterministic mathematical model and Bayesian reversed-engineering framework that enables estimation of the molecular signaling pathway from electrical recordings and considers both the system uncertainty and cell-to-cell variability. Our computational method selects the most plausible molecular pathway from multiple candidates while satisfying model simplicity and considering all possible parameter ranges. The model quantitatively reproduces MP shifts depending on cGMP levels and MP variability potential in different experimental conditions. Lastly, our model predicts that chloride channel inhibition by cGMP-dependent protein kinase (PKG) is essential in the core system for regulation of the MP shifts.

Modeling thrombosis in silico: Frontiers, challenges, unresolved problems and milestones

Hemostasis is a complex physiological mechanism that functions to maintain vascular integrity under any conditions. Its primary components are blood platelets and a coagulation network that interact to form the hemostatic plug, a combination of cell aggregate and gelatinous fibrin clot that stops bleeding upon vascular injury. Disorders of hemostasis result in bleeding or thrombosis, and are the major immediate cause of mortality and morbidity in the world. Regulation of hemostasis and thrombosis is immensely complex, as it depends on blood cell adhesion and mechanics, hydrodynamics and mass transport of various species, huge signal transduction networks in platelets, as well as spatiotemporal regulation of the blood coagulation network. Mathematical and computational modeling has been increasingly used to gain insight into this complexity over the last 30 years, but the limitations of the existing models remain profound. Here we review state-of-the-art-methods for computational modeling of thrombosis with the specific focus on the analysis of unresolved challenges. They include: a) fundamental issues related to physics of platelet aggregates and fibrin gels; b) computational challenges and limitations for solution of the models that combine cell adhesion, hydrodynamics and chemistry; c) biological mysteries and unknown parameters of processes; d) biophysical complexities of the spatiotemporal networks' regulation. Both relatively classical approaches and innovative computational techniques for their solution are considered; the subjects discussed with relation to thrombosis modeling include coarse-graining, continuum versus particle-based modeling, multiscale models, hybrid models, parameter estimation and others. Fundamental understanding gained from theoretical models are highlighted and a description of future prospects in the field and the nearest possible aims are given.

Parameter Estimation in Mathematical Models of Viral Infections Using R

In recent years, mathematical modeling approaches have played a central role in understanding and quantifying mechanisms in different viral infectious diseases. In this approach, biology-based hypotheses are expressed via mathematical relations and then tested based on empirical data. The simulation results can be used to either identify underlying mechanisms and provide predictions of infection outcomes or to evaluate the efficacy of a treatment.Conducting parameter estimation for mathematical models is not an easy task. Here we detail an approach to conduct parameter estimation and to evaluate the results using the free software R. The method is applicable to influenza virus dynamics at different complexity levels, widening experimentalists' capabilities in understanding their data. The parameter estimation approach presented here can be also applied to other viral infections or biological applications.

Impact of individual behaviour change on the spread of emerging infectious diseases

Human behaviour plays an important role in the spread of emerging infectious diseases, and understanding the influence of behaviour changes on epidemics can be key to improving control efforts. However, how the dynamics of individual behaviour changes affects the development of emerging infectious disease is a key public health issue. To develop different formula for individual behaviour change and introduce how to embed it into a dynamic model of infectious diseases, we choose A/H1N1 and Ebola as typical examples, combined with the epidemic reported cases and media related news reports. Thus, the logistic model with the health belief model is used to determine behaviour decisions through the health belief model constructs. Furthermore, we propose 4 candidate infectious disease models without and with individual behaviour change and use approximate Bayesian computation based on sequential Monte Carlo method for model selection. The main results indicate that the classical compartment model without behaviour change and the model with average rate of behaviour change depicted by an exponential function could fit the observed data best. The results provide a new way on how to choose an infectious disease model to predict the disease prevalence trend or to evaluate the influence of intervention measures on disease control. However, sensitivity analyses indicate that the accumulated number of hospital notifications and deaths could be largely reduced as the rate of behaviour change increases. Therefore, in terms of mitigating emerging infectious diseases, both media publicity focused on how to guide people's behaviour change and positive responses of individuals are critical.

An efficient moments-based inference method for within-host bacterial infection dynamics

Over the last ten years, isogenic tagging (IT) has revolutionised the study of bacterial infection dynamics in laboratory animal models. However, quantitative analysis of IT data has been hindered by the piecemeal development of relevant statistical models. The most promising approach relies on stochastic Markovian models of bacterial population dynamics within and among organs. Here we present an efficient numerical method to fit such stochastic dynamic models to in vivo experimental IT data. A common approach to statistical inference with stochastic dynamic models relies on producing large numbers of simulations, but this remains a slow and inefficient method for all but simple problems, especially when tracking bacteria in multiple locations simultaneously. Instead, we derive and solve the systems of ordinary differential equations for the two lower-order moments of the stochastic variables (mean, variance and covariance). For any given model structure, and assuming linear dynamic rates, we demonstrate how the model parameters can be efficiently and accurately estimated by divergence minimisation. We then apply our method to an experimental dataset and compare the estimates and goodness-of-fit to those obtained by maximum likelihood estimation. While both sets of parameter estimates had overlapping confidence regions, the new method produced lower values for the division and death rates of bacteria: these improved the goodness-of-fit at the second time point at the expense of that of the first time point. This flexible framework can easily be applied to a range of experimental systems. Its computational efficiency paves the way for model comparison and optimal experimental design.

Numerical algebraic geometry for model selection and its application to the life sciences

Researchers working with mathematical models are often confronted by the related problems of parameter estimation, model validation and model selection. These are all optimization problems, well known to be challenging due to nonlinearity, non-convexity and multiple local optima. Furthermore, the challenges are compounded when only partial data are available. Here, we consider polynomial models (e.g. mass-action chemical reaction networks at steady state) and describe a framework for their analysis based on optimization using numerical algebraic geometry. Specifically, we use probability-one polynomial homotopy continuation methods to compute all critical points of the objective function, then filter to recover the global optima. Our approach exploits the geometrical structures relating models and data, and we demonstrate its utility on examples from cell signalling, synthetic biology and epidemiology.

Modeling the architecture of the regulatory system controlling methylenomycin production in Streptomyces coelicolor

Background

The antibiotic methylenomycin A is produced naturally by Streptomyces coelicolor A3(2), a model organism for streptomycetes. This compound is of particular interest to synthetic biologists because all of the associated biosynthetic, regulatory and resistance genes are located on a single cluster on the SCP1 plasmid, making the entire module easily transferable between different bacterial strains. Understanding further the regulation and biosynthesis of the methylenomycin producing gene cluster could assist in the identification of motifs that can be exploited in synthetic regulatory systems for the rational engineering of novel natural products and antibiotics.

Results

We identify and validate a plausible architecture for the regulatory system controlling methylenomycin production in S. coelicolor using mathematical modeling approaches. Model selection via an approximate Bayesian computation (ABC) approach identifies three candidate model architectures that are most likely to produce the available experimental data, from a set of 48 possible candidates. Subsequent global optimization of the parameters of these model architectures identifies a single model that most accurately reproduces the dynamical response of the system, as captured by time series data on methylenomycin production. Further analyses of variants of this model architecture that capture the effects of gene knockouts also reproduce qualitative experimental results observed in mutant S. coelicolor strains.

Conclusions

The mechanistic mathematical model developed in this study recapitulates current biological knowledge of the regulation and biosynthesis of the methylenomycin producing gene cluster, and can be used in future studies to make testable predictions and formulate experiments to further improve our understanding of this complex regulatory system.

Multi-level and hybrid modelling approaches for systems biology

During the last decades, high-throughput techniques allowed for the extraction of a huge amount of data from biological systems, unveiling more of their underling complexity. Biological systems encompass a wide range of space and time scales, functioning according to flexible hierarchies of mechanisms making an intertwined and dynamic interplay of regulations. This becomes particularly evident in processes such as ontogenesis, where regulative assets change according to process context and timing, making structural phenotype and architectural complexities emerge from a single cell, through local interactions. The information collected from biological systems are naturally organized according to the functional levels composing the system itself. In systems biology, biological information often comes from overlapping but different scientific domains, each one having its own way of representing phenomena under study. That is, the different parts of the system to be modelled may be described with different formalisms. For a model to have improved accuracy and capability for making a good knowledge base, it is good to comprise different system levels, suitably handling the relative formalisms. Models which are both multi-level and hybrid satisfy both these requirements, making a very useful tool in computational systems biology. This paper reviews some of the main contributions in this field.

How to deal with parameters for whole-cell modelling

Dynamical systems describing whole cells are on the verge of becoming a reality. But as models of reality, they are only useful if we have realistic parameters for the molecular reaction rates and cell physiological processes. There is currently no suitable framework to reliably estimate hundreds, let alone thousands, of reaction rate parameters. Here, we map out the relative weaknesses and promises of different approaches aimed at redressing this issue. While suitable procedures for estimation or inference of the whole (vast) set of parameters will, in all likelihood, remain elusive, some hope can be drawn from the fact that much of the cellular behaviour may be explained in terms of smaller sets of parameters. Identifying such parameter sets and assessing their behaviour is now becoming possible even for very large systems of equations, and we expect such methods to become central tools in the development and analysis of whole-cell models.

Inferring Mechanistic Parameters from Amyloid Formation Kinetics by Approximate Bayesian Computation

Amyloid formation is implicated in a number of human diseases, and is thought to proceed via a nucleation-dependent polymerization mechanism. Experimenters often wish to relate changes in amyloid formation kinetics, for example, in response to small molecules to specific mechanistic steps along this pathway. However, fitting kinetic fibril formation data to a complex model including explicit rate constants results in an ill-posed problem with a vast number of potential solutions. The levels of uncertainty remaining in parameters calculated from these models, arising both from experimental noise and high levels of degeneracy or codependency in parameters, is often unclear. Here, we demonstrate that a combination of explicit mathematical models with an approximate Bayesian computation approach can be used to assign the mechanistic effects of modulators on amyloid fibril formation. We show that even when exact rate constants cannot be extracted, parameters derived from these rate constants can be recovered and used to assign mechanistic effects and their relative magnitudes with a great deal of confidence. Furthermore, approximate Bayesian computation provides a robust method for visualizing uncertainty remaining in the model parameters, regardless of its origin. We apply these methods to the problem of heparin-mediated tau polymerization, which displays complex kinetic behavior not amenable to analysis by more traditional methods. Our analysis indicates that the role of heparin cannot be explained by enhancement of nucleation alone, as has been previously proposed. The methods described here are applicable to a wide range of systems, as models can be easily adapted to account for new reactions and reversibility.

Effects of the Ordering of Natural Selection and Population Regulation Mechanisms on Wright-Fisher Models

We explore the effect of different mechanisms of natural selection on the evolution of populations for one- and two-locus systems. We compare the effect of viability and fecundity selection in the context of the Wright-Fisher model with selection under the assumption of multiplicative fitness. We show that these two modes of natural selection correspond to different orderings of the processes of population regulation and natural selection in the Wright-Fisher model. We find that under the Wright-Fisher model these two different orderings can affect the distribution of trajectories of haplotype frequencies evolving with genetic recombination. However, the difference in the distribution of trajectories is only appreciable when the population is in significant linkage disequilibrium. We find that as linkage disequilibrium decays the trajectories for the two different models rapidly become indistinguishable. We discuss the significance of these findings in terms of biological examples of viability and fecundity selection, and speculate that the effect may be significant when factors such as gene migration maintain a degree of linkage disequilibrium.

Universal signal generator for dynamic cell stimulation

Dynamic cell stimulation is a powerful technique for probing gene networks and for applications in stem cell differentiation, immunomodulation and signaling. We developed a robust and flexible method and associated microfluidic devices to generate a wide-range of precisely formulated dynamic chemical signals to stimulate live cells and measure their dynamic response. This signal generator is capable of digital to analog conversion (DAC) through combinatoric selection of discrete input concentrations, and outperforms existing methods by both achievable resolution, dynamic range and simplicity in design. It requires no calibration, has minimal space requirements and can be easily integrated into microfluidic cell culture devices. The signal generator hardware and software we developed allows to choose the waveform, period and amplitude of chemical input signals and features addition of well-defined chemical noise to study the role of stochasticity in cellular information processing.

A large-scale stochastic spatiotemporal model for Aedes albopictus-borne chikungunya epidemiology

Chikungunya is a viral disease transmitted to humans primarily via the bites of infected Aedes mosquitoes. The virus caused a major epidemic in the Indian Ocean in 2004, affecting millions of inhabitants, while cases have also been observed in Europe since 2007. We developed a stochastic spatiotemporal model of Aedes albopictus-borne chikungunya transmission based on our recently developed environmentally-driven vector population dynamics model. We designed an integrated modelling framework incorporating large-scale gridded climate datasets to investigate disease outbreaks on Reunion Island and in Italy. We performed Bayesian parameter inference on the surveillance data, and investigated the validity and applicability of the underlying biological assumptions. The model successfully represents the outbreak and measures of containment in Italy, suggesting wider applicability in Europe. In its current configuration, the model implies two different viral strains, thus two different outbreaks, for the two-stage Reunion Island epidemic. Characterisation of the posterior distributions indicates a possible relationship between the second larger outbreak on Reunion Island and the Italian outbreak. The model suggests that vector control measures, with different modes of operation, are most effective when applied in combination: adult vector intervention has a high impact but is short-lived, larval intervention has a low impact but is long-lasting, and quarantining infected territories, if applied strictly, is effective in preventing large epidemics. We present a novel approach in analysing chikungunya outbreaks globally using a single environmentally-driven mathematical model. Our study represents a significant step towards developing a globally applicable Ae. albopictus-borne chikungunya transmission model, and introduces a guideline for extending such models to other vector-borne diseases.

Distinct cellular states determine calcium signaling response

The heterogeneity in mammalian cells signaling response is largely a result of pre‐existing cell‐to‐cell variability. It is unknown whether cell‐to‐cell variability rises from biochemical stochastic fluctuations or distinct cellular states. Here, we utilize calcium response to adenosine trisphosphate as a model for investigating the structure of heterogeneity within a population of cells and analyze whether distinct cellular response states coexist. We use a functional definition of cellular state that is based on a mechanistic dynamical systems model of calcium signaling. Using Bayesian parameter inference, we obtain high confidence parameter value distributions for several hundred cells, each fitted individually. Clustering the inferred parameter distributions revealed three major distinct cellular states within the population. The existence of distinct cellular states raises the possibility that the observed variability in response is a result of structured heterogeneity between cells. The inferred parameter distribution predicts, and experiments confirm that variability in IP3R response explains the majority of calcium heterogeneity. Our work shows how mechanistic models and single‐cell parameter fitting can uncover hidden population structure and demonstrate the need for parameter inference at the single‐cell level.

Mechanistic Modelling and Bayesian Inference Elucidates the Variable Dynamics of Double-Strand Break Repair

DNA double-strand breaks are lesions that form during metabolism, DNA replication and exposure to mutagens. When a double-strand break occurs one of a number of repair mechanisms is recruited, all of which have differing propensities for mutational events. Despite DNA repair being of crucial importance, the relative contribution of these mechanisms and their regulatory interactions remain to be fully elucidated. Understanding these mutational processes will have a profound impact on our knowledge of genomic instability, with implications across health, disease and evolution. Here we present a new method to model the combined activation of non-homologous end joining, single strand annealing and alternative end joining, following exposure to ionising radiation. We use Bayesian statistics to integrate eight biological data sets of double-strand break repair curves under varying genetic knockouts and confirm that our model is predictive by re-simulating and comparing to additional data. Analysis of the model suggests that there are at least three disjoint modes of repair, which we assign as fast, slow and intermediate. Our results show that when multiple data sets are combined, the rate for intermediate repair is variable amongst genetic knockouts. Further analysis suggests that the ratio between slow and intermediate repair depends on the presence or absence of DNA-PKcs and Ku70, which implies that non-homologous end joining and alternative end joining are not independent. Finally, we consider the proportion of double-strand breaks within each mechanism as a time series and predict activity as a function of repair rate. We outline how our insights can be directly tested using imaging and sequencing techniques and conclude that there is evidence of variable dynamics in alternative repair pathways. Our approach is an important step towards providing a unifying theoretical framework for the dynamics of DNA repair processes.

Quantifying differences in cell line population dynamics using CellPD

BACKGROUND

The increased availability of high-throughput datasets has revealed a need for reproducible and accessible analyses which can quantitatively relate molecular changes to phenotypic behavior. Existing tools for quantitative analysis generally require expert knowledge.

RESULTS

CellPD (cell phenotype digitizer) facilitates quantitative phenotype analysis, allowing users to fit mathematical models of cell population dynamics without specialized training. CellPD requires one input (a spreadsheet) and generates multiple outputs including parameter estimation reports, high-quality plots, and minable XML files. We validated CellPD's estimates by comparing it with a previously published tool (cellGrowth) and with Microsoft Excel's built-in functions. CellPD correctly estimates the net growth rate of cell cultures and is more robust to data sparsity than cellGrowth. When we tested CellPD's usability, biologists (without training in computational modeling) ran CellPD correctly on sample data within 30 min. To demonstrate CellPD's ability to aid in the analysis of high throughput data, we created a synthetic high content screening (HCS) data set, where a simulated cell line is exposed to two hypothetical drug compounds at several doses. CellPD correctly estimates the drug-dependent birth, death, and net growth rates. Furthermore, CellPD's estimates quantify and distinguish between the cytostatic and cytotoxic effects of both drugs-analyses that cannot readily be performed with spreadsheet software such as Microsoft Excel or without specialized computational expertise and programming environments.

CONCLUSIONS

CellPD is an open source tool that can be used by scientists (with or without a background in computational or mathematical modeling) to quantify key aspects of cell phenotypes (such as cell cycle and death parameters). Early applications of CellPD may include drug effect quantification, functional analysis of gene knockout experiments, data quality control, minable big data generation, and integration of biological data with computational models.

Connecting the dots: network data and models in HIV epidemiology

Effective HIV prevention requires knowledge of the structure and dynamics of the social networks across which infections are transmitted. These networks most commonly comprise chains of sexual relationships, but in some populations, sharing of contaminated needles is also an important, or even the main mechanism that connects people in the network. Whereas network data have long been collected during survey interviews, new data sources have become increasingly common in recent years, because of advances in molecular biology and the use of partner notification services in HIV prevention and treatment programmes. We review current and emerging methods for collecting HIV-related network data, as well as modelling frameworks commonly used to infer network parameters and map potential HIV transmission pathways within the network. We discuss the relative strengths and weaknesses of existing methods and models, and we propose a research agenda for advancing network analysis in HIV epidemiology. We make the case for a combination approach that integrates multiple data sources into a coherent statistical framework.

Inferring extrinsic noise from single-cell gene expression data using approximate Bayesian computation

Background

Gene expression is known to be an intrinsically stochastic process which can involve single-digit numbers of mRNA molecules in a cell at any given time. The modelling of such processes calls for the use of exact stochastic simulation methods, most notably the Gillespie algorithm. However, this stochasticity, also termed “intrinsic noise”, does not account for all the variability between genetically identical cells growing in a homogeneous environment.

Despite substantial experimental efforts, determining appropriate model parameters continues to be a challenge. Methods based on approximate Bayesian computation can be used to obtain posterior parameter distributions given the observed data. However, such inference procedures require large numbers of simulations of the model and exact stochastic simulation is computationally costly.

In this work we focus on the specific case of trying to infer model parameters describing reaction rates and extrinsic noise on the basis of measurements of molecule numbers in individual cells at a given time point.

Results

To make the problem computationally tractable we develop an exact, model-specific, stochastic simulation algorithm for the commonly used two-state model of gene expression. This algorithm relies on certain assumptions and favourable properties of the model to forgo the simulation of the whole temporal trajectory of protein numbers in the system, instead returning only the number of protein and mRNA molecules present in the system at a specified time point. The computational gain is proportional to the number of protein molecules created in the system and becomes significant for systems involving hundreds or thousands of protein molecules.

Conclusions

We employ this simulation algorithm with approximate Bayesian computation to jointly infer the model’s rate and noise parameters from published gene expression data. Our analysis indicates that for most genes the extrinsic contributions to noise will be small to moderate but certainly are non-negligible.

Electronic supplementary material

The online version of this article (doi:10.1186/s12918-016-0324-x) contains supplementary material, which is available to authorized users.

An integrated approach to infer dynamic protein-gene interactions - A case study of the human P53 protein

Investigating the dynamics of genetic regulatory networks through high throughput experimental data, such as microarray gene expression profiles, is a very important but challenging task. One of the major hindrances in building detailed mathematical models for genetic regulation is the large number of unknown model parameters. To tackle this challenge, a new integrated method is proposed by combining a top-down approach and a bottom-up approach. First, the top-down approach uses probabilistic graphical models to predict the network structure of DNA repair pathway that is regulated by the p53 protein. Two networks are predicted, namely a network of eight genes with eight inferred interactions and an extended network of 21 genes with 17 interactions. Then, the bottom-up approach using differential equation models is developed to study the detailed genetic regulations based on either a fully connected regulatory network or a gene network obtained by the top-down approach. Model simulation error, parameter identifiability and robustness property are used as criteria to select the optimal network. Simulation results together with permutation tests of input gene network structures indicate that the prediction accuracy and robustness property of the two predicted networks using the top-down approach are better than those of the corresponding fully connected networks. In particular, the proposed approach reduces computational cost significantly for inferring model parameters. Overall, the new integrated method is a promising approach for investigating the dynamics of genetic regulation.

A Statistical Approach Reveals Designs for the Most Robust Stochastic Gene Oscillators

The engineering of transcriptional networks presents many challenges due to the inherent uncertainty in the system structure, changing cellular context, and stochasticity in the governing dynamics. One approach to address these problems is to design and build systems that can function across a range of conditions; that is they are robust to uncertainty in their constituent components. Here we examine the parametric robustness landscape of transcriptional oscillators, which underlie many important processes such as circadian rhythms and the cell cycle, plus also serve as a model for the engineering of complex and emergent phenomena. The central questions that we address are: Can we build genetic oscillators that are more robust than those already constructed? Can we make genetic oscillators arbitrarily robust? These questions are technically challenging due to the large model and parameter spaces that must be efficiently explored. Here we use a measure of robustness that coincides with the Bayesian model evidence, combined with an efficient Monte Carlo method to traverse model space and concentrate on regions of high robustness, which enables the accurate evaluation of the relative robustness of gene network models governed by stochastic dynamics. We report the most robust two and three gene oscillator systems, plus examine how the number of interactions, the presence of autoregulation, and degradation of mRNA and protein affects the frequency, amplitude, and robustness of transcriptional oscillators. We also find that there is a limit to parametric robustness, beyond which there is nothing to be gained by adding additional feedback. Importantly, we provide predictions on new oscillator systems that can be constructed to verify the theory and advance design and modeling approaches to systems and synthetic biology.

Effect of STAT3 inhibitor in chronic myeloid leukemia associated signaling pathway: a mathematical modeling, simulation and systems biology study

Chronic myeloid leukemia (CML) is a hematopoietic stem-cell disorder which proliferates due to abnormal growth of basophil cells. Several proangiogenic molecules have been reported to be associated in CML progression, including the hepatocyte growth factor (HGF). However, detail mechanism about the cellular distribution and function of HGF in CML is yet to be revealed. The proliferation of hematopoietic cells are regulated by some of the growth factors like interleukin 3 (IL-3), IL-6, erythropoietin, thrombopoietin, etc. In this study IL-6 pathways have been taken into consideration which induces JAK/STAT and MAPK pathways to decipher the CML progression stages. An attempt has been made to model these pathways with the help of ordinary differential equations (ODEs) and estimating unknown parameters through fminsearch optimization algorithm. Some of the specific component like STAT3, of the pathway has been analyzed in detail and their role in CML progression has been elucidated. The roles of STAT3 inhibitors into the treatment of CML have been thoroughly studied and optimum concentration of the inhibitors have been predicted.

Model calibration and uncertainty analysis in signaling networks

For a long time the biggest challenges in modeling cellular signal transduction networks has been the inference of crucial pathway components and the qualitative description of their interactions. As a result of the emergence of powerful high-throughput experiments, it is now possible to measure data of high temporal and spatial resolution and to analyze signaling dynamics quantitatively. In addition, this increase of high-quality data is the basis for a better understanding of model limitations and their influence on the predictive power of models. We review established approaches in signal transduction network modeling with a focus on ordinary differential equation models as well as related developments in model calibration. As central aspects of the calibration process we discuss possibilities of model adaptation based on data-driven parameter optimization and the concomitant objective of reducing model uncertainties.

Quantitative Systems Pharmacology: A Framework for Context

Quantitative Systems Pharmacology (QSP) is receiving increased attention. As the momentum builds and the expectations grow it is important to (re)assess and formalize the basic concepts and approaches. In this short review, I argue that QSP, in addition to enabling the rational integration of data and development of complex models, maybe more importantly, provides the foundations for developing an integrated framework for the assessment of drugs and their impact on disease within a broader context expanding the envelope to account in great detail for physiology, environment and prior history. I articulate some of the critical enablers, major obstacles and exciting opportunities manifesting themselves along the way. Charting such overarching themes will enable practitioners to identify major and defining factors as the field progressively moves towards personalized and precision health care delivery.

Mathematical and Statistical Techniques for Systems Medicine: The Wnt Signaling Pathway as a Case Study

The last decade has seen an explosion in models that describe phenomena in systems medicine. Such models are especially useful for studying signaling pathways, such as the Wnt pathway. In this chapter we use the Wnt pathway to showcase current mathematical and statistical techniques that enable modelers to gain insight into (models of) gene regulation and generate testable predictions. We introduce a range of modeling frameworks, but focus on ordinary differential equation (ODE) models since they remain the most widely used approach in systems biology and medicine and continue to offer great potential. We present methods for the analysis of a single model, comprising applications of standard dynamical systems approaches such as nondimensionalization, steady state, asymptotic and sensitivity analysis, and more recent statistical and algebraic approaches to compare models with data. We present parameter estimation and model comparison techniques, focusing on Bayesian analysis and coplanarity via algebraic geometry. Our intention is that this (non-exhaustive) review may serve as a useful starting point for the analysis of models in systems medicine.