A Bayesian approach to estimating hidden variables as well as missing and wrong molecular interactions in ordinary differential equation-based mathematical models

Artificial neural networks enable genome-scale simulations of intracellular signaling

Mammalian cells adapt their functional state in response to external signals in form of ligands that bind receptors on the cell-surface. Mechanistically, this involves signal-processing through a complex network of molecular interactions that govern transcription factor activity patterns. Computer simulations of the information flow through this network could help predict cellular responses in health and disease. Here we develop a recurrent neural network framework constrained by prior knowledge of the signaling network with ligand-concentrations as input and transcription factor-activity as output. Applied to synthetic data, it predicts unseen test-data (Pearson correlation r = 0.98) and the effects of gene knockouts (r = 0.8). We stimulate macrophages with 59 different ligands, with and without the addition of lipopolysaccharide, and collect transcriptomics data. The framework predicts this data under cross-validation (r = 0.8) and knockout simulations suggest a role for RIPK1 in modulating the lipopolysaccharide response. This work demonstrates the feasibility of genome-scale simulations of intracellular signaling.

A Bayesian Approach to Extracting Kinetic Information from Artificial Enzymatic Networks

In order to create artificial enzymatic networks capable of increasingly complex behavior, an improved methodology in understanding and controlling the kinetics of these networks is needed. Here, we introduce a Bayesian analysis method allowing for the accurate inference of enzyme kinetic parameters and determination of most likely reaction mechanisms, by combining data from different experiments and network topologies in a single probabilistic analysis framework. This Bayesian approach explicitly allows us to continuously improve our parameter estimates and behavior predictions by iteratively adding new data to our models, while automatically taking into account uncertainties introduced by the experimental setups or the chemical processes in general. We demonstrate the potential of this approach by characterizing systems of enzymes compartmentalized in beads inside flow reactors. The methods we introduce here provide a new approach to the design of increasingly complex artificial enzymatic networks, making the design of such networks more efficient, and robust against the accumulation of experimental errors.

Numerical method for parameter inference of systems of nonlinear ordinary differential equations with partial observations

Parameter inference of dynamical systems is a challenging task faced by many researchers and practitioners across various fields. In many applications, it is common that only limited variables are observable. In this paper, we propose a method for parameter inference of a system of nonlinear coupled ordinary differential equations with partial observations. Our method combines fast Gaussian process-based gradient matching and deterministic optimization algorithms. By using initial values obtained by Bayesian steps with low sampling numbers, our deterministic optimization algorithm is both accurate, robust and efficient with partial observations and large noise.

SEEDS: data driven inference of structural model errors and unknown inputs for dynamic systems biology

SUMMARY

Dynamic models formulated as ordinary differential equations can provide information about the mechanistic and causal interactions in biological systems to guide targeted interventions and to design further experiments. Inaccurate knowledge about the structure, functional form and parameters of interactions is a major obstacle to mechanistic modeling. A further challenge is the open nature of biological systems which receive unknown inputs from their environment. The R-package SEEDS implements two recently developed algorithms to infer structural model errors and unknown inputs from output measurements. This information can facilitate efficient model recalibration as well as experimental design in the case of misfits between the initial model and data.

AVAILABILITY AND IMPLEMENTATION

For the R-package seeds, see the CRAN server https://cran.r-project.org/package=seeds.

Searching for Errors in Models of Complex Dynamic Systems

Mathematical modeling is seen as a key step to understand, predict, and control the temporal dynamics of interacting systems in such diverse areas like physics, biology, medicine, and economics. However, for large and complex systems we usually have only partial knowledge about the network, the coupling functions, and the interactions with the environment governing the dynamic behavior. This incomplete knowledge induces structural model errors which can in turn be the cause of erroneous model predictions or misguided interpretations. Uncovering the location of such structural model errors in large networks can be a daunting task for a modeler. Here, we present a data driven method to search for structural model errors and to confine their position in large and complex dynamic networks. We introduce a coherence measure for pairs of network nodes, which indicates, how difficult it is to distinguish these nodes as sources of an error. By clustering network nodes into coherence groups and inferring the cluster inputs we can decide, which cluster is affected by an error. We demonstrate the utility of our method for the C. elegans neural network, for a signal transduction model for UV-B light induced morphogenesis and for synthetic examples.

Development of a hybrid model for a partially known intracellular signaling pathway through correction term estimation and neural network modeling

Developing an accurate first-principle model is an important step in employing systems biology approaches to analyze an intracellular signaling pathway. However, an accurate first-principle model is difficult to be developed since it requires in-depth mechanistic understandings of the signaling pathway. Since underlying mechanisms such as the reaction network structure are not fully understood, significant discrepancy exists between predicted and actual signaling dynamics. Motivated by these considerations, this work proposes a hybrid modeling approach that combines a first-principle model and an artificial neural network (ANN) model so that predictions of the hybrid model surpass those of the original model. First, the proposed approach determines an optimal subset of model states whose dynamics should be corrected by the ANN by examining the correlation between each state and outputs through relative order. Second, an L2-regularized least-squares problem is solved to infer values of the correction terms that are necessary to minimize the discrepancy between the model predictions and available measurements. Third, an ANN is developed to generalize relationships between the values of the correction terms and the system dynamics. Lastly, the original first-principle model is coupled with the developed ANN to finalize the hybrid model development so that the model will possess generalized prediction capabilities while retaining the model interpretability. We have successfully validated the proposed methodology with two case studies, simplified apoptosis and lipopolysaccharide-induced NFκB signaling pathways, to develop hybrid models with in silico and in vitro measurements, respectively.

An intracellular signaling pathway is often represented by a set of nonlinear ordinary differential equations, which translate our current knowledge about the signaling pathway into a testable mathematical model. However, predictions from such models are often subject to high uncertainty since many signaling pathways are only partially known beforehand. In this study, we propose a systematic approach to develop a hybrid model to improve model accuracy by combining machine learning and the first-principle modeling. Specifically, model correction terms are learned from discrepancy between model predictions and measurements, and these terms are added to the first-principle model to enhance the prediction accuracy. Once these correction terms are learned from the data, an artificial neural network (ANN) model is developed to find an empirical relation between the model and the correction terms so that the developed ANN can be used to posses improved predictive capabilities even in new operating conditions (i.e., generalizability). The final hybrid model is then constructed by coupling the first-principle model with the developed ANN.

Systems biology informed deep learning for inferring parameters and hidden dynamics

Mathematical models of biological reactions at the system-level lead to a set of ordinary differential equations with many unknown parameters that need to be inferred using relatively few experimental measurements. Having a reliable and robust algorithm for parameter inference and prediction of the hidden dynamics has been one of the core subjects in systems biology, and is the focus of this study. We have developed a new systems-biology-informed deep learning algorithm that incorporates the system of ordinary differential equations into the neural networks. Enforcing these equations effectively adds constraints to the optimization procedure that manifests itself as an imposed structure on the observational data. Using few scattered and noisy measurements, we are able to infer the dynamics of unobserved species, external forcing, and the unknown model parameters. We have successfully tested the algorithm for three different benchmark problems.

Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models

In this paper, we address the system identification problem in the context of biological modelling. We present and demonstrate a methodology for (i) assessing the possibility of inferring the unknown quantities in a dynamic model and (ii) effectively estimating them from output data. We introduce the term Full Input-State-Parameter Observability (FISPO) analysis to refer to the simultaneous assessment of state, input and parameter observability (note that parameter observability is also known as identifiability). This type of analysis has often remained elusive in the presence of unmeasured inputs. The method proposed in this paper can be applied to a general class of nonlinear ordinary differential equations models. We apply this approach to three models from the recent literature. First, we determine whether it is theoretically possible to infer the states, parameters and inputs, taking only the model equations into account. When this analysis detects deficiencies, we reformulate the model to make it fully observable. Then we move to numerical scenarios and apply an optimization-based technique to estimate the states, parameters and inputs. The results demonstrate the feasibility of an integrated strategy for (i) analysing the theoretical possibility of determining the states, parameters and inputs to a system and (ii) solving the practical problem of actually estimating their values.

Computational Methods for Estimating Molecular System from Membrane Potential Recordings in Nerve Growth Cone

Biological cells express intracellular biomolecular information to the extracellular environment as various physical responses. We show a novel computational approach to estimate intracellular biomolecular pathways from growth cone electrophysiological responses. Previously, it was shown that cGMP signaling regulates membrane potential (MP) shifts that control the growth cone turning direction during neuronal development. We present here an integrated deterministic mathematical model and Bayesian reversed-engineering framework that enables estimation of the molecular signaling pathway from electrical recordings and considers both the system uncertainty and cell-to-cell variability. Our computational method selects the most plausible molecular pathway from multiple candidates while satisfying model simplicity and considering all possible parameter ranges. The model quantitatively reproduces MP shifts depending on cGMP levels and MP variability potential in different experimental conditions. Lastly, our model predicts that chloride channel inhibition by cGMP-dependent protein kinase (PKG) is essential in the core system for regulation of the MP shifts.