A new hierarchical approach to multi-level model abstraction for simplifying ODE models of biological networks and a case study: The G1/S Checkpoint/DNA damage signalling pathways of mammalian cell cycle

Model reduction is an important topic in studies of biological systems. By reducing the complexity of large models through multi-level models while keeping the essence (biological meaning) of the model, model reduction can help answer many important questions about these systems. In this paper, we present a new reduction method based on hierarchical representation and a lumping approach. We used G1/S checkpoint pathway represented in Ordinary Differential Equations (ODE) in Iwamoto et al. (2011) as a case study to present this reduction method. The approach consists of two parts; the first part represents the biological network as a hierarchy (multiple levels) based on protein binding relations, which allowed us to model the biological network at different levels of abstraction. The second part applies different levels (level 1, 2 and 3) of lumping the species together depending on the level of the hierarchy, resulting in a reduced and transformed model for each level. The model at each level is a representation of the whole system and can address questions pertinent to that level. We develop and simulate reduced models for levels-1, 2 and 3 of lumping for the G1/S checkpoint pathway and evaluate the biological plausibility of the proposed method by comparing the results with the original ODE model of Iwamoto et al. (2011). The results for continuous dynamics of the G1/S checkpoint pathway with or without DNA-damage for reduced models of level- 1, 2 and 3 of lumping are in very good agreement and consistent with the original model results and with biological findings. Therefore, the reduced models (level-1, 2 and 3) can be used to study cell cycle progression in G1 and the effects of DNA damage on it. It is suitable for reducing complex ODE biological network models while retaining accurate continuous dynamics of the system. The 3 levels of the reduced models respectively achieved 20%, 26% and 31% reduction of the base model. Moreover, the reduced model is more efficient to run (30%, 44% and 52% time reduction for the three levels) and generate solutions than the original ODE model. Simplification of complex mathematical models is possible and the proposed reduction method has the potential to make an impact across many fields of biomedical research.

Rapid data-driven model reduction of nonlinear dynamical systems including chemical reaction networks using ℓ1-regularization

Large-scale nonlinear dynamical systems, such as models of atmospheric hydrodynamics, chemical reaction networks, and electronic circuits, often involve thousands or more interacting components. In order to identify key components in the complex dynamical system as well as to accelerate simulations, model reduction is often desirable. In this work, we develop a new data-driven method utilizing ℓ¹-regularization for model reduction of nonlinear dynamical systems, which involves minimal parameterization and has polynomial-time complexity, allowing it to easily handle large-scale systems with as many as thousands of components in a matter of minutes. A primary objective of our model reduction method is interpretability, that is to identify key components of the dynamical system that contribute to behaviors of interest, rather than just finding an efficient projection of the dynamical system onto lower dimensions. Our method produces a family of reduced models that exhibit a trade-off between model complexity and estimation error. We find empirically that our method chooses reduced models with good extrapolation properties, an important consideration in practical applications. The reduction and extrapolation performance of our method are illustrated by applications to the Lorenz model and chemical reaction rate equations, where performance is found to be competitive with or better than state-of-the-art approaches.

Parallel Tempering with Lasso for model reduction in systems biology

Systems Biology models reveal relationships between signaling inputs and observable molecular or cellular behaviors. The complexity of these models, however, often obscures key elements that regulate emergent properties. We use a Bayesian model reduction approach that combines Parallel Tempering with Lasso regularization to identify minimal subsets of reactions in a signaling network that are sufficient to reproduce experimentally observed data. The Bayesian approach finds distinct reduced models that fit data equivalently. A variant of this approach that uses Lasso to perform selection at the level of reaction modules is applied to the NF-κB signaling network to test the necessity of feedback loops for responses to pulsatile and continuous pathway stimulation. Taken together, our results demonstrate that Bayesian parameter estimation combined with regularization can isolate and reveal core motifs sufficient to explain data from complex signaling systems.

Cells respond to diverse environmental cues using complex networks of interacting proteins and other biomolecules. Mathematical and computational models have become invaluable tools to understand these networks and make informed predictions to rationally perturb cell behavior. However, the complexity of detailed models that try to capture all known biochemical elements of signaling networks often makes it difficult to determine the key regulatory elements that are responsible for specific cell behaviors. Here, we present a Bayesian computational approach, PTLasso, to automatically extract minimal subsets of detailed models that are sufficient to explain experimental data. The method simultaneously calibrates and reduces models, and the Bayesian approach samples globally, allowing us to find alternate mechanistic explanations for the data if present. We demonstrate the method on both synthetic and real biological data and show that PTLasso is an effective method to isolate distinct parts of a larger signaling model that are sufficient for specific data.

Principal process analysis of biological models

BACKGROUND

Understanding the dynamical behaviour of biological systems is challenged by their large number of components and interactions. While efforts have been made in this direction to reduce model complexity, they often prove insufficient to grasp which and when model processes play a crucial role. Answering these questions is fundamental to unravel the functioning of living organisms.

RESULTS

We design a method for dealing with model complexity, based on the analysis of dynamical models by means of Principal Process Analysis. We apply the method to a well-known model of circadian rhythms in mammals. The knowledge of the system trajectories allows us to decompose the system dynamics into processes that are active or inactive with respect to a certain threshold value. Process activities are graphically represented by Boolean and Dynamical Process Maps. We detect model processes that are always inactive, or inactive on some time interval. Eliminating these processes reduces the complex dynamics of the original model to the much simpler dynamics of the core processes, in a succession of sub-models that are easier to analyse. We quantify by means of global relative errors the extent to which the simplified models reproduce the main features of the original system dynamics and apply global sensitivity analysis to test the influence of model parameters on the errors.

CONCLUSION

The results obtained prove the robustness of the method. The analysis of the sub-model dynamics allows us to identify the source of circadian oscillations. We find that the negative feedback loop involving proteins PER, CRY, CLOCK-BMAL1 is the main oscillator, in agreement with previous modelling and experimental studies. In conclusion, Principal Process Analysis is a simple-to-use method, which constitutes an additional and useful tool for analysing the complex dynamical behaviour of biological systems.

Learning reduced kinetic Monte Carlo models of complex chemistry from molecular dynamics

We propose a novel statistical learning framework for automatically and efficiently building reduced kinetic Monte Carlo (KMC) models of large-scale elementary reaction networks from data generated by a single or few molecular dynamics simulations (MD). Existing approaches for identifying species and reactions from molecular dynamics typically use bond length and duration criteria, where bond duration is a fixed parameter motivated by an understanding of bond vibrational frequencies. In contrast, we show that for highly reactive systems, bond duration should be a model parameter that is chosen to maximize the predictive power of the resulting statistical model. We demonstrate our method on a high temperature, high pressure system of reacting liquid methane, and show that the learned KMC model is able to extrapolate more than an order of magnitude in time for key molecules. Additionally, our KMC model of elementary reactions enables us to isolate the most important set of reactions governing the behavior of key molecules found in the MD simulation. We develop a new data-driven algorithm to reduce the chemical reaction network which can be solved either as an integer program or efficiently using L1 regularization, and compare our results with simple count-based reduction. For our liquid methane system, we discover that rare reactions do not play a significant role in the system, and find that less than 7% of the approximately 2000 reactions observed from molecular dynamics are necessary to reproduce the molecular concentration over time of methane. The framework described in this work paves the way towards a genomic approach to studying complex chemical systems, where expensive MD simulation data can be reused to contribute to an increasingly large and accurate genome of elementary reactions and rates.

Driving the Model to Its Limit: Profile Likelihood Based Model Reduction

In systems biology, one of the major tasks is to tailor model complexity to information content of the data. A useful model should describe the data and produce well-determined parameter estimates and predictions. Too small of a model will not be able to describe the data whereas a model which is too large tends to overfit measurement errors and does not provide precise predictions. Typically, the model is modified and tuned to fit the data, which often results in an oversized model. To restore the balance between model complexity and available measurements, either new data has to be gathered or the model has to be reduced. In this manuscript, we present a data-based method for reducing non-linear models. The profile likelihood is utilised to assess parameter identifiability and designate likely candidates for reduction. Parameter dependencies are analysed along profiles, providing context-dependent suggestions for the type of reduction. We discriminate four distinct scenarios, each associated with a specific model reduction strategy. Iterating the presented procedure eventually results in an identifiable model, which is capable of generating precise and testable predictions. Source code for all toy examples is provided within the freely available, open-source modelling environment Data2Dynamics based on MATLAB available at http://www.data2dynamics.org/, as well as the R packages dMod/cOde available at https://github.com/dkaschek/. Moreover, the concept is generally applicable and can readily be used with any software capable of calculating the profile likelihood.

Diagnostics for stochastic genome-scale modeling via model slicing and debugging

Modeling of biological behavior has evolved from simple gene expression plots represented by mathematical equations to genome-scale systems biology networks. However, due to obstacles in complexity and scalability of creating genome-scale models, several biological modelers have turned to programming or scripting languages and away from modeling fundamentals. In doing so, they have traded the ability to have exchangeable, standardized model representation formats, while those that remain true to standardized model representation are faced with challenges in model complexity and analysis. We have developed a model diagnostic methodology inspired by program slicing and debugging and demonstrate the effectiveness of the methodology on a genome-scale metabolic network model published in the BioModels database. The computer-aided identification revealed specific points of interest such as reversibility of reactions, initialization of species amounts, and parameter estimation that improved a candidate cell's adenosine triphosphate production. We then compared the advantages of our methodology over other modeling techniques such as model checking and model reduction. A software application that implements the methodology is available at http://gel.ym.edu.tw/gcs/.

A model reduction method for biochemical reaction networks

BACKGROUND

In this paper we propose a model reduction method for biochemical reaction networks governed by a variety of reversible and irreversible enzyme kinetic rate laws, including reversible Michaelis-Menten and Hill kinetics. The method proceeds by a stepwise reduction in the number of complexes, defined as the left and right-hand sides of the reactions in the network. It is based on the Kron reduction of the weighted Laplacian matrix, which describes the graph structure of the complexes and reactions in the network. It does not rely on prior knowledge of the dynamic behaviour of the network and hence can be automated, as we demonstrate. The reduced network has fewer complexes, reactions, variables and parameters as compared to the original network, and yet the behaviour of a preselected set of significant metabolites in the reduced network resembles that of the original network. Moreover the reduced network largely retains the structure and kinetics of the original model.

RESULTS

We apply our method to a yeast glycolysis model and a rat liver fatty acid beta-oxidation model. When the number of state variables in the yeast model is reduced from 12 to 7, the difference between metabolite concentrations in the reduced and the full model, averaged over time and species, is only 8%. Likewise, when the number of state variables in the rat-liver beta-oxidation model is reduced from 42 to 29, the difference between the reduced model and the full model is 7.5%.

CONCLUSIONS

The method has improved our understanding of the dynamics of the two networks. We found that, contrary to the general disposition, the first few metabolites which were deleted from the network during our stepwise reduction approach, are not those with the shortest convergence times. It shows that our reduction approach performs differently from other approaches that are based on time-scale separation. The method can be used to facilitate fitting of the parameters or to embed a detailed model of interest in a more coarse-grained yet realistic environment.

Simultaneous model discrimination and parameter estimation in dynamic models of cellular systems

Background

Model development is a key task in systems biology, which typically starts from an initial model candidate and, involving an iterative cycle of hypotheses-driven model modifications, leads to new experimentation and subsequent model identification steps. The final product of this cycle is a satisfactory refined model of the biological phenomena under study. During such iterative model development, researchers frequently propose a set of model candidates from which the best alternative must be selected. Here we consider this problem of model selection and formulate it as a simultaneous model selection and parameter identification problem. More precisely, we consider a general mixed-integer nonlinear programming (MINLP) formulation for model selection and identification, with emphasis on dynamic models consisting of sets of either ODEs (ordinary differential equations) or DAEs (differential algebraic equations).

Results

We solved the MINLP formulation for model selection and identification using an algorithm based on Scatter Search (SS). We illustrate the capabilities and efficiency of the proposed strategy with a case study considering the KdpD/KdpE system regulating potassium homeostasis in Escherichia coli. The proposed approach resulted in a final model that presents a better fit to the in silico generated experimental data.

Conclusions

The presented MINLP-based optimization approach for nested-model selection and identification is a powerful methodology for model development in systems biology. This strategy can be used to perform model selection and parameter estimation in one single step, thus greatly reducing the number of experiments and computations of traditional modeling approaches.

Mixed-integer nonlinear optimisation approach to coarse-graining biochemical networks

Quantitative modelling and analysis of biochemical networks is challenging because of the inherent complexities and nonlinearities of the system and the limited availability of parameter values. Even if a mathematical model of the network can be developed, the lack of large-scale good-quality data makes accurate estimation of a large number of parameters impossible. Hence, coarse-grained models (CGMs) consisting of essential biochemical mechanisms are more suitable for computational analysis and for studying important systemic functions. The central question in constructing a CGM is which mechanisms should be deemed 'essential' and which can be ignored. Also, how should parameter values be defined when data are sparse? A mixed-integer nonlinear-programming (MINLP) based optimisation approach to coarse-graining is presented. Starting with a detailed biochemical model with associated computational details (reaction network and mathematical description) and data on the biochemical system, the structure and the parameters of a CGM can be determined simultaneously. In this optimisation problem, the authors use a genetic algorithm to simultaneously identify parameter values and remove unimportant reactions. The methodology is exemplified by developing two CGMs for the GTPase-cycle module of M1 muscarinic acetylcholine receptor, Gq, and regulator of G protein signalling 4 [RGS4, a GTPase-activating protein (GAP)] starting from a detailed model of 48 reactions. Both the CGMs have only 17 reactions, fit experimental data well and predict, as does the detailed model, four limiting signalling regimes (LSRs) corresponding to the extremes of receptor and GAP concentration. The authors demonstrate that coarse-graining, in addition to resulting in a reduced-order model, also provides insights into the mechanisms in the network. The best CGM obtained for the GTPase cycle also contains an unconventional mechanism and its predictions explain an old problem in pharmacology, the biphasic (bell-shaped) response to certain drugs. The MINLP methodology is broadly applicable to larger and complex (dense) biochemical modules.

Oscillator model reduction preserving the phase response: application to the circadian clock

Mathematical model reduction is a long-standing technique used both to gain insight into model subprocesses and to reduce the computational costs of simulation and analysis. A reduced model must retain essential features of the full model, which, traditionally, have been the trajectories of certain state variables. For biological clocks, timing, or phase, characteristics must be preserved. A key performance criterion for a clock is the ability to adjust its phase correctly in response to external signals. We present a novel model reduction technique that removes components from a single-oscillator clock model and discover that four feedback loops are redundant with respect to its phase response behavior. Using a coupled multioscillator model of a circadian clock, we demonstrate that by preserving the phase response behavior of a single oscillator, we preserve timing behavior at the multioscillator level.

Approximation of slow attracting manifolds in chemical kinetics by trajectory-based optimization approaches

Many common kinetic model reduction approaches are explicitly based on inherent multiple time scales and often assume and directly exploit a clear time scale separation into fast and slow reaction processes. They approximate the system dynamics with a dimension-reduced model after eliminating the fast modes by enslaving them to the slow ones. The corresponding restrictive assumption of full relaxation of fast modes often renders the resulting approximation of slow attracting manifolds inaccurate as a representation of the reduced model and makes the numerical solution of the nonlinear "reduction equations" particularly difficult in many cases where the gap in intrinsic time scales is not large enough. We demonstrate that trajectory optimization approaches can avoid such severe restrictions by computing numerical solutions that correspond to "maximally relaxed" dynamical modes in a suitable sense. We present a framework of trajectory-based optimization for model reduction in chemical kinetics and a general class of reduction criteria characterizing the relaxation of chemical forces along reaction trajectories. These criteria can be motivated geometrically exploiting ideas from differential geometry and fundamental physics and turn out to be highly successful in example applications. Within this framework, we provide results for the computational approximation of slow attracting low-dimensional manifolds in terms of families of optimal trajectories for a six-component hydrogen combustion mechanism.

Simplification of stochastic chemical reaction models with fast and slow dynamics

Biological systems often involve chemical reactions occurring in low-molecule-number regimes, where fluctuations are not negligible and thus stochastic models are required to capture the system behaviour. The resulting models are generally quite large and complex, involving many reactions and species. For clarity and computational tractability, it is important to be able to simplify these systems to equivalent ones involving fewer elements. While many model simplification approaches have been developed for deterministic systems, there has been limited work on applying these approaches to stochastic modelling. Here, we describe a method that reduces the complexity of stochastic biochemical network models, and apply this method to the reduction of a mammalian signalling cascade and a detailed model of the process of bacterial gene expression. Our results indicate that the simplified model gives an accurate representation for not only the average numbers of all species, but also for the associated fluctuations and statistical parameters.

Reduced-order modelling of biochemical networks: application to the GTPase-cycle signalling module

Biochemical systems embed complex networks and hence development and analysis of their detailed models pose a challenge for computation. Coarse-grained biochemical models, called reduced-order models (ROMs), consisting of essential biochemical mechanisms are more useful for computational analysis and for studying important features of a biochemical network. The authors present a novel method to model-reduction by identifying potentially important parameters using multidimensional sensitivity analysis. A ROM is generated for the GTPase-cycle module of m1 muscarinic acetylcholine receptor, Gq, and regulator of G-protein signalling 4 (a GTPase-activating protein or GAP) starting from a detailed model of 48 reactions. The resulting ROM has only 17 reactions. The ROM suggested that complexes of G-protein coupled receptor (GPCR) and GAP--which were proposed in the detailed model as a hypothesis--are required to fit the experimental data. Models previously published in the literature are also simulated and compared with the ROM. Through this comparison, a minimal ROM, that also requires complexes of GPCR and GAP, with just 15 parameters is generated. The proposed reduced-order modelling methodology is scalable to larger networks and provides a general framework for the reduction of models of biochemical systems.

Derivation of a quantitative minimal model from a detailed elementary-step mechanism supported by mathematical coupling analysis

Accurate experimental data increasingly allow the development of detailed elementary-step mechanisms for complex chemical and biochemical reaction systems. Model reduction techniques are widely applied to obtain representations in lower-dimensional phase space which are more suitable for mathematical analysis, efficient numerical simulation, and model-based control tasks. Here, we exploit a recently implemented numerical algorithm for error-controlled computation of the minimum dimension required for a still accurate reduced mechanism based on automatic time scale decomposition and relaxation of fast modes. We determine species contributions to the active (slow) dynamical modes of the reaction system and exploit this information in combination with quasi-steady-state and partial-equilibrium approximations for explicit model reduction of a novel detailed chemical mechanism for the Ru-catalyzed light-sensitive Belousov-Zhabotinsky reaction. The existence of a minimum dimension of seven is demonstrated to be mandatory for the reduced model to show good quantitative consistency with the full model in numerical simulations. We derive such a maximally reduced seven-variable model from the detailed elementary-step mechanism and demonstrate that it reproduces quantitatively accurately the dynamical features of the full model within a given accuracy tolerance.

Automatic network coupling analysis for dynamical systems based on detailed kinetic models

We introduce a numerical complexity reduction method for the automatic identification and analysis of dynamic network decompositions in (bio)chemical kinetics based on error-controlled computation of a minimal model dimension represented by the number of (locally) active dynamical modes. Our algorithm exploits a generalized sensitivity analysis along state trajectories and subsequent singular value decomposition of sensitivity matrices for the identification of these dominant dynamical modes. It allows for a dynamic coupling analysis of (bio)chemical species in kinetic models that can be exploited for the piecewise computation of a minimal model on small time intervals and offers valuable functional insight into highly nonlinear reaction mechanisms and network dynamics. We present results for the identification of network decompositions in a simple oscillatory chemical reaction, time scale separation based model reduction in a Michaelis-Menten enzyme system and network decomposition of a detailed model for the oscillatory peroxidase-oxidase enzyme system.

Computing minimal entropy production trajectories: An approach to model reduction in chemical kinetics

Advanced experimental techniques in chemistry and physics provide increasing access to detailed deterministic mass action models for chemical reaction kinetics. Especially in complex technical or biochemical systems the huge amount of species and reaction pathways involved in a detailed modeling approach call for efficient methods of model reduction. These should be automatic and based on a firm mathematical analysis of the ordinary differential equations underlying the chemical kinetics in deterministic models. A main purpose of model reduction is to enable accurate numerical simulations of even high dimensional and spatially extended reaction systems. The latter include physical transport mechanisms and are modeled by partial differential equations. Their numerical solution for hundreds or thousands of species within a reasonable time will exceed computer capacities available now and in a foreseeable future. The central idea of model reduction is to replace the high dimensional dynamics by a low dimensional approximation with an appropriate degree of accuracy. Here I present a global approach to model reduction based on the concept of minimal entropy production and its numerical implementation. For given values of a single species concentration in a chemical system all other species concentrations are computed under the assumption that the system is as close as possible to its attractor, the thermodynamic equilibrium, in the sense that all modes of thermodynamic forces are maximally relaxed except the one, which drives the remaining system dynamics. This relaxation is expressed in terms of minimal entropy production for single reaction steps along phase space trajectories.