Customized Steady-State Constraints for Parameter Estimation in Non-Linear Ordinary Differential Equation Models

Benchmarking methods for computing local sensitivities in ordinary differential equation models at dynamic and steady states

Estimating parameters of dynamic models from experimental data is a challenging, and often computationally-demanding task. It requires a large number of model simulations and objective function gradient computations, if gradient-based optimization is used. In many cases, steady-state computation is a part of model simulation, either due to steady-state data or an assumption that the system is at steady state at the initial time point. Various methods are available for steady-state and gradient computation. Yet, the most efficient pair of methods (one for steady states, one for gradients) for a particular model is often not clear. In order to facilitate the selection of methods, we explore six method pairs for computing the steady state and sensitivities at steady state using six real-world problems. The method pairs involve numerical integration or Newton's method to compute the steady-state, and-for both forward and adjoint sensitivity analysis-numerical integration or a tailored method to compute the sensitivities at steady-state. Our evaluation shows that all method pairs provide accurate steady-state and gradient values, and that the two method pairs that combine numerical integration for the steady-state with a tailored method for the sensitivities at steady-state were the most robust, and amongst the most computationally-efficient. We also observed that while Newton's method for steady-state computation yields a substantial speedup compared to numerical integration, it may lead to a large number of simulation failures. Overall, our study provides a concise overview across current methods for computing sensitivities at steady state. While our study shows that there is no universally-best method pair, it also provides guidance to modelers in choosing the right methods for a problem at hand.

Basal MET phosphorylation is an indicator of hepatocyte dysregulation in liver disease

Chronic liver diseases are worldwide on the rise. Due to the rapidly increasing incidence, in particular in Western countries, metabolic dysfunction-associated steatotic liver disease (MASLD) is gaining importance as the disease can develop into hepatocellular carcinoma. Lipid accumulation in hepatocytes has been identified as the characteristic structural change in MASLD development, but molecular mechanisms responsible for disease progression remained unresolved. Here, we uncover in primary hepatocytes from a preclinical model fed with a Western diet (WD) an increased basal MET phosphorylation and a strong downregulation of the PI3K-AKT pathway. Dynamic pathway modeling of hepatocyte growth factor (HGF) signal transduction combined with global proteomics identifies that an elevated basal MET phosphorylation rate is the main driver of altered signaling leading to increased proliferation of WD-hepatocytes. Model-adaptation to patient-derived hepatocytes reveal patient-specific variability in basal MET phosphorylation, which correlates with patient outcome after liver surgery. Thus, dysregulated basal MET phosphorylation could be an indicator for the health status of the liver and thereby inform on the risk of a patient to suffer from liver failure after surgery.

Efficient computation of adjoint sensitivities at steady-state in ODE models of biochemical reaction networks

Dynamical models in the form of systems of ordinary differential equations have become a standard tool in systems biology. Many parameters of such models are usually unknown and have to be inferred from experimental data. Gradient-based optimization has proven to be effective for parameter estimation. However, computing gradients becomes increasingly costly for larger models, which are required for capturing the complex interactions of multiple biochemical pathways. Adjoint sensitivity analysis has been pivotal for working with such large models, but methods tailored for steady-state data are currently not available. We propose a new adjoint method for computing gradients, which is applicable if the experimental data include steady-state measurements. The method is based on a reformulation of the backward integration problem to a system of linear algebraic equations. The evaluation of the proposed method using real-world problems shows a speedup of total simulation time by a factor of up to 4.4. Our results demonstrate that the proposed approach can achieve a substantial improvement in computation time, in particular for large-scale models, where computational efficiency is critical.

Disentangling ERBB Signaling in Breast Cancer Subtypes-A Model-Based Analysis

Targeted therapies have shown striking success in the treatment of cancer over the last years. However, their specific effects on an individual tumor appear to be varying and difficult to predict. Using an integrative modeling approach that combines mechanistic and regression modeling, we gained insights into the response mechanisms of breast cancer cells due to different ligand-drug combinations. The multi-pathway model, capturing ERBB receptor signaling as well as downstream MAPK and PI3K pathways was calibrated on time-resolved data of the luminal breast cancer cell lines MCF7 and T47D across an array of four ligands and five drugs. The same model was then successfully applied to triple negative and HER2-positive breast cancer cell lines, requiring adjustments mostly for the respective receptor compositions within these cell lines. The additional relevance of cell-line-specific mutations in the MAPK and PI3K pathway components was identified via L1 regularization, where the impact of these mutations on pathway activation was uncovered. Finally, we predicted and experimentally validated the proliferation response of cells to drug co-treatments. We developed a unified mathematical model that can describe the ERBB receptor and downstream signaling in response to therapeutic drugs targeting this clinically relevant signaling network in cell line that represent three major subtypes of breast cancer. Our data and model suggest that alterations in this network could render anti-HER therapies relevant beyond the HER2-positive subtype.

Dynamic modeling of Nrf2 pathway activation in liver cells after toxicant exposure

Cells are exposed to oxidative stress and reactive metabolites every day. The Nrf2 signaling pathway responds to oxidative stress by upregulation of antioxidants like glutathione (GSH) to compensate the stress insult and re-establish homeostasis. Although mechanisms describing the interaction between the key pathway constituents Nrf2, Keap1 and p62 are widely reviewed and discussed in literature, quantitative dynamic models bringing together these mechanisms with time-resolved data are limited. Here, we present an ordinary differential equation (ODE) based dynamic model to describe the dynamic response of Nrf2, Keap1, Srxn1 and GSH to oxidative stress caused by the soft-electrophile diethyl maleate (DEM). The time-resolved data obtained by single-cell confocal microscopy of green fluorescent protein (GFP) reporters and qPCR of the Nrf2 pathway components complemented with siRNA knock down experiments, is accurately described by the calibrated mathematical model. We show that the quantitative model can describe the activation of the Nrf2 pathway by compounds with a different mechanism of activation, including drugs which are known for their ability to cause drug induced liver-injury (DILI) i.e., diclofenac (DCF) and omeprazole (OMZ). Finally, we show that our model can reveal differences in the processes leading to altered activation dynamics amongst DILI inducing drugs.

Parameterization of mechanistic models from qualitative data using an efficient optimal scaling approach

Quantitative dynamical models facilitate the understanding of biological processes and the prediction of their dynamics. These models usually comprise unknown parameters, which have to be inferred from experimental data. For quantitative experimental data, there are several methods and software tools available. However, for qualitative data the available approaches are limited and computationally demanding. Here, we consider the optimal scaling method which has been developed in statistics for categorical data and has been applied to dynamical systems. This approach turns qualitative variables into quantitative ones, accounting for constraints on their relation. We derive a reduced formulation for the optimization problem defining the optimal scaling. The reduced formulation possesses the same optimal points as the established formulation but requires less degrees of freedom. Parameter estimation for dynamical models of cellular pathways revealed that the reduced formulation improves the robustness and convergence of optimizers. This resulted in substantially reduced computation times. We implemented the proposed approach in the open-source Python Parameter EStimation TOolbox (pyPESTO) to facilitate reuse and extension. The proposed approach enables efficient parameterization of quantitative dynamical models using qualitative data.

Disentangling molecular mechanisms regulating sensitization of interferon alpha signal transduction

Tightly interlinked feedback regulators control the dynamics of intracellular responses elicited by the activation of signal transduction pathways. Interferon alpha (IFNα) orchestrates antiviral responses in hepatocytes, yet mechanisms that define pathway sensitization in response to prestimulation with different IFNα doses remained unresolved. We establish, based on quantitative measurements obtained for the hepatoma cell line Huh7.5, an ordinary differential equation model for IFNα signal transduction that comprises the feedback regulators STAT1, STAT2, IRF9, USP18, SOCS1, SOCS3, and IRF2. The model-based analysis shows that, mediated by the signaling proteins STAT2 and IRF9, prestimulation with a low IFNα dose hypersensitizes the pathway. In contrast, prestimulation with a high dose of IFNα leads to a dose-dependent desensitization, mediated by the negative regulators USP18 and SOCS1 that act at the receptor. The analysis of basal protein abundance in primary human hepatocytes reveals high heterogeneity in patient-specific amounts of STAT1, STAT2, IRF9, and USP18. The mathematical modeling approach shows that the basal amount of USP18 determines patient-specific pathway desensitization, while the abundance of STAT2 predicts the patient-specific IFNα signal response.

Benchmark problems for dynamic modeling of intracellular processes

Motivation

Dynamic models are used in systems biology to study and understand cellular processes like gene regulation or signal transduction. Frequently, ordinary differential equation (ODE) models are used to model the time and dose dependency of the abundances of molecular compounds as well as interactions and translocations. A multitude of computational approaches, e.g. for parameter estimation or uncertainty analysis have been developed within recent years. However, many of these approaches lack proper testing in application settings because a comprehensive set of benchmark problems is yet missing.

Results

We present a collection of 20 benchmark problems in order to evaluate new and existing methodologies, where an ODE model with corresponding experimental data is referred to as problem. In addition to the equations of the dynamical system, the benchmark collection provides observation functions as well as assumptions about measurement noise distributions and parameters. The presented benchmark models comprise problems of different size, complexity and numerical demands. Important characteristics of the models and methodological requirements are summarized, estimated parameters are provided, and some example studies were performed for illustrating the capabilities of the presented benchmark collection.

Availability and implementation

The models are provided in several standardized formats, including an easy-to-use human readable form and machine-readable SBML files. The data is provided as Excel sheets. All files are available at https://github.com/Benchmarking-Initiative/Benchmark-Models, including step-by-step explanations and MATLAB code to process and simulate the models.

Supplementary information

Supplementary data are available at Bioinformatics online.

Benchmarking optimization methods for parameter estimation in large kinetic models

Motivation

Kinetic models contain unknown parameters that are estimated by optimizing the fit to experimental data. This task can be computationally challenging due to the presence of local optima and ill-conditioning. While a variety of optimization methods have been suggested to surmount these issues, it is difficult to choose the best one for a given problem a priori. A systematic comparison of parameter estimation methods for problems with tens to hundreds of optimization variables is currently missing, and smaller studies provided contradictory findings.

Results

We use a collection of benchmarks to evaluate the performance of two families of optimization methods: (i) multi-starts of deterministic local searches and (ii) stochastic global optimization metaheuristics; the latter may be combined with deterministic local searches, leading to hybrid methods. A fair comparison is ensured through a collaborative evaluation and a consideration of multiple performance metrics. We discuss possible evaluation criteria to assess the trade-off between computational efficiency and robustness. Our results show that, thanks to recent advances in the calculation of parametric sensitivities, a multi-start of gradient-based local methods is often a successful strategy, but a better performance can be obtained with a hybrid metaheuristic. The best performer combines a global scatter search metaheuristic with an interior point local method, provided with gradients estimated with adjoint-based sensitivities. We provide an implementation of this method to render it available to the scientific community.

Availability and implementation

The code to reproduce the results is provided as Supplementary Material and is available at Zenodo https://doi.org/10.5281/zenodo.1304034.

Supplementary information

Supplementary data are available at Bioinformatics online.

Fitting mathematical models of biochemical pathways to steady state perturbation response data without simulating perturbation experiments

Fitting Ordinary Differential Equation (ODE) models of signal transduction networks (STNs) to experimental data is a challenging problem. Computational parameter fitting algorithms simulate a model many times with different sets of parameter values until the simulated STN behaviour match closely with experimental data. This process can be slow when the model is fitted to measurements of STN responses to numerous perturbations, since this requires simulating the model as many times as the number of perturbations for each set of parameter values. Here, I propose an approach that avoids simulating perturbation experiments when fitting ODE models to steady state perturbation response (SSPR) data. Instead of fitting the model directly to SSPR data, it finds model parameters which provides a close match between the scaled Jacobian matrices (SJM) of the model, which are numerically calculated using the model's rate equations and estimated from SSPR data using modular response analysis (MRA). The numerical estimation of SJM of an ODE model does not require simulating perturbation experiments, saving significant computation time. The effectiveness of this approach is demonstrated by fitting ODE models of the Mitogen Activated Protein Kinase (MAPK) pathway using simulated and real SSPR data.

Continuous analogue to iterative optimization for PDE-constrained inverse problems

The parameters of many physical processes are unknown and have to be inferred from experimental data. The corresponding parameter estimation problem is often solved using iterative methods such as steepest descent methods combined with trust regions. For a few problem classes also continuous analogues of iterative methods are available. In this work, we expand the application of continuous analogues to function spaces and consider PDE (partial differential equation)-constrained optimization problems. We derive a class of continuous analogues, here coupled ODE (ordinary differential equation)-PDE models, and prove their convergence to the optimum under mild assumptions. We establish sufficient bounds for local stability and convergence for the tuning parameter of this class of continuous analogues, the retraction parameter. To evaluate the continuous analogues, we study the parameter estimation for a model of gradient formation in biological tissues. We observe good convergence properties, indicating that the continuous analogues are an interesting alternative to state-of-the-art iterative optimization methods.

Predicting ligand-dependent tumors from multi-dimensional signaling features

Targeted therapies have shown significant patient benefit in about 5-10% of solid tumors that are addicted to a single oncogene. Here, we explore the idea of ligand addiction as a driver of tumor growth. High ligand levels in tumors have been shown to be associated with impaired patient survival, but targeted therapies have not yet shown great benefit in unselected patient populations. Using an approach of applying Bagged Decision Trees (BDT) to high-dimensional signaling features derived from a computational model, we can predict ligand dependent proliferation across a set of 58 cell lines. This mechanistic, multi-pathway model that features receptor heterodimerization, was trained on seven cancer cell lines and can predict signaling across two independent cell lines by adjusting only the receptor expression levels for each cell line. Interestingly, for patient samples the predicted tumor growth response correlates with high growth factor expression in the tumor microenvironment, which argues for a co-evolution of both factors in vivo.

Tailored parameter optimization methods for ordinary differential equation models with steady-state constraints

BACKGROUND

Ordinary differential equation (ODE) models are widely used to describe (bio-)chemical and biological processes. To enhance the predictive power of these models, their unknown parameters are estimated from experimental data. These experimental data are mostly collected in perturbation experiments, in which the processes are pushed out of steady state by applying a stimulus. The information that the initial condition is a steady state of the unperturbed process provides valuable information, as it restricts the dynamics of the process and thereby the parameters. However, implementing steady-state constraints in the optimization often results in convergence problems.

RESULTS

In this manuscript, we propose two new methods for solving optimization problems with steady-state constraints. The first method exploits ideas from optimization algorithms on manifolds and introduces a retraction operator, essentially reducing the dimension of the optimization problem. The second method is based on the continuous analogue of the optimization problem. This continuous analogue is an ODE whose equilibrium points are the optima of the constrained optimization problem. This equivalence enables the use of adaptive numerical methods for solving optimization problems with steady-state constraints. Both methods are tailored to the problem structure and exploit the local geometry of the steady-state manifold and its stability properties. A parameterization of the steady-state manifold is not required. The efficiency and reliability of the proposed methods is evaluated using one toy example and two applications. The first application example uses published data while the second uses a novel dataset for Raf/MEK/ERK signaling. The proposed methods demonstrated better convergence properties than state-of-the-art methods employed in systems and computational biology. Furthermore, the average computation time per converged start is significantly lower. In addition to the theoretical results, the analysis of the dataset for Raf/MEK/ERK signaling provides novel biological insights regarding the existence of feedback regulation.

CONCLUSION

Many optimization problems considered in systems and computational biology are subject to steady-state constraints. While most optimization methods have convergence problems if these steady-state constraints are highly nonlinear, the methods presented recover the convergence properties of optimizers which can exploit an analytical expression for the parameter-dependent steady state. This renders them an excellent alternative to methods which are currently employed in systems and computational biology.