Structural identifiability of systems biology models: a critical comparison of methods

Delineating parameter unidentifiabilities in complex models

Scientists use mathematical modeling as a tool for understanding and predicting the properties of complex physical systems. In highly parametrized models there often exist relationships between parameters over which model predictions are identical, or nearly identical. These are known as structural or practical unidentifiabilities, respectively. They are hard to diagnose and make reliable parameter estimation from data impossible. They furthermore imply the existence of an underlying model simplification. We describe a scalable method for detecting unidentifiabilities, as well as the functional relations defining them, for generic models. This allows for model simplification, and appreciation of which parameters (or functions thereof) cannot be estimated from data. Our algorithm can identify features such as redundant mechanisms and fast time-scale subsystems, as well as the regimes in parameter space over which such approximations are valid. We base our algorithm on a quantification of regional parametric sensitivity that we call 'multiscale sloppiness'. Traditionally, the link between parametric sensitivity and the conditioning of the parameter estimation problem is made locally, through the Fisher information matrix. This is valid in the regime of infinitesimal measurement uncertainty. We demonstrate the duality between multiscale sloppiness and the geometry of confidence regions surrounding parameter estimates made where measurement uncertainty is non-negligible. Further theoretical relationships are provided linking multiscale sloppiness to the likelihood-ratio test. From this, we show that a local sensitivity analysis (as typically done) is insufficient for determining the reliability of parameter estimation, even with simple (non)linear systems. Our algorithm can provide a tractable alternative. We finally apply our methods to a large-scale, benchmark systems biology model of necrosis factor (NF)-κB, uncovering unidentifiabilities.

Computational modeling of the cell-autonomous mammalian circadian oscillator

This review summarizes various mathematical models of cell-autonomous mammalian circadian clock. We present the basics necessary for understanding of the cell-autonomous mammalian circadian oscillator, modern experimental data essential for its reconstruction and some special problems related to the validation of mathematical circadian oscillator models. This work compares existing mathematical models of circadian oscillator and the results of the computational studies of the oscillating systems. Finally, we discuss applications of the mathematical models of mammalian circadian oscillator for solving specific problems in circadian rhythm biology.

Analysis of Practical Identifiability of a Viral Infection Model

Mathematical modelling approaches have granted a significant contribution to life sciences and beyond to understand experimental results. However, incomplete and inadequate assessments in parameter estimation practices hamper the parameter reliability, and consequently the insights that ultimately could arise from a mathematical model. To keep the diligent works in modelling biological systems from being mistrusted, potential sources of error must be acknowledged. Employing a popular mathematical model in viral infection research, existing means and practices in parameter estimation are exemplified. Numerical results show that poor experimental data is a main source that can lead to erroneous parameter estimates despite the use of innovative parameter estimation algorithms. Arbitrary choices of initial conditions as well as data asynchrony distort the parameter estimates but are often overlooked in modelling studies. This work stresses the existence of several sources of error buried in reports of modelling biological systems, voicing the need for assessing the sources of error, consolidating efforts in solving the immediate difficulties, and possibly reconsidering the use of mathematical modelling to quantify experimental data.

Structural identifiability of equilibrium ligand-binding parameters

Precise mathematical descriptions of ligand–protein interactions are hindered by the inability to experimentally measure affinity and cooperativity, although these parameters can be estimated from agonist binding models. Middendorf and Aldrich present a method to determine the accuracy of parameters estimated in this way.

Understanding the interactions of proteins with their ligands requires knowledge of molecular properties, such as binding site affinities and the effects that binding at one site exerts on binding at other sites (cooperativity). These properties cannot be measured directly and are usually estimated by fitting binding data with models that contain these quantities as parameters. In this study, we present a general method for answering the critical question of whether these parameters are identifiable (i.e., whether their estimates are accurate and unique). In cases in which parameter estimates are not unique, our analysis provides insight into the fundamental causes of nonidentifiability. This approach can thus serve as a guide for the proper design and analysis of protein–ligand binding experiments. We show that the equilibrium total binding relation can be reduced to a conserved mathematical form for all models composed solely of bimolecular association reactions and to a related, conserved form for all models composed of arbitrary combinations of binding and conformational equilibria. This canonical mathematical structure implies a universal parameterization of the binding relation that is consistent with virtually any physically reasonable binding model, for proteins with any number of binding sites. Matrix algebraic methods are used to prove that these universal parameter sets are structurally identifiable (SI; i.e., identifiable under conditions of noiseless data). A general approach for assessing and understanding the factors governing practical identifiability (i.e., the identifiability under conditions of real, noisy data) of these SI parameter sets is presented in the companion paper by Middendorf and Aldrich (2017. J. Gen. Physiol.https://doi.org/10.1085/jgp.201611703).

Estimation and Identifiability of Model Parameters in Human Nociceptive Processing Using Yes-No Detection Responses to Electrocutaneous Stimulation

Healthy or pathological states of nociceptive subsystems determine different stimulus-response relations measured from quantitative sensory testing. In turn, stimulus-response measurements may be used to assess these states. In a recently developed computational model, six model parameters characterize activation of nerve endings and spinal neurons. However, both model nonlinearity and limited information in yes-no detection responses to electrocutaneous stimuli challenge to estimate model parameters. Here, we address the question whether and how one can overcome these difficulties for reliable parameter estimation. First, we fit the computational model to experimental stimulus-response pairs by maximizing the likelihood. To evaluate the balance between model fit and complexity, i.e., the number of model parameters, we evaluate the Bayesian Information Criterion. We find that the computational model is better than a conventional logistic model regarding the balance. Second, our theoretical analysis suggests to vary the pulse width among applied stimuli as a necessary condition to prevent structural non-identifiability. In addition, the numerically implemented profile likelihood approach reveals structural and practical non-identifiability. Our model-based approach with integration of psychophysical measurements can be useful for a reliable assessment of states of the nociceptive system.

Structural Identifiability of Dynamic Systems Biology Models

A powerful way of gaining insight into biological systems is by creating a nonlinear differential equation model, which usually contains many unknown parameters. Such a model is called structurally identifiable if it is possible to determine the values of its parameters from measurements of the model outputs. Structural identifiability is a prerequisite for parameter estimation, and should be assessed before exploiting a model. However, this analysis is seldom performed due to the high computational cost involved in the necessary symbolic calculations, which quickly becomes prohibitive as the problem size increases. In this paper we show how to analyse the structural identifiability of a very general class of nonlinear models by extending methods originally developed for studying observability. We present results about models whose identifiability had not been previously determined, report unidentifiabilities that had not been found before, and show how to modify those unidentifiable models to make them identifiable. This method helps prevent problems caused by lack of identifiability analysis, which can compromise the success of tasks such as experiment design, parameter estimation, and model-based optimization. The procedure is called STRIKE-GOLDD (STRuctural Identifiability taKen as Extended-Generalized Observability with Lie Derivatives and Decomposition), and it is implemented in a MATLAB toolbox which is available as open source software. The broad applicability of this approach facilitates the analysis of the increasingly complex models used in systems biology and other areas.

Structural and Practical Identifiability Issues of Immuno-Epidemiological Vector-Host Models with Application to Rift Valley Fever

In this article, we discuss the structural and practical identifiability of a nested immuno-epidemiological model of arbovirus diseases, where host-vector transmission rate, host recovery, and disease-induced death rates are governed by the within-host immune system. We incorporate the newest ideas and the most up-to-date features of numerical methods to fit multi-scale models to multi-scale data. For an immunological model, we use Rift Valley Fever Virus (RVFV) time-series data obtained from livestock under laboratory experiments, and for an epidemiological model we incorporate a human compartment to the nested model and use the number of human RVFV cases reported by the CDC during the 2006-2007 Kenya outbreak. We show that the immunological model is not structurally identifiable for the measurements of time-series viremia concentrations in the host. Thus, we study the non-dimensionalized and scaled versions of the immunological model and prove that both are structurally globally identifiable. After fixing estimated parameter values for the immunological model derived from the scaled model, we develop a numerical method to fit observable RVFV epidemiological data to the nested model for the remaining parameter values of the multi-scale system. For the given (CDC) data set, Monte Carlo simulations indicate that only three parameters of the epidemiological model are practically identifiable when the immune model parameters are fixed. Alternatively, we fit the multi-scale data to the multi-scale model simultaneously. Monte Carlo simulations for the simultaneous fitting suggest that the parameters of the immunological model and the parameters of the immuno-epidemiological model are practically identifiable. We suggest that analytic approaches for studying the structural identifiability of nested models are a necessity, so that identifiable parameter combinations can be derived to reparameterize the nested model to obtain an identifiable one. This is a crucial step in developing multi-scale models which explain multi-scale data.

Current state and challenges for dynamic metabolic modeling

While the stoichiometry of metabolism is probably the best studied cellular level, the dynamics in metabolism can still not be well described, predicted and, thus, engineered. Unknowns in the metabolic flux behavior arise from kinetic interactions, especially allosteric control mechanisms. While the stoichiometry of enzymes is preserved in vitro, their activity and kinetic behavior differs from the in vivo situation. Next to this challenge, it is infeasible to test the interaction of each enzyme with each intracellular metabolite in vitro exhaustively. As a consequence, the whole interacting metabolome has to be studied in vivo to identify the relevant enzymes properties. In this review we discuss current approaches for in vivo perturbation experiments, that is, stimulus response experiments using different setups and quantitative analytical approaches, including dynamic carbon tracing. Next to reliable and informative data, advanced modeling approaches and computational tools are required to identify kinetic mechanisms and their parameters.

Inference for Stochastic Chemical Kinetics Using Moment Equations and System Size Expansion

Quantitative mechanistic models are valuable tools for disentangling biochemical pathways and for achieving a comprehensive understanding of biological systems. However, to be quantitative the parameters of these models have to be estimated from experimental data. In the presence of significant stochastic fluctuations this is a challenging task as stochastic simulations are usually too time-consuming and a macroscopic description using reaction rate equations (RREs) is no longer accurate. In this manuscript, we therefore consider moment-closure approximation (MA) and the system size expansion (SSE), which approximate the statistical moments of stochastic processes and tend to be more precise than macroscopic descriptions. We introduce gradient-based parameter optimization methods and uncertainty analysis methods for MA and SSE. Efficiency and reliability of the methods are assessed using simulation examples as well as by an application to data for Epo-induced JAK/STAT signaling. The application revealed that even if merely population-average data are available, MA and SSE improve parameter identifiability in comparison to RRE. Furthermore, the simulation examples revealed that the resulting estimates are more reliable for an intermediate volume regime. In this regime the estimation error is reduced and we propose methods to determine the regime boundaries. These results illustrate that inference using MA and SSE is feasible and possesses a high sensitivity.

In this manuscript, we introduce efficient methods for parameter estimation for stochastic processes. The stochasticity of chemical reactions can influence the average behavior of the considered system. For some biological systems, a microscopic, stochastic description is computationally intractable but a macroscopic, deterministic description too inaccurate. This inaccuracy manifests itself in an error in parameter estimates, which impede the predictive power of the proposed model. Until now, no rigorous analysis on the magnitude of the estimation error exists. We show by means of two simulation examples that using mesoscopic descriptions based on the system size expansions and moment-closure approximations can reduce this estimation error compared to inference using a macroscopic description. This reduction is most pronounced in an intermediate volume regime where the influence of stochasticity on the average behavior is moderately strong. For the JAK/STAT pathway where experimental data is available, we show that one parameter that was not structurally identifiable when using a macroscopic description becomes structurally identifiable when using a mesoscopic description for parameter estimation.

Parameter redundancy in discrete state-space and integrated models

Discrete state-space models are used in ecology to describe the dynamics of wild animal populations, with parameters, such as the probability of survival, being of ecological interest. For a particular parametrization of a model it is not always clear which parameters can be estimated. This inability to estimate all parameters is known as parameter redundancy or a model is described as nonidentifiable. In this paper we develop methods that can be used to detect parameter redundancy in discrete state-space models. An exhaustive summary is a combination of parameters that fully specify a model. To use general methods for detecting parameter redundancy a suitable exhaustive summary is required. This paper proposes two methods for the derivation of an exhaustive summary for discrete state-space models using discrete analogues of methods for continuous state-space models. We also demonstrate that combining multiple data sets, through the use of an integrated population model, may result in a model in which all parameters are estimable, even though models fitted to the separate data sets may be parameter redundant.

Structural identifiability of cyclic graphical models of biological networks with latent variables

Background

Graphical models have long been used to describe biological networks for a variety of important tasks such as the determination of key biological parameters, and the structure of graphical model ultimately determines whether such unknown parameters can be unambiguously obtained from experimental observations (i.e., the identifiability problem). Limited by resources or technical capacities, complex biological networks are usually partially observed in experiment, which thus introduces latent variables into the corresponding graphical models. A number of previous studies have tackled the parameter identifiability problem for graphical models such as linear structural equation models (SEMs) with or without latent variables. However, the limited resolution and efficiency of existing approaches necessarily calls for further development of novel structural identifiability analysis algorithms.

Results

An efficient structural identifiability analysis algorithm is developed in this study for a broad range of network structures. The proposed method adopts the Wright’s path coefficient method to generate identifiability equations in forms of symbolic polynomials, and then converts these symbolic equations to binary matrices (called identifiability matrix). Several matrix operations are introduced for identifiability matrix reduction with system equivalency maintained. Based on the reduced identifiability matrices, the structural identifiability of each parameter is determined. A number of benchmark models are used to verify the validity of the proposed approach. Finally, the network module for influenza A virus replication is employed as a real example to illustrate the application of the proposed approach in practice.

Conclusions

The proposed approach can deal with cyclic networks with latent variables. The key advantage is that it intentionally avoids symbolic computation and is thus highly efficient. Also, this method is capable of determining the identifiability of each single parameter and is thus of higher resolution in comparison with many existing approaches. Overall, this study provides a basis for systematic examination and refinement of graphical models of biological networks from the identifiability point of view, and it has a significant potential to be extended to more complex network structures or high-dimensional systems.

Electronic supplementary material

The online version of this article (doi:10.1186/s12918-016-0287-y) contains supplementary material, which is available to authorized users.

A simple method for identifying parameter correlations in partially observed linear dynamic models

Background

Parameter estimation represents one of the most significant challenges in systems biology. This is because biological models commonly contain a large number of parameters among which there may be functional interrelationships, thus leading to the problem of non-identifiability. Although identifiability analysis has been extensively studied by analytical as well as numerical approaches, systematic methods for remedying practically non-identifiable models have rarely been investigated.

Results

We propose a simple method for identifying pairwise correlations and higher order interrelationships of parameters in partially observed linear dynamic models. This is made by derivation of the output sensitivity matrix and analysis of the linear dependencies of its columns. Consequently, analytical relations between the identifiability of the model parameters and the initial conditions as well as the input functions can be achieved. In the case of structural non-identifiability, identifiable combinations can be obtained by solving the resulting homogenous linear equations. In the case of practical non-identifiability, experiment conditions (i.e. initial condition and constant control signals) can be provided which are necessary for remedying the non-identifiability and unique parameter estimation. It is noted that the approach does not consider noisy data. In this way, the practical non-identifiability issue, which is popular for linear biological models, can be remedied. Several linear compartment models including an insulin receptor dynamics model are taken to illustrate the application of the proposed approach.

Conclusions

Both structural and practical identifiability of partially observed linear dynamic models can be clarified by the proposed method. The result of this method provides important information for experimental design to remedy the practical non-identifiability if applicable. The derivation of the method is straightforward and thus the algorithm can be easily implemented into a software packet.

Electronic supplementary material

The online version of this article (doi:10.1186/s12918-015-0234-3) contains supplementary material, which is available to authorized users.

Modeling Influenza Virus Infection: A Roadmap for Influenza Research

Influenza A virus (IAV) infection represents a global threat causing seasonal outbreaks and pandemics. Additionally, secondary bacterial infections, caused mainly by Streptococcus pneumoniae, are one of the main complications and responsible for the enhanced morbidity and mortality associated with IAV infections. In spite of the significant advances in our knowledge of IAV infections, holistic comprehension of the interplay between IAV and the host immune response (IR) remains largely fragmented. During the last decade, mathematical modeling has been instrumental to explain and quantify IAV dynamics. In this paper, we review not only the state of the art of mathematical models of IAV infection but also the methodologies exploited for parameter estimation. We focus on the adaptive IR control of IAV infection and the possible mechanisms that could promote a secondary bacterial coinfection. To exemplify IAV dynamics and identifiability issues, a mathematical model to explain the interactions between adaptive IR and IAV infection is considered. Furthermore, in this paper we propose a roadmap for future influenza research. The development of a mathematical modeling framework with a secondary bacterial coinfection, immunosenescence, host genetic factors and responsiveness to vaccination will be pivotal to advance IAV infection understanding and treatment optimization.

Clustering reveals limits of parameter identifiability in multi-parameter models of biochemical dynamics

Background

Compared to engineering or physics problems, dynamical models in quantitative biology typically depend on a relatively large number of parameters. Progress in developing mathematics to manipulate such multi-parameter models and so enable their efficient interplay with experiments has been slow. Existing solutions are significantly limited by model size.

Results

In order to simplify analysis of multi-parameter models a method for clustering of model parameters is proposed. It is based on a derived statistically meaningful measure of similarity between groups of parameters. The measure quantifies to what extend changes in values of some parameters can be compensated by changes in values of other parameters. The proposed methodology provides a natural mathematical language to precisely communicate and visualise effects resulting from compensatory changes in values of parameters. As a results, a relevant insight into identifiability analysis and experimental planning can be obtained. Analysis of NF- κB and MAPK pathway models shows that highly compensative parameters constitute clusters consistent with the network topology. The method applied to examine an exceptionally rich set of published experiments on the NF- κB dynamics reveals that the experiments jointly ensure identifiability of only 60 % of model parameters. The method indicates which further experiments should be performed in order to increase the number of identifiable parameters.

Conclusions

We currently lack methods that simplify broadly understood analysis of multi-parameter models. The introduced tools depict mutually compensative effects between parameters to provide insight regarding role of individual parameters, identifiability and experimental design. The method can also find applications in related methodological areas of model simplification and parameters estimation.

Electronic supplementary material

The online version of this article (doi:10.1186/s12918-015-0205-8) contains supplementary material, which is available to authorized users.

Development of a robust DNA damage model including persistent telomere-associated damage with application to secondary cancer risk assessment

Mathematical modelling has been instrumental to understand kinetics of radiation-induced DNA damage repair and associated secondary cancer risk. The widely accepted two-lesion kinetic (TLK) model assumes two kinds of double strand breaks, simple and complex ones, with different repair rates. Recently, persistent DNA damage associated with telomeres was reported as a new kind of DNA damage. We therefore extended existing versions of the TLK model by new categories of DNA damage and re-evaluated those models using extensive data. We subjected different versions of the TLK model to a rigorous model discrimination approach. This enabled us to robustly select a best approximating parsimonious model that can both recapitulate and predict transient and persistent DNA damage after ionizing radiation. Models and data argue for i) nonlinear dose-damage relationships, and ii) negligible saturation of repair kinetics even for high doses. Additionally, we show that simulated radiation-induced persistent telomere-associated DNA damage foci (TAF) can be used to predict excess relative risk (ERR) of developing secondary leukemia after fractionated radiotherapy. We suggest that TAF may serve as an additional measure to predict cancer risk after radiotherapy using high dose rates. This may improve predicting risk-dose dependency of ionizing radiation especially for long-term therapies.

A Priori Identifiability of Target-Mediated Drug Disposition Models and Approximations

A priori identifiability of mathematical models assures that for a given input/output experiment, the parameter set has one unique solution within a defined space, independent of the experimental design. Many biologic therapeutics exhibit target-mediated drug disposition (TMDD), and use of the full compartmental model describing this system is well documented. In practice, estimation of the full parameter set for TMDD models, given real-world clinical data, is characterized by convergence difficulties and unstable solutions. Still, the formal assessment of the a priori identifiability of these systems has yet to be reported. The exact arithmetic rank (EAR) approach was used to test the a priori identifiability of a TMDD model as well as model approximations. The full TMDD and quasi-equilibrium/rapid binding (QE/RB), quasi-steady state (QSS), and Michaelis-Menten (MM) approximations were fully identifiable, a priori, regardless of whether observations were taken from a single or multiple compartments. The results of these identifiability analyses indicated that the difficulty with TMDD model convergence, a posteriori, lies in the experimental design, not in the mathematical identifiability in the lack of samples from several compartments. Experiments can be tailored to resolve these structurally non-identifiable parameters, notwithstanding practical implementation challenges. This work highlights the importance of identifiability analyses, specifically how they can influence experimental design and selection of the appropriate model structure to describe a dynamic biological system.

Image based validation of dynamical models for cell reorientation

A key feature of directed cell movement is the ability of cells to reorient quickly in response to changes in the direction of an extracellular stimulus. Mathematical models have suggested quite different regulatory mechanisms to explain reorientation, raising the question of how we can validate these models in a rigorous way. In this study, we fit three reaction-diffusion models to experimental data of Dictyostelium amoebae reorienting in response to alternating gradients of mechanical shear flow. The experimental readouts we use to fit are spatio-temporal distributions of a fluorescent reporter for cortical F-actin labeling the cell front. Experiments performed under different conditions are fitted simultaneously to challenge the models with different types of cellular dynamics. Although the model proposed by Otsuji is unable to provide a satisfactory fit, those suggested by Meinhardt and Levchenko fit equally well. Further, we show that reduction of the three-variable Meinhardt model to a two-variable model also provides an excellent fit, but has the advantage of all parameters being uniquely identifiable. Our work demonstrates that model selection and identifiability analysis, commonly applied to temporal dynamics problems in systems biology, can be a powerful tool when extended to spatio-temporal imaging data.

Summary of the DREAM8 Parameter Estimation Challenge: Toward Parameter Identification for Whole-Cell Models

Whole-cell models that explicitly represent all cellular components at the molecular level have the potential to predict phenotype from genotype. However, even for simple bacteria, whole-cell models will contain thousands of parameters, many of which are poorly characterized or unknown. New algorithms are needed to estimate these parameters and enable researchers to build increasingly comprehensive models. We organized the Dialogue for Reverse Engineering Assessments and Methods (DREAM) 8 Whole-Cell Parameter Estimation Challenge to develop new parameter estimation algorithms for whole-cell models. We asked participants to identify a subset of parameters of a whole-cell model given the model’s structure and in silico “experimental” data. Here we describe the challenge, the best performing methods, and new insights into the identifiability of whole-cell models. We also describe several valuable lessons we learned toward improving future challenges. Going forward, we believe that collaborative efforts supported by inexpensive cloud computing have the potential to solve whole-cell model parameter estimation.

Whole-cell models promise to enable rational bioengineering by predicting how cells behave. Even for simple bacteria, whole-cell models require thousands of parameters, many of which are poorly characterized or unknown. New approaches are needed to estimate these parameters. We organized the Dialogue for Reverse Engineering Assessments and Methods (DREAM) 8 Whole-Cell Parameter Estimation Challenge to develop new approaches for whole-cell model parameter identification. Here we describe the challenge, the best performing methods, new insights into the identifiability of whole-cell models, and several lessons we learned for improving future challenges. Going forward, we believe that collaborative efforts have the potential to produce powerful tools for identifying whole-cell models.

A unified framework for estimating parameters of kinetic biological models

Background

Utilizing kinetic models of biological systems commonly require computational approaches to estimate parameters, posing a variety of challenges due to their highly non-linear and dynamic nature, which is further complicated by the issue of non-identifiability. We propose a novel parameter estimation framework by combining approaches for solving identifiability with a recently introduced filtering technique that can uniquely estimate parameters where conventional methods fail. This framework first conducts a thorough analysis to identify and classify the non-identifiable parameters and provides a guideline for solving them. If no feasible solution can be found, the framework instead initializes the filtering technique with informed prior to yield a unique solution.

Results

This framework has been applied to uniquely estimate parameter values for the sucrose accumulation model in sugarcane culm tissue and a gene regulatory network. In the first experiment the results show the progression of improvement in reliable and unique parameter estimation through the use of each tool to reduce and remove non-identifiability. The latter experiment illustrates the common situation where no further measurement data is available to solve the non-identifiability. These results show the successful application of the informed prior as well as the ease with which parallel data sources may be utilized without increasing the model complexity.

Conclusion

The proposed unified framework is distinct from other approaches by providing a robust and complete solution which yields reliable and unique parameter estimation even in the face of non-identifiability.

Electronic supplementary material

The online version of this article (doi:10.1186/s12859-015-0500-9) contains supplementary material, which is available to authorized users.

BioPreDyn-bench: a suite of benchmark problems for dynamic modelling in systems biology

Background

Dynamic modelling is one of the cornerstones of systems biology. Many research efforts are currently being invested in the development and exploitation of large-scale kinetic models. The associated problems of parameter estimation (model calibration) and optimal experimental design are particularly challenging. The community has already developed many methods and software packages which aim to facilitate these tasks. However, there is a lack of suitable benchmark problems which allow a fair and systematic evaluation and comparison of these contributions.

Results

Here we present BioPreDyn-bench, a set of challenging parameter estimation problems which aspire to serve as reference test cases in this area. This set comprises six problems including medium and large-scale kinetic models of the bacterium E. coli, baker’s yeast S. cerevisiae, the vinegar fly D. melanogaster, Chinese Hamster Ovary cells, and a generic signal transduction network. The level of description includes metabolism, transcription, signal transduction, and development. For each problem we provide (i) a basic description and formulation, (ii) implementations ready-to-run in several formats, (iii) computational results obtained with specific solvers, (iv) a basic analysis and interpretation.

Conclusions

This suite of benchmark problems can be readily used to evaluate and compare parameter estimation methods. Further, it can also be used to build test problems for sensitivity and identifiability analysis, model reduction and optimal experimental design methods. The suite, including codes and documentation, can be freely downloaded from the BioPreDyn-bench website, https://sites.google.com/site/biopredynbenchmarks/.

Electronic supplementary material

The online version of this article (doi:10.1186/s12918-015-0144-4) contains supplementary material, which is available to authorized users.

A novel protocol for model calibration in biological wastewater treatment

Activated sludge models (ASMs) have been widely used for process design, operation and optimization in wastewater treatment plants. However, it is still a challenge to achieve an efficient calibration for reliable application by using the conventional approaches. Hereby, we propose a novel calibration protocol, i.e. Numerical Optimal Approaching Procedure (NOAP), for the systematic calibration of ASMs. The NOAP consists of three key steps in an iterative scheme flow: i) global factors sensitivity analysis for factors fixing; ii) pseudo-global parameter correlation analysis for non-identifiable factors detection; and iii) formation of a parameter subset through an estimation by using genetic algorithm. The validity and applicability are confirmed using experimental data obtained from two independent wastewater treatment systems, including a sequencing batch reactor and a continuous stirred-tank reactor. The results indicate that the NOAP can effectively determine the optimal parameter subset and successfully perform model calibration and validation for these two different systems. The proposed NOAP is expected to use for automatic calibration of ASMs and be applied potentially to other ordinary differential equations models.

Analysis of passive cardiac constitutive laws for parameter estimation using 3D tagged MRI

An unresolved issue in patient-specific models of cardiac mechanics is the choice of an appropriate constitutive law, able to accurately capture the passive behavior of the myocardium, while still having uniquely identifiable parameters tunable from available clinical data. In this paper, we aim to facilitate this choice by examining the practical identifiability and model fidelity of constitutive laws often used in cardiac mechanics. Our analysis focuses on the use of novel 3D tagged MRI, providing detailed displacement information in three dimensions. The practical identifiability of each law is examined by generating synthetic 3D tags from in silico simulations, allowing mapping of the objective function landscape over parameter space and comparison of minimizing parameter values with original ground truth values. Model fidelity was tested by comparing these laws with the more complex transversely isotropic Guccione law, by characterizing their passive end-diastolic pressure–volume relation behavior, as well as by considering the in vivo case of a healthy volunteer. These results show that a reduced form of the Holzapfel–Ogden law provides the best balance between identifiability and model fidelity across the tests considered.

Estimating HIV-1 fitness characteristics from cross-sectional genotype data

Despite the success of highly active antiretroviral therapy (HAART) in the management of human immunodeficiency virus (HIV)-1 infection, virological failure due to drug resistance development remains a major challenge. Resistant mutants display reduced drug susceptibilities, but in the absence of drug, they generally have a lower fitness than the wild type, owing to a mutation-incurred cost. The interaction between these fitness costs and drug resistance dictates the appearance of mutants and influences viral suppression and therapeutic success. Assessing in vivo viral fitness is a challenging task and yet one that has significant clinical relevance. Here, we present a new computational modelling approach for estimating viral fitness that relies on common sparse cross-sectional clinical data by combining statistical approaches to learn drug-specific mutational pathways and resistance factors with viral dynamics models to represent the host-virus interaction and actions of drug mechanistically. We estimate in vivo fitness characteristics of mutant genotypes for two antiretroviral drugs, the reverse transcriptase inhibitor zidovudine (ZDV) and the protease inhibitor indinavir (IDV). Well-known features of HIV-1 fitness landscapes are recovered, both in the absence and presence of drugs. We quantify the complex interplay between fitness costs and resistance by computing selective advantages for different mutants. Our approach extends naturally to multiple drugs and we illustrate this by simulating a dual therapy with ZDV and IDV to assess therapy failure. The combined statistical and dynamical modelling approach may help in dissecting the effects of fitness costs and resistance with the ultimate aim of assisting the choice of salvage therapies after treatment failure.

Mutations conferring drug resistance represent major threats to the therapeutic success of highly active antiretroviral therapy (HAART) against human immunodeficiency virus (HIV)-1 infection. Viral mutants differ in their fitness and assessing viral fitness is a challenging task. In this article, we estimate drug-specific mutational pathways by learning from clinical data using statistical techniques and incorporate these into mathematical models of in vivo viral infection dynamics. This approach enables us to estimate mutant fitness characteristics. We illustrate our method by predicting fitness characteristics of mutant genotypes for two different antiretroviral therapies with the drugs zidovudine and indinavir. We recover several established features of mutant fitnesses and quantify fitness characteristics both in the absence and presence of drugs. Our model extends naturally to multiple drugs and we illustrate this by simulating a dual therapy with ZDV and IDV to assess therapy failure. Additionally, our modelling approach relies only on cross-sectional clinical data. We believe that such an approach is a highly valuable tool in assisting the choice of salvage therapies after treatment failure.

On finding and using identifiable parameter combinations in nonlinear dynamic systems biology models and COMBOS: a novel web implementation

Parameter identifiability problems can plague biomodelers when they reach the quantification stage of development, even for relatively simple models. Structural identifiability (SI) is the primary question, usually understood as knowing which of P unknown biomodel parameters p1,…, pi,…, pP are-and which are not-quantifiable in principle from particular input-output (I-O) biodata. It is not widely appreciated that the same database also can provide quantitative information about the structurally unidentifiable (not quantifiable) subset, in the form of explicit algebraic relationships among unidentifiable pi. Importantly, this is a first step toward finding what else is needed to quantify particular unidentifiable parameters of interest from new I-O experiments. We further develop, implement and exemplify novel algorithms that address and solve the SI problem for a practical class of ordinary differential equation (ODE) systems biology models, as a user-friendly and universally-accessible web application (app)-COMBOS. Users provide the structural ODE and output measurement models in one of two standard forms to a remote server via their web browser. COMBOS provides a list of uniquely and non-uniquely SI model parameters, and-importantly-the combinations of parameters not individually SI. If non-uniquely SI, it also provides the maximum number of different solutions, with important practical implications. The behind-the-scenes symbolic differential algebra algorithms are based on computing Gröbner bases of model attributes established after some algebraic transformations, using the computer-algebra system Maxima. COMBOS was developed for facile instructional and research use as well as modeling. We use it in the classroom to illustrate SI analysis; and have simplified complex models of tumor suppressor p53 and hormone regulation, based on explicit computation of parameter combinations. It's illustrated and validated here for models of moderate complexity, with and without initial conditions. Built-in examples include unidentifiable 2 to 4-compartment and HIV dynamics models.

The inverse problem in mathematical biology

Biological systems present particular challengers to model for the purposes of formulating predictions of generating biological insight. These systems are typically multi-scale, complex, and empirical observations are often sparse and subject to variability and uncertainty. This manuscript will review some of these specific challenges and introduce current methods used by modelers to construct meaningful solutions, in the context of preserving biological relevance. Opportunities to expand these methods are also discussed.

Determining identifiable parameter combinations using subset profiling

Identifiability is a necessary condition for successful parameter estimation of dynamic system models. A major component of identifiability analysis is determining the identifiable parameter combinations, the functional forms for the dependencies between unidentifiable parameters. Identifiable combinations can help in model reparameterization and also in determining which parameters may be experimentally measured to recover model identifiability. Several numerical approaches to determining identifiability of differential equation models have been developed, however the question of determining identifiable combinations remains incompletely addressed. In this paper, we present a new approach which uses parameter subset selection methods based on the Fisher Information Matrix, together with the profile likelihood, to effectively estimate identifiable combinations. We demonstrate this approach on several example models in pharmacokinetics, cellular biology, and physiology.

Ensemble inference and inferability of gene regulatory networks

The inference of gene regulatory network (GRN) from gene expression data is an unsolved problem of great importance. This inference has been stated, though not proven, to be underdetermined implying that there could be many equivalent (indistinguishable) solutions. Motivated by this fundamental limitation, we have developed new framework and algorithm, called TRaCE, for the ensemble inference of GRNs. The ensemble corresponds to the inherent uncertainty associated with discriminating direct and indirect gene regulations from steady-state data of gene knock-out (KO) experiments. We applied TRaCE to analyze the inferability of random GRNs and the GRNs of E. coli and yeast from single- and double-gene KO experiments. The results showed that, with the exception of networks with very few edges, GRNs are typically not inferable even when the data are ideal (unbiased and noise-free). Finally, we compared the performance of TRaCE with top performing methods of DREAM4 in silico network inference challenge.

On linear models and parameter identifiability in experimental biological systems

A key problem in the biological sciences is to be able to reliably estimate model parameters from experimental data. This is the well-known problem of parameter identifiability. Here, methods are developed for biologists and other modelers to design optimal experiments to ensure parameter identifiability at a structural level. The main results of the paper are to provide a general methodology for extracting parameters of linear models from an experimentally measured scalar function - the transfer function - and a framework for the identifiability analysis of complex model structures using linked models. Linked models are composed by letting the output of one model become the input to another model which is then experimentally measured. The linked model framework is shown to be applicable to designing experiments to identify the measured sub-model and recover the input from the unmeasured sub-model, even in cases that the unmeasured sub-model is not identifiable. Applications for a set of common model features are demonstrated, and the results combined in an example application to a real-world experimental system. These applications emphasize the insight into answering "where to measure" and "which experimental scheme" questions provided by both the parameter extraction methodology and the linked model framework. The aim is to demonstrate the tools' usefulness in guiding experimental design to maximize parameter information obtained, based on the model structure.

Structural identifiability analyses of candidate models for in vitro Pitavastatin hepatic uptake

In this paper a review of the application of four different techniques (a version of the similarity transformation approach for autonomous uncontrolled systems, a non-differential input/output observable normal form approach, the characteristic set differential algebra and a recent algebraic input/output relationship approach) to determine the structural identifiability of certain in vitro nonlinear pharmacokinetic models is provided. The Organic Anion Transporting Polypeptide (OATP) substrate, Pitavastatin, is used as a probe on freshly isolated animal and human hepatocytes. Candidate pharmacokinetic non-linear compartmental models have been derived to characterise the uptake process of Pitavastatin. As a prerequisite to parameter estimation, structural identifiability analyses are performed to establish that all unknown parameters can be identified from the experimental observations available.

Kinetic models in industrial biotechnology - Improving cell factory performance

An increasing number of industrial bioprocesses capitalize on living cells by using them as cell factories that convert sugars into chemicals. These processes range from the production of bulk chemicals in yeasts and bacteria to the synthesis of therapeutic proteins in mammalian cell lines. One of the tools in the continuous search for improved performance of such production systems is the development and application of mathematical models. To be of value for industrial biotechnology, mathematical models should be able to assist in the rational design of cell factory properties or in the production processes in which they are utilized. Kinetic models are particularly suitable towards this end because they are capable of representing the complex biochemistry of cells in a more complete way compared to most other types of models. They can, at least in principle, be used to in detail understand, predict, and evaluate the effects of adding, removing, or modifying molecular components of a cell factory and for supporting the design of the bioreactor or fermentation process. However, several challenges still remain before kinetic modeling will reach the degree of maturity required for routine application in industry. Here we review the current status of kinetic cell factory modeling. Emphasis is on modeling methodology concepts, including model network structure, kinetic rate expressions, parameter estimation, optimization methods, identifiability analysis, model reduction, and model validation, but several applications of kinetic models for the improvement of cell factories are also discussed.

cuTauLeaping: a GPU-powered tau-leaping stochastic simulator for massive parallel analyses of biological systems

Tau-leaping is a stochastic simulation algorithm that efficiently reconstructs the temporal evolution of biological systems, modeled according to the stochastic formulation of chemical kinetics. The analysis of dynamical properties of these systems in physiological and perturbed conditions usually requires the execution of a large number of simulations, leading to high computational costs. Since each simulation can be executed independently from the others, a massive parallelization of tau-leaping can bring to relevant reductions of the overall running time. The emerging field of General Purpose Graphic Processing Units (GPGPU) provides power-efficient high-performance computing at a relatively low cost. In this work we introduce cuTauLeaping, a stochastic simulator of biological systems that makes use of GPGPU computing to execute multiple parallel tau-leaping simulations, by fully exploiting the Nvidia's Fermi GPU architecture. We show how a considerable computational speedup is achieved on GPU by partitioning the execution of tau-leaping into multiple separated phases, and we describe how to avoid some implementation pitfalls related to the scarcity of memory resources on the GPU streaming multiprocessors. Our results show that cuTauLeaping largely outperforms the CPU-based tau-leaping implementation when the number of parallel simulations increases, with a break-even directly depending on the size of the biological system and on the complexity of its emergent dynamics. In particular, cuTauLeaping is exploited to investigate the probability distribution of bistable states in the Schlögl model, and to carry out a bidimensional parameter sweep analysis to study the oscillatory regimes in the Ras/cAMP/PKA pathway in S. cerevisiae.

Advancing metabolic models with kinetic information

Kinetic models are crucial to quantitatively understand and predict how functional behavior emerges from dynamic concentration changes of cellular components. The current challenge is on resolving uncertainties about parameter values of reaction kinetics. Additionally, there are also major structural uncertainties due to unknown molecular interactions and only putatively assigned regulatory functions. What if one or few key regulators of biochemical reactions are missing in a metabolic model? By reviewing current advances in building kinetic models of metabolism, we found that such models experience a paradigm shift away from fitting parameters towards identifying key regulatory interactions.

Structural identifiability of viscoelastic mechanical systems

We solve the local and global structural identifiability problems for viscoelastic mechanical models represented by networks of springs and dashpots. We propose a very simple characterization of both local and global structural identifiability based on identifiability tables, with the purpose of providing a guideline for constructing arbitrarily complex, identifiable spring-dashpot networks. We illustrate how to use our results in a number of examples and point to some applications in cardiovascular modeling.

Determination of parameter identifiability in nonlinear biophysical models: A Bayesian approach

A major goal of biophysics is to understand the physical mechanisms of biological molecules and systems. Mechanistic models are evaluated based on their ability to explain carefully controlled experiments. By fitting models to data, biophysical parameters that cannot be measured directly can be estimated from experimentation. However, it might be the case that many different combinations of model parameters can explain the observations equally well. In these cases, the model parameters are not identifiable: the experimentation has not provided sufficient constraining power to enable unique estimation of their true values. We demonstrate that this pitfall is present even in simple biophysical models. We investigate the underlying causes of parameter non-identifiability and discuss straightforward methods for determining when parameters of simple models can be inferred accurately. However, for models of even modest complexity, more general tools are required to diagnose parameter non-identifiability. We present a method based in Bayesian inference that can be used to establish the reliability of parameter estimates, as well as yield accurate quantification of parameter confidence.

Reverse engineering and identification in systems biology: strategies, perspectives and challenges

The interplay of mathematical modelling with experiments is one of the central elements in systems biology. The aim of reverse engineering is to infer, analyse and understand, through this interplay, the functional and regulatory mechanisms of biological systems. Reverse engineering is not exclusive of systems biology and has been studied in different areas, such as inverse problem theory, machine learning, nonlinear physics, (bio)chemical kinetics, control theory and optimization, among others. However, it seems that many of these areas have been relatively closed to outsiders. In this contribution, we aim to compare and highlight the different perspectives and contributions from these fields, with emphasis on two key questions: (i) why are reverse engineering problems so hard to solve, and (ii) what methods are available for the particular problems arising from systems biology?

Insights into the mechanisms of thymus involution and regeneration by modeling the glucocorticoid-induced perturbation of thymocyte populations dynamics

T-cells develop in the thymus and based on CD4 and CD8 expressions there are four main thymocyte populations in a normal mouse thymus. Currently, there are several mathematical models that describe the dynamics of thymocyte populations in a normal thymus, but only a few of them model the transient perturbation of their homeostasis. Our aim is to model the perturbation in the dynamics of each thymocyte population which is induced by the administration of a glucocorticoid, i.e. dexamethasone. The proposed approach relies on extending a four compartment thymus model based on differential equations by adding perturbation terms either globally (at the level of each equation) or locally (at the level of proliferation, death, and transfer rates). By fitting the perturbed model with experimental data on mice thymi collected before and after the administration of dexamethasone, it was possible to estimate the relevant parameters using a population-based stochastic search method. The fitted model is further used to conduct a quantitative analysis on the differentiated impact of dexamethasone on each T-cell population and on proliferation, death, and transfer processes. The obtained quantitative information on the perturbation could be used to explore and modify the flow of thymocytes between thymus compartments in order to elucidate the mechanisms of thymus involution and its subsequent regeneration. Since glucocorticoids are raised in many pathological situations, such a model could be useful in evaluating the impact of diseases on thymocyte dynamics in the thymus.

Identification of regulatory structure and kinetic parameters of biochemical networks via mixed-integer dynamic optimization

Background

Recovering the network topology and associated kinetic parameter values from time-series data are central topics in systems biology. Nevertheless, methods that simultaneously do both are few and lack generality.

Results

Here, we present a rigorous approach for simultaneously estimating the parameters and regulatory topology of biochemical networks from time-series data. The parameter estimation task is formulated as a mixed-integer dynamic optimization problem with: (i) binary variables, used to model the existence of regulatory interactions and kinetic effects of metabolites in the network processes; and (ii) continuous variables, denoting metabolites concentrations and kinetic parameters values. The approach simultaneously optimizes the Akaike criterion, which captures the trade-off between complexity (measured by the number of parameters), and accuracy of the fitting. This simultaneous optimization mitigates a possible overfitting that could result from addition of spurious regulatory interactions.

Conclusion

The capabilities of our approach were tested in one benchmark problem. Our algorithm is able to identify a set of plausible network topologies with their associated parameters.

Reverse-engineering post-transcriptional regulation of gap genes in Drosophila melanogaster

Systems biology proceeds through repeated cycles of experiment and modeling. One way to implement this is reverse engineering, where models are fit to data to infer and analyse regulatory mechanisms. This requires rigorous methods to determine whether model parameters can be properly identified. Applying such methods in a complex biological context remains challenging. We use reverse engineering to study post-transcriptional regulation in pattern formation. As a case study, we analyse expression of the gap genes Krüppel, knirps, and giant in Drosophila melanogaster. We use detailed, quantitative datasets of gap gene mRNA and protein expression to solve and fit a model of post-transcriptional regulation, and establish its structural and practical identifiability. Our results demonstrate that post-transcriptional regulation is not required for patterning in this system, but is necessary for proper control of protein levels. Our work demonstrates that the uniqueness and specificity of a fitted model can be rigorously determined in the context of spatio-temporal pattern formation. This greatly increases the potential of reverse engineering for the study of development and other, similarly complex, biological processes.

The analysis of pattern-forming gene networks is largely focussed on transcriptional regulation. However, post-transcriptional events, such as translation and regulation of protein stability also play important roles in the establishment of protein expression patterns and levels. In this study, we use a reverse-engineering approach—fitting mathematical models to quantitative expression data—to analyse post-transcriptional regulation of the Drosophila gap genes Krüppel, knirps and giant, involved in segment determination during early embryogenesis. Rigorous fitting requires us to establish whether our models provide a robust and unique solution. We demonstrate, for the first time, that this can be done in the context of a complex spatio-temporal regulatory system. This is an important methodological advance for reverse-engineering developmental processes. Our results indicate that post-transcriptional regulation is not required for pattern formation, but is necessary for proper regulation of gap protein levels. Specifically, we predict that translation rates must be tuned for rapid early accumulation, and protein stability must be increased for persistence of high protein levels at late stages of gap gene expression.

Identification of parameter correlations for parameter estimation in dynamic biological models

Background

One of the challenging tasks in systems biology is parameter estimation in nonlinear dynamic models. A biological model usually contains a large number of correlated parameters leading to non-identifiability problems. Although many approaches have been developed to address both structural and practical non-identifiability problems, very few studies have been made to systematically investigate parameter correlations.

Results

In this study we present an approach that is able to identify both pairwise parameter correlations and higher order interrelationships among parameters in nonlinear dynamic models. Correlations are interpreted as surfaces in the subspaces of correlated parameters. Based on the correlation information obtained in this way both structural and practical non-identifiability can be clarified. Moreover, it can be concluded from the correlation analysis that a minimum number of data sets with different inputs for experimental design are needed to relieve the parameter correlations, which corresponds to the maximum number of correlated parameters among the correlation groups.

Conclusions

The information of pairwise and higher order interrelationships among parameters in biological models gives a deeper insight into the cause of non-identifiability problems. The result of our correlation analysis provides a necessary condition for experimental design in order to acquire suitable measurement data for unique parameter estimation.

Optimization and model reduction in the high dimensional parameter space of a budding yeast cell cycle model

Background

Parameter estimation from experimental data is critical for mathematical modeling of protein regulatory networks. For realistic networks with dozens of species and reactions, parameter estimation is an especially challenging task. In this study, we present an approach for parameter estimation that is effective in fitting a model of the budding yeast cell cycle (comprising 26 nonlinear ordinary differential equations containing 126 rate constants) to the experimentally observed phenotypes (viable or inviable) of 119 genetic strains carrying mutations of cell cycle genes.

Results

Starting from an initial guess of the parameter values, which correctly captures the phenotypes of only 72 genetic strains, our parameter estimation algorithm quickly improves the success rate of the model to 105–111 of the 119 strains. This success rate is comparable to the best values achieved by a skilled modeler manually choosing parameters over many weeks. The algorithm combines two search and optimization strategies. First, we use Latin hypercube sampling to explore a region surrounding the initial guess. From these samples, we choose ∼20 different sets of parameter values that correctly capture wild type viability. These sets form the starting generation of differential evolution that selects new parameter values that perform better in terms of their success rate in capturing phenotypes. In addition to producing highly successful combinations of parameter values, we analyze the results to determine the parameters that are most critical for matching experimental outcomes and the most competitive strains whose correct outcome with a given parameter vector forces numerous other strains to have incorrect outcomes. These “most critical parameters” and “most competitive strains” provide biological insights into the model. Conversely, the “least critical parameters” and “least competitive strains” suggest ways to reduce the computational complexity of the optimization.

Conclusions

Our approach proves to be a useful tool to help systems biologists fit complex dynamical models to large experimental datasets. In the process of fitting the model to the data, the tool identifies suggestive correlations among aspects of the model and the data.