Structural identifiability of systems biology models: a critical comparison of methods

Bayesian model selection for the Drosophila gap gene network

Background

The gap gene system controls the early cascade of the segmentation pathway in Drosophila melanogaster as well as other insects. Owing to its tractability and key role in embryo patterning, this system has been the focus for both computational modelers and experimentalists. The gap gene expression dynamics can be considered strictly as a one-dimensional process and modeled as a system of reaction-diffusion equations. While substantial progress has been made in modeling this phenomenon, there still remains a deficit of approaches to evaluate competing hypotheses. Most of the model development has happened in isolation and there has been little attempt to compare candidate models.

Results

The Bayesian framework offers a means of doing formal model evaluation. Here, we demonstrate how this framework can be used to compare different models of gene expression. We focus on the Papatsenko-Levine formalism, which exploits a fractional occupancy based approach to incorporate activation of the gap genes by the maternal genes and cross-regulation by the gap genes themselves. The Bayesian approach provides insight about relationship between system parameters. In the regulatory pathway of segmentation, the parameters for number of binding sites and binding affinity have a negative correlation. The model selection analysis supports a stronger binding affinity for Bicoid compared to other regulatory edges, as shown by a larger posterior mean. The procedure doesn’t show support for activation of Kruppel by Bicoid.

Conclusions

We provide an efficient solver for the general representation of the Papatsenko-Levine model. We also demonstrate the utility of Bayes factor for evaluating candidate models for spatial pattering models. In addition, by using the parallel tempering sampler, the convergence of Markov chains can be remarkably improved and robust estimates of Bayes factors obtained.

Electronic supplementary material

The online version of this article (10.1186/s12859-019-2888-0) contains supplementary material, which is available to authorized users.

Optimising mechanical ventilation through model-based methods and automation

Mechanical ventilation (MV) is a core life-support therapy for patients suffering from respiratory failure or acute respiratory distress syndrome (ARDS). Respiratory failure is a secondary outcome of a range of injuries and diseases, and results in almost half of all intensive care unit (ICU) patients receiving some form of MV. Funding the increasing demand for ICU is a major issue and MV, in particular, can double the cost per day due to significant patient variability, over-sedation, and the large amount of clinician time required for patient management. Reducing cost in this area requires both a decrease in the average duration of MV by improving care, and a reduction in clinical workload. Both could be achieved by safely automating all or part of MV care via model-based dynamic systems modelling and control methods are ideally suited to address these problems. This paper presents common lung models, and provides a vision for a more automated future and explores predictive capacity of some current models. This vision includes the use of model-based methods to gain real-time insight to patient condition, improve safety through the forward prediction of outcomes to changes in MV, and develop virtual patients for in-silico design and testing of clinical protocols. Finally, the use of dynamic systems models and system identification to guide therapy for improved personalised control of oxygenation and MV therapy in the ICU will be considered. Such methods are a major part of the future of medicine, which includes greater personalisation and predictive capacity to both optimise care and reduce costs. This review thus presents the state of the art in how dynamic systems and control methods can be applied to transform this core area of ICU medicine.

Parameter estimation and identifiability in a neural population model for electro-cortical activity

Electroencephalography (EEG) provides a non-invasive measure of brain electrical activity. Neural population models, where large numbers of interacting neurons are considered collectively as a macroscopic system, have long been used to understand features in EEG signals. By tuning dozens of input parameters describing the excitatory and inhibitory neuron populations, these models can reproduce prominent features of the EEG such as the alpha-rhythm. However, the inverse problem, of directly estimating the parameters from fits to EEG data, remains unsolved. Solving this multi-parameter non-linear fitting problem will potentially provide a real-time method for characterizing average neuronal properties in human subjects. Here we perform unbiased fits of a 22-parameter neural population model to EEG data from 82 individuals, using both particle swarm optimization and Markov chain Monte Carlo sampling. We estimate how much is learned about individual parameters by computing Kullback-Leibler divergences between posterior and prior distributions for each parameter. Results indicate that only a single parameter, that determining the dynamics of inhibitory synaptic activity, is directly identifiable, while other parameters have large, though correlated, uncertainties. We show that the eigenvalues of the Fisher information matrix are roughly uniformly spaced over a log scale, indicating that the model is sloppy, like many of the regulatory network models in systems biology. These eigenvalues indicate that the system can be modeled with a low effective dimensionality, with inhibitory synaptic activity being prominent in driving system behavior.

A Bayesian framework for the analysis of systems biology models of the brain

Systems biology models are used to understand complex biological and physiological systems. Interpretation of these models is an important part of developing this understanding. These models are often fit to experimental data in order to understand how the system has produced various phenomena or behaviour that are seen in the data. In this paper, we have outlined a framework that can be used to perform Bayesian analysis of complex systems biology models. In particular, we have focussed on analysing a systems biology of the brain using both simulated and measured data. By using a combination of sensitivity analysis and approximate Bayesian computation, we have shown that it is possible to obtain distributions of parameters that can better guard against misinterpretation of results, as compared to a maximum likelihood estimate based approach. This is done through analysis of simulated and experimental data. NIRS measurements were simulated using the same simulated systemic input data for the model in a ‘healthy’ and ‘impaired’ state. By analysing both of these datasets, we show that different parameter spaces can be distinguished and compared between different physiological states or conditions. Finally, we analyse experimental data using the new Bayesian framework and the previous maximum likelihood estimate approach, showing that the Bayesian approach provides a more complete understanding of the parameter space.

Systems biology models are mathematical representations of biological processes that reproduce the overall behaviour of a biological system. They are comprised by a number of parameters representing biological information. We use them to understand the behaviour of biological systems, such as the brain. We do this by fitting the model’s parameter to observed or simulated data; and by looking at how these values change during the fitting process we investigate the behaviour of our system. We are interested in understanding differences between a healthy and an injured brain. Here we outline a statistical framework that uses a Bayesian approach during the fitting process that can provide us with a distribution of parameters rather than single parameter number. We apply this method when simulating the physiological responses between a healthy and a vascular compromised brain to a drop in oxygenation. We then use experimental data that demonstrates the healthy brain response to an increase in arterial CO2 and fit our brain model predictions to the measurements. In both instances we show that our approach provides more information about the overlap between healthy and unhealthy brain states than a fitting process that provides a single value parameter estimate.

Chebyshev Approximation and the Global Geometry of Model Predictions

Complex nonlinear models are typically ill conditioned or sloppy; their predictions are significantly affected by only a small subset of parameter combinations, and parameters are difficult to reconstruct from model behavior. Despite forming an important universality class and arising frequently in practice when performing a nonlinear fit to data, formal and systematic explanations of sloppiness are lacking. By unifying geometric interpretations of sloppiness with Chebyshev approximation theory, we rigorously explain sloppiness as a consequence of model smoothness. Our approach results in universal bounds on model predictions for classes of smooth models, capturing global geometric features that are intrinsic to their model manifolds, and characterizing a universality class of models. We illustrate this universality using three models from disparate fields (physics, chemistry, biology): exponential curves, reaction rates from an enzyme-catalyzed chemical reaction, and an epidemiology model of an infected population.

Identifiability from a Few Species for a Class of Biochemical Reaction Networks

Under mass-action kinetics, biochemical reaction networks give rise to polynomial autonomous dynamical systems whose parameters are often difficult to estimate. We deal in this paper with the problem of identifying the kinetic parameters of a class of biochemical networks which are abundant, such as multisite phosphorylation systems and phosphorylation cascades (for example, MAPK cascades). For any system of this class, we explicitly exhibit a single species for each connected component of the associated digraph such that the successive total derivatives of its concentration allow us to identify all the parameters occurring in the component. The number of derivatives needed is bounded essentially by the length of the corresponding connected component of the digraph. Moreover, in the particular case of the cascades, we show that the parameters can be identified from a bounded number of successive derivatives of the last product of the last layer. This theoretical result induces also a heuristic interpolation-based identifiability procedure to recover the values of the rate constants from exact measurements.

Cardiovascular regulation in response to multiple hemorrhages: analysis and parameter estimation

Mathematical models can provide useful insights explaining behavior observed in experimental data; however, rigorous analysis is needed to select a subset of model parameters that can be informed by available data. Here we present a method to estimate an identifiable set of parameters based on baseline left ventricular pressure and volume time series data. From this identifiable subset, we then select, based on current understanding of cardiovascular control, parameters that vary in time in response to blood withdrawal, and estimate these parameters over a series of blood withdrawals. These time-varying parameters are first estimated using piecewise linear splines minimizing the mean squared error between measured and computed left ventricular pressure and volume data over four consecutive blood withdrawals. As a final step, the trends in these splines are fit with empirical functional expressions selected to describe cardiovascular regulation during blood withdrawal. Our analysis at baseline found parameters representing timing of cardiac contraction, systemic vascular resistance, and cardiac contractility to be identifiable. Of these parameters, vascular resistance and cardiac contractility were varied in time. Data used for this study were measured in a control Sprague-Dawley rat. To our knowledge, this is the first study to analyze the response to multiple blood withdrawals both experimentally and theoretically, as most previous studies focus on analyzing the response to one large blood withdrawal. Results show that during each blood withdrawal both systemic vascular resistance and contractility decrease acutely and partially recover, and they decrease chronically across the series of blood withdrawals.

GenSSI 2.0: multi-experiment structural identifiability analysis of SBML models

Motivation

Mathematical modeling using ordinary differential equations is used in systems biology to improve the understanding of dynamic biological processes. The parameters of ordinary differential equation models are usually estimated from experimental data. To analyze a priori the uniqueness of the solution of the estimation problem, structural identifiability analysis methods have been developed.

Results

We introduce GenSSI 2.0, an advancement of the software toolbox GenSSI (Generating Series for testing Structural Identifiability). GenSSI 2.0 is the first toolbox for structural identifiability analysis to implement Systems Biology Markup Language import, state/parameter transformations and multi-experiment structural identifiability analysis. In addition, GenSSI 2.0 supports a range of MATLAB versions and is computationally more efficient than its previous version, enabling the analysis of more complex models.

Availability and implementation

GenSSI 2.0 is an open-source MATLAB toolbox and available at https://github.com/genssi-developer/GenSSI.

Supplementary information

Supplementary data are available at Bioinformatics online.

Parameter estimation in models of biological oscillators: an automated regularised estimation approach

BACKGROUND

Dynamic modelling is a core element in the systems biology approach to understanding complex biosystems. Here, we consider the problem of parameter estimation in models of biological oscillators described by deterministic nonlinear differential equations. These problems can be extremely challenging due to several common pitfalls: (i) a lack of prior knowledge about parameters (i.e. massive search spaces), (ii) convergence to local optima (due to multimodality of the cost function), (iii) overfitting (fitting the noise instead of the signal) and (iv) a lack of identifiability. As a consequence, the use of standard estimation methods (such as gradient-based local ones) will often result in wrong solutions. Overfitting can be particularly problematic, since it produces very good calibrations, giving the impression of an excellent result. However, overfitted models exhibit poor predictive power. Here, we present a novel automated approach to overcome these pitfalls. Its workflow makes use of two sequential optimisation steps incorporating three key algorithms: (1) sampling strategies to systematically tighten the parameter bounds reducing the search space, (2) efficient global optimisation to avoid convergence to local solutions, (3) an advanced regularisation technique to fight overfitting. In addition, this workflow incorporates tests for structural and practical identifiability.

RESULTS

We successfully evaluate this novel approach considering four difficult case studies regarding the calibration of well-known biological oscillators (Goodwin, FitzHugh-Nagumo, Repressilator and a metabolic oscillator). In contrast, we show how local gradient-based approaches, even if used in multi-start fashion, are unable to avoid the above-mentioned pitfalls.

CONCLUSIONS

Our approach results in more efficient estimations (thanks to the bounding strategy) which are able to escape convergence to local optima (thanks to the global optimisation approach). Further, the use of regularisation allows us to avoid overfitting, resulting in more generalisable calibrated models (i.e. models with greater predictive power).

Assessing parameter identifiability in compartmental dynamic models using a computational approach: application to infectious disease transmission models

Background

Mathematical modeling is now frequently used in outbreak investigations to understand underlying mechanisms of infectious disease dynamics, assess patterns in epidemiological data, and forecast the trajectory of epidemics. However, the successful application of mathematical models to guide public health interventions lies in the ability to reliably estimate model parameters and their corresponding uncertainty. Here, we present and illustrate a simple computational method for assessing parameter identifiability in compartmental epidemic models.

Methods

We describe a parametric bootstrap approach to generate simulated data from dynamical systems to quantify parameter uncertainty and identifiability. We calculate confidence intervals and mean squared error of estimated parameter distributions to assess parameter identifiability. To demonstrate this approach, we begin with a low-complexity SEIR model and work through examples of increasingly more complex compartmental models that correspond with applications to pandemic influenza, Ebola, and Zika.

Results

Overall, parameter identifiability issues are more likely to arise with more complex models (based on number of equations/states and parameters). As the number of parameters being jointly estimated increases, the uncertainty surrounding estimated parameters tends to increase, on average, as well. We found that, in most cases, R₀ is often robust to parameter identifiability issues affecting individual parameters in the model. Despite large confidence intervals and higher mean squared error of other individual model parameters, R₀ can still be estimated with precision and accuracy.

Conclusions

Because public health policies can be influenced by results of mathematical modeling studies, it is important to conduct parameter identifiability analyses prior to fitting the models to available data and to report parameter estimates with quantified uncertainty. The method described is helpful in these regards and enhances the essential toolkit for conducting model-based inferences using compartmental dynamic models.

Electronic supplementary material

The online version of this article (10.1186/s12976-018-0097-6) contains supplementary material, which is available to authorized users.

Workflow Description to Dynamically Model β-Arrestin Signaling Networks

Dynamic models of signaling networks allow the formulation of hypotheses on the topology and kinetic rate laws characterizing a given molecular network, in-depth exploration, and confrontation with kinetic biological data. Despite its standardization, dynamic modeling of signaling networks still requires successive technical steps that need to be carefully performed. Here, we detail these steps by going through the mathematical and statistical framework. We explain how it can be applied to the understanding of β-arrestin-dependent signaling networks. We illustrate our methodology through the modeling of β-arrestin recruitment kinetics at the follicle-stimulating hormone (FSH) receptor supported by in-house bioluminescence resonance energy transfer (BRET) data.

Mechanistic description of spatial processes using integrative modelling of noise-corrupted imaging data

Spatial patterns are ubiquitous on the subcellular, cellular and tissue level, and can be studied using imaging techniques such as light and fluorescence microscopy. Imaging data provide quantitative information about biological systems; however, mechanisms causing spatial patterning often remain elusive. In recent years, spatio-temporal mathematical modelling has helped to overcome this problem. Yet, outliers and structured noise limit modelling of whole imaging data, and models often consider spatial summary statistics. Here, we introduce an integrated data-driven modelling approach that can cope with measurement artefacts and whole imaging data. Our approach combines mechanistic models of the biological processes with robust statistical models of the measurement process. The parameters of the integrated model are calibrated using a maximum-likelihood approach. We used this integrated modelling approach to study in vivo gradients of the chemokine (C-C motif) ligand 21 (CCL21). CCL21 gradients guide dendritic cells and are important in the adaptive immune response. Using artificial data, we verified that the integrated modelling approach provides reliable parameter estimates in the presence of measurement noise and that bias and variance of these estimates are reduced compared to conventional approaches. The application to experimental data allowed the parametrization and subsequent refinement of the model using additional mechanisms. Among other results, model-based hypothesis testing predicted lymphatic vessel-dependent concentration of heparan sulfate, the binding partner of CCL21. The selected model provided an accurate description of the experimental data and was partially validated using published data. Our findings demonstrate that integrated statistical modelling of whole imaging data is computationally feasible and can provide novel biological insights.

Multi-experiment nonlinear mixed effect modeling of single-cell translation kinetics after transfection

Single-cell time-lapse studies have advanced the quantitative understanding of cellular pathways and their inherent cell-to-cell variability. However, parameters retrieved from individual experiments are model dependent and their estimation is limited, if based on solely one kind of experiment. Hence, methods to integrate data collected under different conditions are expected to improve model validation and information content. Here we present a multi-experiment nonlinear mixed effect modeling approach for mechanistic pathway models, which allows the integration of multiple single-cell perturbation experiments. We apply this approach to the translation of green fluorescent protein after transfection using a massively parallel read-out of micropatterned single-cell arrays. We demonstrate that the integration of data from perturbation experiments allows the robust reconstruction of cell-to-cell variability, i.e., parameter densities, while each individual experiment provides insufficient information. Indeed, we show that the integration of the datasets on the population level also improves the estimates for individual cells by breaking symmetries, although each of them is only measured in one experiment. Moreover, we confirmed that the suggested approach is robust with respect to batch effects across experimental replicates and can provide mechanistic insights into the nature of batch effects. We anticipate that the proposed multi-experiment nonlinear mixed effect modeling approach will serve as a basis for the analysis of cellular heterogeneity in single-cell dynamics.

Global optimization using Gaussian processes to estimate biological parameters from image data

Parameter estimation is a major challenge in computational modeling of biological processes. This is especially the case in image-based modeling where the inherently quantitative output of the model is measured against image data, which is typically noisy and non-quantitative. In addition, these models can have a high computational cost, limiting the number of feasible simulations, and therefore rendering most traditional parameter estimation methods unsuitable. In this paper, we present a pipeline that uses Gaussian process learning to estimate biological parameters from noisy, non-quantitative image data when the model has a high computational cost. This approach is first successfully tested on a parametric function with the goal of retrieving the original parameters. We then apply it to estimating parameters in a biological setting by fitting artificial in-situ hybridization (ISH) data of the developing murine limb bud. We expect that this method will be of use in a variety of modeling scenarios where quantitative data is missing and the use of standard parameter estimation approaches in biological modeling is prohibited by the computational cost of the model.

Efficacy of control measures in the control of Ebola, Liberia 2014-2015

The largest outbreak of Ebola to date is the 2014 West Africa Ebola outbreak, with more than 10,000 cases and over 4000 deaths reported in Liberia alone. To control the spread of the outbreak, multiple interventions were implemented: identification and isolation of cases, contact tracing, quarantining of suspected contacts, proper personal protection, safely conducted burials, improved education, social awareness and individual protective measures. Devising rigorous methodologies for the evaluation of the effectiveness of the control measures implemented to stop an outbreak is of paramount importance. In this paper, we evaluate the effectiveness of the 2014 Ebola outbreak interventions. We rely on model selection to determine the best model that explains the 2014 Ebola outbreak data in Liberia which is the simplest model with a social distancing term. We couple structural and practical identifiability analysis with the computation of confidence intervals to pinpoint the uncertainty in the parameter estimations. Finally, we evaluate the efficacy of control measures using the Ebola model with social distancing. Among all the control measures, we find that social distancing had the most impact on the control of the 2014 Ebola epidemic in Libreria followed by isolation and quarantining.

Practical unidentifiability of a simple vector-borne disease model: Implications for parameter estimation and intervention assessment

Mathematical modeling has an extensive history in vector-borne disease epidemiology, and is increasingly used for prediction, intervention design, and understanding mechanisms. Many studies rely on parameter estimation to link models and data, and to tailor predictions and counterfactuals to specific settings. However, few studies have formally evaluated whether vector-borne disease models can properly estimate the parameters of interest given the constraints of a particular dataset. Identifiability analysis allows us to examine whether model parameters can be estimated uniquely-a lack of consideration of such issues can result in misleading or incorrect parameter estimates and model predictions. Here, we evaluate both structural (theoretical) and practical identifiability of a commonly used compartmental model of mosquito-borne disease, using the 2010 dengue epidemic in Taiwan as a case study. We show that while the model is structurally identifiable, it is practically unidentifiable under a range of human and mosquito time series measurement scenarios. In particular, the transmission parameters form a practically identifiable combination and thus cannot be estimated separately, potentially leading to incorrect predictions of the effects of interventions. However, in spite of the unidentifiability of the individual parameters, the basic reproduction number was successfully estimated across the unidentifiable parameter ranges. These identifiability issues can be resolved by directly measuring several additional human and mosquito life-cycle parameters both experimentally and in the field. While we only consider the simplest case for the model, we show that a commonly used model of vector-borne disease is unidentifiable from human and mosquito incidence data, making it difficult or impossible to estimate parameters or assess intervention strategies. This work illustrates the importance of examining identifiability when linking models with data to make predictions and inferences, and particularly highlights the importance of combining laboratory, field, and case data if we are to successfully estimate epidemiological and ecological parameters using models.

Parameter Identification For Continuous Fluidized Bed Spray Agglomeration

Agglomeration represents an important particle formation process used in many industries. One particularly attractive process setup is continuous fluidized bed spray agglomeration, which features good mixing as well as high heat and mass transfer on the one hand and constant product throughput with constant quality as well as high flow rates compared to batch mode on the other hand. Particle properties such as agglomerate size or porosity significantly affect overall product properties such as re-hydration behavior and dissolubility. These can be influenced by different operating parameters. In this manuscript, a population balance model for a continuous fluidized bed spray agglomeration is presented and adapted to experimental data. Focus is on the description of the dynamic behavior in continuous operation mode in a certain neighborhood around steady-state. Different kernel candidates are evaluated and it is shown that none of the kernels are able to match the first six minutes with time independent parameters. Afterwards, a good fit can be obtained, where the Brownian and the volume independent kernel models match best with the experimental data. Model fit is improved for identification on a shifted time domain neglecting the initial start-up phase. Here, model identifiability is shown and parameter confidence intervals are computed via parametric bootstrap.

Determining minimal output sets that ensure structural identifiability

The process of inferring parameter values from experimental data can be a cumbersome task. In addition, the collection of experimental data can be time consuming and costly. This paper covers both these issues by addressing the following question: “Which experimental outputs should be measured to ensure that unique model parameters can be calculated?”. Stated formally, we examine the topic of minimal output sets that guarantee a model’s structural identifiability. To that end, we introduce an algorithm that guides a researcher as to which model outputs to measure. Our algorithm consists of an iterative structural identifiability analysis and can determine multiple minimal output sets of a model. This choice in different output sets offers researchers flexibility during experimental design. Our method can determine minimal output sets of large differential equation models within short computational times.

Identification of parameters in systems biology

We consider the inverse problem for the identification of the finite dimensional set of parameters for systems of nonlinear ordinary differential equations (ODEs) arising in systems biology. A numerical method which combines Bellman's quasilinearization with sensitivity analysis and Tikhonov's regularization is implemented. We apply the method to various biological models such as the classical Lotka-Volterra system, bistable switch model in genetic regulatory networks, gene regulation and repressilator models from synthetic biology. The numerical results and application to real data demonstrate the quadratic convergence.

Can Population Modelling Principles be Used to Identify Key PBPK Parameters for Paediatric Clearance Predictions? An Innovative Application of Optimal Design Theory

PURPOSE

Physiologically-based pharmacokinetic (PBPK) models are essential in drug development, but require parameters that are not always obtainable. We developed a methodology to investigate the feasibility and requirements for precise and accurate estimation of PBPK parameters using population modelling of clinical data and illustrate this for two key PBPK parameters for hepatic metabolic clearance, namely whole liver unbound intrinsic clearance (CLint_,u,WL) and hepatic blood flow (Qh) in children.

METHODS

First, structural identifiability was enabled through re-parametrization and the definition of essential trial design components. Subsequently, requirements for the trial components to yield precise estimation of the PBPK parameters and their inter-individual variability were established using a novel application of population optimal design theory. Finally, the performance of the proposed trial design was assessed using stochastic simulation and estimation.

RESULTS

Precise estimation of CLint_,u,WL and Qh and their inter-individual variability was found to require a trial with two drugs, of which one has an extraction ratio (ER) ≤ 0.27 and the other has an ER ≥ 0.93. The proposed clinical trial design was found to lead to precise and accurate parameter estimates and was robust to parameter uncertainty.

CONCLUSION

The proposed framework can be applied to other PBPK parameters and facilitate the development of PBPK models.

Identifiability Analysis of Three Control-Oriented Models for Use in Artificial Pancreas Systems

OBJECTIVE

Our aim is to analyze the identifiability of three commonly used control-oriented models for glucose control in patients with type 1 diabetes (T1D).

METHODS

Structural and practical identifiability analysis were performed on three published control-oriented models for glucose control in patients with type 1 diabetes (T1D): the subcutaneous oral glucose minimal model (SOGMM), the intensive control insulin-nutrition-glucose (ICING) model, and the minimal model control-oriented (MMC). Structural identifiability was addressed with a combination of the generating series (GS) approach and identifiability tableaus whereas practical identifiability was studied by means of (1) global ranking of parameters via sensitivity analysis together with the Latin hypercube sampling method (LHS) and (2) collinearity analysis among parameters. For practical identifiability and model identification, continuous glucose monitor (CGM), insulin pump, and meal records were selected from a set of patients (n = 5) on continuous subcutaneous insulin infusion (CSII) that underwent a clinical trial in an outpatient setting. The performance of the identified models was analyzed by means of the root mean square (RMS) criterion.

RESULTS

A reliable set of identifiable parameters was found for every studied model after analyzing the possible identifiability issues of the original parameter sets. According to an importance factor ([Formula: see text]), it was shown that insulin sensitivity is not the most influential parameter from the dynamical point of view, that is, is not the parameter impacting the outputs the most of the three models, contrary to what is assumed in the literature. For the test data, the models demonstrated similar performance with most RMS values around 20 mg/dl (min: 15.64 mg/dl, max: 51.32 mg/dl). However, MMC failed to identify the model for patient 4. Also, considering the three models, the MMC model showed the higher parameter variability when reidentified every 6 hours.

CONCLUSION

This study shows that both structural and practical identifiability analysis need to be considered prior to the model identification/individualization in patients with T1D. It was shown that all the studied models are able to represent the CGM data, yet their usefulness in a hypothetical artificial pancreas could be a matter of debate. In spite that the three models do not capture all the dynamics and metabolic effects as a maximal model (ie, our FDA-accepted UVa/Padova simulator), SOGMM and ICING appear to be more appealing than MMC regarding both the performance and parameter variability after reidentification. Although the model predictions of ICING are comparable to the ones of the SOGMM model, the large parameter set makes the model prone to overfitting if all parameters are identified. Moreover, the existence of a high nonlinear function like [Formula: see text] prevents the use of tools from the linear systems theory.

Identifiability and Reconstruction of Biochemical Reaction Networks from Population Snapshot Data

Inference of biochemical network models from experimental data is a crucial problem in systems and synthetic biology that includes parameter calibration but also identification of unknown interactions. Stochastic modelling from single-cell data is known to improve identifiability of reaction network parameters for specific systems. However, general results are lacking, and the advantage over deterministic, population-average approaches has not been explored for network reconstruction. In this work, we study identifiability and propose new reconstruction methods for biochemical interaction networks. Focusing on population-snapshot data and networks with reaction rates affine in the state, for parameter estimation, we derive general methods to test structural identifiability and demonstrate them in connection with practical identifiability for a reporter gene in silico case study. In the same framework, we next develop a two-step approach to the reconstruction of unknown networks of interactions. We apply it to compare the achievable network reconstruction performance in a deterministic and a stochastic setting, showing the advantage of the latter, and demonstrate it on population-snapshot data from a simulated example.

Bayesian parameter estimation for biochemical reaction networks using region-based adaptive parallel tempering

Motivation

Mathematical models have become standard tools for the investigation of cellular processes and the unraveling of signal processing mechanisms. The parameters of these models are usually derived from the available data using optimization and sampling methods. However, the efficiency of these methods is limited by the properties of the mathematical model, e.g. non-identifiabilities, and the resulting posterior distribution. In particular, multi-modal distributions with long valleys or pronounced tails are difficult to optimize and sample. Thus, the developement or improvement of optimization and sampling methods is subject to ongoing research.

Results

We suggest a region-based adaptive parallel tempering algorithm which adapts to the problem-specific posterior distributions, i.e. modes and valleys. The algorithm combines several established algorithms to overcome their individual shortcomings and to improve sampling efficiency. We assessed its properties for established benchmark problems and two ordinary differential equation models of biochemical reaction networks. The proposed algorithm outperformed state-of-the-art methods in terms of calculation efficiency and mixing. Since the algorithm does not rely on a specific problem structure, but adapts to the posterior distribution, it is suitable for a variety of model classes.

Availability and implementation

The code is available both as Supplementary Material and in a Git repository written in MATLAB.

Supplementary information

Supplementary data are available at Bioinformatics online.

Structural and Practical Identifiability Analysis of Zika Epidemiological Models

The Zika virus (ZIKV) epidemic has caused an ongoing threat to global health security and spurred new investigations of the virus. Use of epidemiological models for arbovirus diseases can be a powerful tool to assist in prevention and control of the emerging disease. In this article, we introduce six models of ZIKV, beginning with a general vector-borne model and gradually including different transmission routes of ZIKV. These epidemiological models use various combinations of disease transmission (vector and direct) and infectious classes (asymptomatic and pregnant), with addition to loss of immunity being included. The disease-induced death rate is omitted from the models. We test the structural and practical identifiability of the models to find whether unknown model parameters can uniquely be determined. The models were fit to obtain time-series data of cumulative incidences and pregnant infections from the Florida Department of Health Daily Zika Update Reports. The average relative estimation errors (AREs) were computed from the Monte Carlo simulations to further analyze the identifiability of the models. We show that direct transmission rates are not practically identifiable; however, fixed recovery rates improve identifiability overall. We found ARE is low for each model (only slightly higher for those that account for a pregnant class) and help to confirm a reproduction number greater than one at the start of the Florida epidemic. Basic reproduction number, [Formula: see text], is an epidemiologically important threshold value which gives the number of secondary cases generated by one infected individual in a totally susceptible population in duration of infectiousness. Elasticity of the reproduction numbers suggests that the mosquito-to-human ratio, mosquito life span and biting rate have the greatest potential for reducing the reproduction number of Zika, and therefore, corresponding control measures need to be focused on.

Adaptive optimal input design and parametric estimation of nonlinear dynamical systems: application to neuronal modeling

OBJECTIVE

Many physical models of biological processes including neural systems are characterized by parametric nonlinear dynamical relations between driving inputs, internal states, and measured outputs of the process. Fitting such models using experimental data (data assimilation) is a challenging task since the physical process often operates in a noisy, possibly non-stationary environment; moreover, conducting multiple experiments under controlled and repeatable conditions can be impractical, time consuming or costly. The accuracy of model identification, therefore, is dictated principally by the quality and dynamic richness of collected data over single or few experimental sessions. Accordingly, it is highly desirable to design efficient experiments that, by exciting the physical process with smart inputs, yields fast convergence and increased accuracy of the model.

APPROACH

We herein introduce an adaptive framework in which optimal input design is integrated with square root cubature Kalman filters (OID-SCKF) to develop an online estimation procedure that first, converges significantly quicker, thereby permitting model fitting over shorter time windows, and second, enhances model accuracy when only few process outputs are accessible. The methodology is demonstrated on common nonlinear models and on a four-area neural mass model with noisy and limited measurements. Estimation quality (speed and accuracy) is benchmarked against high-performance SCKF-based methods that commonly employ dynamically rich informed inputs for accurate model identification.

MAIN RESULTS

For all the tested models, simulated single-trial and ensemble averages showed that OID-SCKF exhibited (i) faster convergence of parameter estimates and (ii) lower dependence on inter-trial noise variability with gains up to around 1000 ms in speed and 81% increase in variability for the neural mass models. In terms of accuracy, OID-SCKF estimation was superior, and exhibited considerably less variability across experiments, in identifying model parameters of (a) systems with challenging model inversion dynamics and (b) systems with fewer measurable outputs that directly relate to the underlying processes.

SIGNIFICANCE

Fast and accurate identification therefore carries particular promise for modeling of transient (short-lived) neuronal network dynamics using a spatially under-sampled set of noisy measurements, as is commonly encountered in neural engineering applications.

Structural and practical identifiability analysis of outbreak models

Estimating the reproduction number of an emerging infectious disease from an epidemiological data is becoming more essential in evaluating the current status of an outbreak. However, these studies are lacking the fundamental prerequisite in parameter estimation problem, namely the structural identifiability of the epidemic model, which determines the possibility of uniquely determining the model parameters from the epidemic data. In this paper, we perform both structural and practical identifiability analysis to classical epidemic models such as SIR (Susceptible-Infected-Recovered), SEIR (Susceptible-Exposed-Infected-Recovered) and an epidemic model with the treatment class (SITR). We performed structural identifiability analysis on these epidemic models using a differential algebra approach to investigate the well-posedness of the parameter estimation problem. Parameters of these models are estimated from different data types, namely prevalence, cumulative incidences and treated individuals. Furthermore, we carried out practical identifiability analysis on these models using Monte Carlo simulations and Fisher's Information Matrix. Our study shows that the SIR model is both structurally and practically identifiable from the prevalence data. It is also structurally identifiable to cumulative incidence observations, but due to high correlations of the parameters, it is practically unidentifiable from the cumulative incidence data. Furthermore, we found that none of these simple epidemic models are practically identifiable from the cumulative incidence data which is the standard type of epidemiological data provided by CDC or WHO. Our analysis with simple SIR model suggest that the health agencies, if possible, should report prevalence rather than incidence data.

Profile likelihood-based analyses of infectious disease models

Ordinary differential equation models are frequently applied to describe the temporal evolution of epidemics. However, ordinary differential equation models are also utilized in other scientific fields. We summarize and transfer state-of-the art approaches from other fields like Systems Biology to infectious disease models. For this purpose, we use a simple SIR model with data from an influenza outbreak at an English boarding school in 1978 and a more complex model of a vector-borne disease with data from the Zika virus outbreak in Colombia in 2015-2016. Besides parameter estimation using a deterministic multistart optimization approach, a multitude of analyses based on the profile likelihood are presented comprising identifiability analysis and model reduction. The analyses were performed using the freely available modeling framework Data2Dynamics (data2dynamics.org) which has been awarded as best performing within the DREAM6 parameter estimation challenge and in the DREAM7 network reconstruction challenge.

Inverse Problems in Systems Biology: A Critical Review

Systems Biology may be assimilated to a symbiotic cyclic interplaying between the forward and inverse problems. Computational models need to be continuously refined through experiments and in turn they help us to make limited experimental resources more efficient. Every time one does an experiment we know that there will be some noise that can disrupt our measurements. Despite the noise certainly is a problem, the inverse problems already involve the inference of missing information, even if the data is entirely reliable. So the addition of a certain limited noise does not fundamentally change the situation but can be used to solve the so-called ill-posed problem, as defined by Hadamard. It can be seen as an extra source of information. Recent studies have shown that complex systems, among others the systems biology, are poorly constrained and ill-conditioned because it is difficult to use experimental data to fully estimate their parameters. For these reasons was born the concept of sloppy models, a sequence of models of increasing complexity that become sloppy in the limit of microscopic accuracy. Furthermore the concept of sloppy models contains also the concept of un-identifiability, because the models are characterized by many parameters that are poorly constrained by experimental data. Then a strategy needs to be designed to infer, analyze, and understand biological systems. The aim of this work is to provide a critical review to the inverse problems in systems biology defining a strategy to determine the minimal set of information needed to overcome the problems arising from dynamic biological models that generally may have many unknown, non-measurable parameters.

Model-Based Methods in the Biopharmaceutical Process Lifecycle

Model-based methods are increasingly used in all areas of biopharmaceutical process technology. They can be applied in the field of experimental design, process characterization, process design, monitoring and control. Benefits of these methods are lower experimental effort, process transparency, clear rationality behind decisions and increased process robustness. The possibility of applying methods adopted from different scientific domains accelerates this trend further. In addition, model-based methods can help to implement regulatory requirements as suggested by recent Quality by Design and validation initiatives. The aim of this review is to give an overview of the state of the art of model-based methods, their applications, further challenges and possible solutions in the biopharmaceutical process life cycle. Today, despite these advantages, the potential of model-based methods is still not fully exhausted in bioprocess technology. This is due to a lack of (i) acceptance of the users, (ii) user-friendly tools provided by existing methods, (iii) implementation in existing process control systems and (iv) clear workflows to set up specific process models. We propose that model-based methods be applied throughout the lifecycle of a biopharmaceutical process, starting with the set-up of a process model, which is used for monitoring and control of process parameters, and ending with continuous and iterative process improvement via data mining techniques.

Dynamical compensation and structural identifiability of biological models: Analysis, implications, and reconciliation

The concept of dynamical compensation has been recently introduced to describe the ability of a biological system to keep its output dynamics unchanged in the face of varying parameters. However, the original definition of dynamical compensation amounts to lack of structural identifiability. This is relevant if model parameters need to be estimated, as is often the case in biological modelling. Care should we taken when using an unidentifiable model to extract biological insight: the estimated values of structurally unidentifiable parameters are meaningless, and model predictions about unmeasured state variables can be wrong. Taking this into account, we explore alternative definitions of dynamical compensation that do not necessarily imply structural unidentifiability. Accordingly, we show different ways in which a model can be made identifiable while exhibiting dynamical compensation. Our analyses enable the use of the new concept of dynamical compensation in the context of parameter identification, and reconcile it with the desirable property of structural identifiability.

A robust behaviour is a desirable feature in many biological systems. The study of mechanisms capable of maintaining the transient response unchanged despite environmental disturbances has recently motivated the introduction of a new concept: Dynamical Compensation (DC). However, the original definition of DC with respect to a parameter amounts to structural unidentifiability of that parameter, which means that it cannot be estimated by measuring the model output. Since most biological models have unknown parameters that need to be estimated, DC can be considered a negative property for the purpose of model identification. In this paper we reconcile these two conflicting views by proposing a new definition of DC that captures its intended biological meaning (i.e. robustness, which should be a systemic property, intrinsic to the dynamics) while making it distinct from structural unidentifiability (which is a modelling property that depends on decisions made by the modeller, such as the choice of model outputs or unknown parameters, and on experimental constraints). Our definition enables a model to have DC with respect to a structurally identifiable parameter, thus increasing the applicability of the concept.

Observability of Complex Systems: Finding the Gap

For a reconstruction of state and parameter values in a dynamic system model, first the question whether these values can be uniquely determined from the data must be answered. This structural model property is known as observability or, in case of parameter calibration only, identifiability. Testing a given model for observability is a well studied problem in the systems and control sciences. However, it is increasingly difficult, if not impossible, to address this property for large size models that, nowadays, are frequently used. We demonstrate the application of a recently developed algorithm that overcomes this problem and is remarkably efficient. As an illustration we show how an observability analysis for a Chinese Hamster Ovary Cell model (34 states, 117 parameters), a JAKSTAT signalling model (31 states, 51 parameters), and a MAP Kinase model (100 states, 88 parameters) can be established in a very short time.

Extending existing structural identifiability analysis methods to mixed-effects models

The concept of structural identifiability for state-space models is expanded to cover mixed-effects state-space models. Two methods applicable for the analytical study of the structural identifiability of mixed-effects models are presented. The two methods are based on previously established techniques for non-mixed-effects models; namely the Taylor series expansion and the input-output form approach. By generating an exhaustive summary, and by assuming an infinite number of subjects, functions of random variables can be derived which in turn determine the distribution of the system's observation function(s). By considering the uniqueness of the analytical statistical moments of the derived functions of the random variables, the structural identifiability of the corresponding mixed-effects model can be determined. The two methods are applied to a set of examples of mixed-effects models to illustrate how they work in practice.

Quantifying rotavirus kinetics in the REH tumor cell line using in vitro data

Globally, rotavirus is the most common cause of diarrhea in children younger than 5 years of age, however, a quantitative understanding of the infection dynamics is still lacking. In this paper, we present the first study to extract viral kinetic parameters for in vitro rotavirus infections in the REH cell tumor line. We use a mathematical model of viral kinetics to extract parameter values by fitting the model to data from rotavirus infection of REH cells. While accurate results for some of the parameters of the mathematical model were not achievable due to its global non-identifiability, we are able to quantify approximately the time course of the infection for the first time. We also find that the basic reproductive number of rotavirus, which gives the number of secondary infections from a single infected cell, is much greater than one. Quantifying the kinetics of rotavirus leads not only to a better understanding of the infection process, but also provides a method for quantitative comparison of kinetics of different strains or for quantifying the effectiveness of antiviral treatment.

Fundamental limitations of network reconstruction from temporal data

Inferring properties of the interaction matrix that characterizes how nodes in a networked system directly interact with each other is a well-known network reconstruction problem. Despite a decade of extensive studies, network reconstruction remains an outstanding challenge. The fundamental limitations governing which properties of the interaction matrix (e.g. adjacency pattern, sign pattern or degree sequence) can be inferred from given temporal data of individual nodes remain unknown. Here, we rigorously derive the necessary conditions to reconstruct any property of the interaction matrix. Counterintuitively, we find that reconstructing any property of the interaction matrix is generically as difficult as reconstructing the interaction matrix itself, requiring equally informative temporal data. Revealing these fundamental limitations sheds light on the design of better network reconstruction algorithms that offer practical improvements over existing methods.

To be certain about the uncertainty: Bayesian statistics for 13 C metabolic flux analysis

¹³ C Metabolic Fluxes Analysis (¹³ C MFA) remains to be the most powerful approach to determine intracellular metabolic reaction rates. Decisions on strain engineering and experimentation heavily rely upon the certainty with which these fluxes are estimated. For uncertainty quantification, the vast majority of ¹³ C MFA studies relies on confidence intervals from the paradigm of Frequentist statistics. However, it is well known that the confidence intervals for a given experimental outcome are not uniquely defined. As a result, confidence intervals produced by different methods can be different, but nevertheless equally valid. This is of high relevance to ¹³ C MFA, since practitioners regularly use three different approximate approaches for calculating confidence intervals. By means of a computational study with a realistic model of the central carbon metabolism of E. coli, we provide strong evidence that confidence intervals used in the field depend strongly on the technique with which they were calculated and, thus, their use leads to misinterpretation of the flux uncertainty. In order to provide a better alternative to confidence intervals in ¹³ C MFA, we demonstrate that credible intervals from the paradigm of Bayesian statistics give more reliable flux uncertainty quantifications which can be readily computed with high accuracy using Markov chain Monte Carlo. In addition, the widely applied chi-square test, as a means of testing whether the model reproduces the data, is examined closer.

A confidence building exercise in data and identifiability: Modeling cancer chemotherapy as a case study

Mathematical modeling has a long history in the field of cancer therapeutics, and there is increasing recognition that it can help uncover the mechanisms that underlie tumor response to treatment. However, making quantitative predictions with such models often requires parameter estimation from data, raising questions of parameter identifiability and estimability. Even in the case of structural (theoretical) identifiability, imperfect data and the resulting practical unidentifiability of model parameters can make it difficult to infer the desired information, and in some cases, to yield biologically correct inferences and predictions. Here, we examine parameter identifiability and estimability using a case study of two compartmental, ordinary differential equation models of cancer treatment with drugs that are cell cycle-specific (taxol) as well as non-specific (oxaliplatin). We proceed through model building, structural identifiability analysis, parameter estimation, practical identifiability analysis and its biological implications, as well as alternative data collection protocols and experimental designs that render the model identifiable. We use the differential algebra/input-output relationship approach for structural identifiability, and primarily the profile likelihood approach for practical identifiability. Despite the models being structurally identifiable, we show that without consideration of practical identifiability, incorrect cell cycle distributions can be inferred, that would result in suboptimal therapeutic choices. We illustrate the usefulness of estimating practically identifiable combinations (in addition to the more typically considered structurally identifiable combinations) in generating biologically meaningful insights. We also use simulated data to evaluate how the practical identifiability of the model would change under alternative experimental designs. These results highlight the importance of understanding the underlying mechanisms rather than purely using parsimony or information criteria/goodness-of-fit to decide model selection questions. The overall roadmap for identifiability testing laid out here can be used to help provide mechanistic insight into complex biological phenomena, reduce experimental costs, and optimize model-driven experimentation.

Toward predictive food process models: A protocol for parameter estimation

Mathematical models, in particular, physics-based models, are essential tools to food product and process design, optimization and control. The success of mathematical models relies on their predictive capabilities. However, describing physical, chemical and biological changes in food processing requires the values of some, typically unknown, parameters. Therefore, parameter estimation from experimental data is critical to achieving desired model predictive properties. This work takes a new look into the parameter estimation (or identification) problem in food process modeling. First, we examine common pitfalls such as lack of identifiability and multimodality. Second, we present the theoretical background of a parameter identification protocol intended to deal with those challenges. And, to finish, we illustrate the performance of the proposed protocol with an example related to the thermal processing of packaged foods.