Similarity transformation approach to identifiability analysis of nonlinear compartmental models

Investigating and forecasting infectious disease dynamics using epidemiological and molecular surveillance data

The integration of viral genomic data into public health surveillance has revolutionized our ability to track and forecast infectious disease dynamics. This review addresses two critical aspects of infectious disease forecasting and monitoring: the methodological workflow for epidemic forecasting and the transformative role of molecular surveillance. We first present a detailed approach for validating epidemic models, emphasizing an iterative workflow that utilizes ordinary differential equation (ODE)-based models to investigate and forecast disease dynamics. We recommend a more structured approach to model validation, systematically addressing key stages such as model calibration, assessment of structural and practical parameter identifiability, and effective uncertainty propagation in forecasts. Furthermore, we underscore the importance of incorporating multiple data streams by applying both simulated and real epidemiological data from the COVID-19 pandemic to produce more reliable forecasts with quantified uncertainty. Additionally, we emphasize the pivotal role of viral genomic data in tracking transmission dynamics and pathogen evolution. By leveraging advanced computational tools such as Bayesian phylogenetics and phylodynamics, researchers can more accurately estimate transmission clusters and reconstruct outbreak histories, thereby improving data-driven modeling and forecasting and informing targeted public health interventions. Finally, we discuss the transformative potential of integrating molecular epidemiology with mathematical modeling to complement and enhance epidemic forecasting and optimize public health strategies.

Classical structural identifiability methodology applied to low-dimensional dynamic systems in receptor theory

Mathematical modelling has become a key tool in pharmacological analysis, towards understanding dynamics of cell signalling and quantifying ligand-receptor interactions. Ordinary differential equation (ODE) models in receptor theory may be used to parameterise such interactions using timecourse data, but attention needs to be paid to the theoretical identifiability of the parameters of interest. Identifiability analysis is an often overlooked step in many bio-modelling works. In this paper we introduce structural identifiability analysis (SIA) to the field of receptor theory by applying three classical SIA methods (transfer function, Taylor Series and similarity transformation) to ligand-receptor binding models of biological importance (single ligand and Motulsky-Mahan competition binding at monomers, and a recently presented model of a single ligand binding at receptor dimers). New results are obtained which indicate the identifiable parameters for a single timecourse for Motulsky-Mahan binding and dimerised receptor binding. Importantly, we further consider combinations of experiments which may be performed to overcome issues of non-identifiability, to ensure the practical applicability of the work. The three SIA methods are demonstrated through a tutorial-style approach, using detailed calculations, which show the methods to be tractable for the low-dimensional ODE models.

Controllability and accessibility analysis of nonlinear biosystems

BACKGROUND

We address the problem of determining the controllability and accessibility of nonlinear biosystems. We consider models described by affine-in-inputs ordinary differential equations, which are adequate for a wide array of biological processes. Roughly speaking, the controllability of a dynamical system determines the possibility of steering it from an initial state to any point in its neighbourhood; accessibility is a weaker form of controllability.

METHODS

While the methodology for analysing the controllability of linear systems is well established, its generalization to the nonlinear case has proven elusive. Thus, a number of related but different properties - including different versions of accessibility, reachability or weak local controllability - have been defined to approach its study, and several partial results exist in lieu of a general test. Here, leveraging the applicable results from differential geometric control theory, we source sufficient conditions to assess nonlinear controllability, as well as a necessary and sufficient condition for accessibility.

RESULTS

We develop an algorithmic procedure to evaluate these conditions efficiently, and we provide its open source implementation. Using this software tool, we analyse the accessibility and controllability of a number of models of biomedical interest. While some of them are fully controllable, we find others that are not, as is the case of some models of EGF and NFκB signalling networks.

CONCLUSIONS

The contributions in this paper facilitate the accessibility and controllability analysis of nonlinear models, not only in biomedicine but also in other areas in which they have been rarely performed to date.

Structural identifiability analysis of epidemic models based on differential equations: a tutorial-based primer

The successful application of epidemic models hinges on our ability to estimate model parameters from limited observations reliably. An often-overlooked step before estimating model parameters consists of ensuring that the model parameters are structurally identifiable from the observed states of the system. In this tutorial-based primer, intended for a diverse audience, including students training in dynamic systems, we review and provide detailed guidance for conducting structural identifiability analysis of differential equation epidemic models based on a differential algebra approach using differential algebra for identifiability of systems (DAISY) and Mathematica (Wolfram Research). This approach aims to uncover any existing parameter correlations that preclude their estimation from the observed variables. We demonstrate this approach through examples, including tutorial videos of compartmental epidemic models previously employed to study transmission dynamics and control. We show that the lack of structural identifiability may be remedied by incorporating additional observations from different model states, assuming that the system's initial conditions are known, using prior information to fix some parameters involved in parameter correlations, or modifying the model based on existing parameter correlations. We also underscore how the results of structural identifiability analysis can help enrich compartmental diagrams of differential-equation models by indicating the observed state variables and the results of the structural identifiability analysis.

Identifiable Paths and Cycles in Linear Compartmental Models

We introduce a class of linear compartmental models called identifiable path/cycle models which have the property that all of the monomial functions of parameters associated to the directed cycles and paths from input compartments to output compartments are identifiable and give sufficient conditions to obtain an identifiable path/cycle model. Removing leaks, we then show how one can obtain a locally identifiable model from an identifiable path/cycle model. These identifiable path/cycle models yield the only identifiable models with certain conditions on their graph structure and thus we provide necessary and sufficient conditions for identifiable models with certain graph properties. A sufficient condition based on the graph structure of the model is also provided so that one can test if a model is an identifiable path/cycle model by examining the graph itself. We also provide some necessary conditions for identifiability based on graph structure. Our proofs use algebraic and combinatorial techniques.

Symmetries in Dynamic Models of Biological Systems: Mathematical Foundations and Implications

Symmetries are ubiquitous in nature. Almost all organisms have some kind of “symmetry”, meaning that their shape does not change under some geometric transformation. This geometrical concept of symmetry is intuitive and easy to recognize. On the other hand, the behavior of many biological systems over time can be described with ordinary differential equations. These dynamic models may also possess “symmetries”, meaning that the time courses of some variables remain invariant under certain transformations. Unlike the previously mentioned symmetries, the ones present in dynamic models are not geometric, but infinitesimal transformations. These mathematical symmetries can be related to certain features of the system’s dynamic behavior, such as robustness or adaptation capabilities. However, they can also arise from questionable modeling choices, which may lead to non-identifiability and non-observability. This paper provides an overview of the types of symmetries that appear in dynamic models, the mathematical tools available for their analyses, the ways in which they are related to system properties, and the implications for biological modeling.

Parameter Estimation in the Age of Degeneracy and Unidentifiability

Parameter estimation from observable or experimental data is a crucial stage in any modeling study. Identifiability refers to one’s ability to uniquely estimate the model parameters from the available data. Structural unidentifiability in dynamic models, the opposite of identifiability, is associated with the notion of degeneracy where multiple parameter sets produce the same pattern. Therefore, the inverse function of determining the model parameters from the data is not well defined. Degeneracy is not only a mathematical property of models, but it has also been reported in biological experiments. Classical studies on structural unidentifiability focused on the notion that one can at most identify combinations of unidentifiable model parameters. We have identified a different type of structural degeneracy/unidentifiability present in a family of models, which we refer to as the Lambda-Omega (Λ-Ω) models. These are an extension of the classical lambda-omega (λ-ω) models that have been used to model biological systems, and display a richer dynamic behavior and waveforms that range from sinusoidal to square wave to spike like. We show that the Λ-Ω models feature infinitely many parameter sets that produce identical stable oscillations, except possible for a phase shift (reflecting the initial phase). These degenerate parameters are not identifiable combinations of unidentifiable parameters as is the case in structural degeneracy. In fact, reducing the number of model parameters in the Λ-Ω models is minimal in the sense that each one controls a different aspect of the model dynamics and the dynamic complexity of the system would be reduced by reducing the number of parameters. We argue that the family of Λ-Ω models serves as a framework for the systematic investigation of degeneracy and identifiability in dynamic models and for the investigation of the interplay between structural and other forms of unidentifiability resulting on the lack of information from the experimental/observational data.

Metaheuristics for pharmacometrics

Metaheuristics is a powerful optimization tool that is increasingly used across disciplines to tackle general purpose optimization problems. Nature-inspired metaheuristic algorithms is a subclass of metaheuristic algorithms and have been shown to be particularly flexible and useful in solving complicated optimization problems in computer science and engineering. A common practice with metaheuristics is to hybridize it with another suitably chosen algorithm for enhanced performance. This paper reviews metaheuristic algorithms and demonstrates some of its utility in tackling pharmacometric problems. Specifically, we provide three applications using one of its most celebrated members, particle swarm optimization (PSO), and show that PSO can effectively estimate parameters in complicated nonlinear mixed-effects models and to gain insights into statistical identifiability issues in a complex compartment model. In the third application, we demonstrate how to hybridize PSO with sparse grid, which is an often-used technique to evaluate high dimensional integrals, to search for D -efficient designs for estimating parameters in nonlinear mixed-effects models with a count outcome. We also show the proposed hybrid algorithm outperforms its competitors when sparse grid is replaced by its competitor, adaptive gaussian quadrature to approximate the integral, or when PSO is replaced by three notable nature-inspired metaheuristic algorithms.

On testing structural identifiability by a simple scaling method: Relying on scaling symmetries can be misleading

A recent paper published in PLOS Computational Biology [1] introduces the Scaling Invariance Method (SIM) for analysing structural local identifiability and observability. These two properties define mathematically the possibility of determining the values of the parameters (identifiability) and states (observability) of a dynamic model by observing its output. In this note we warn that SIM considers scaling symmetries as the only possible cause of non-identifiability and non-observability. We show that other types of symmetries can cause the same problems without being detected by SIM, and that in those cases the method may lead one to conclude that the model is identifiable and observable when it is actually not.

Bayesian analysis of Glucose dynamics during the Oral Glucose Tolerance Test (OGTT)

This paper proposes a model that considers the action and timing of insulin and glucagon in glucose homeostasis after an oral stimulus. We use the Bayesian paradigm to infer kinetic rates, namely insulin and glucagon secretion, gastrointestinal emptying, and basal glucose concentration in blood. We identify two insulin scores related to glucose concentration in both blood and the gastrointestinal tract. The scores allow us to suggest a classification for individuals with impaired insulin sensitivity.

Testing structural identifiability by a simple scaling method

Successful mathematical modeling of biological processes relies on the expertise of the modeler to capture the essential mechanisms in the process at hand and on the ability to extract useful information from empirical data. A model is said to be structurally unidentifiable, if different quantitative sets of parameters provide the same observable outcome. This is typical (but not exclusive) of partially observed problems in which only a few variables can be experimentally measured. Most of the available methods to test the structural identifiability of a model are either too complex mathematically for the general practitioner to be applied, or require involved calculations or numerical computation for complex non-linear models. In this work, we present a new analytical method to test structural identifiability of models based on ordinary differential equations, based on the invariance of the equations under the scaling transformation of its parameters. The method is based on rigorous mathematical results but it is easy and quick to apply, even to test the identifiability of sophisticated highly non-linear models. We illustrate our method by example and compare its performance with other existing methods in the literature.

Application of Systems Engineering Principles and Techniques in Biological Big Data Analytics: A Review

In the past few decades, we have witnessed tremendous advancements in biology, life sciences and healthcare. These advancements are due in no small part to the big data made available by various high-throughput technologies, the ever-advancing computing power, and the algorithmic advancements in machine learning. Specifically, big data analytics such as statistical and machine learning has become an essential tool in these rapidly developing fields. As a result, the subject has drawn increased attention and many review papers have been published in just the past few years on the subject. Different from all existing reviews, this work focuses on the application of systems, engineering principles and techniques in addressing some of the common challenges in big data analytics for biological, biomedical and healthcare applications. Specifically, this review focuses on the following three key areas in biological big data analytics where systems engineering principles and techniques have been playing important roles: the principle of parsimony in addressing overfitting, the dynamic analysis of biological data, and the role of domain knowledge in biological data analytics.

Finding and Breaking Lie Symmetries: Implications for Structural Identifiability and Observability in Biological Modelling

A dynamic model is structurally identifiable (respectively, observable) if it is theoretically possible to infer its unknown parameters (respectively, states) by observing its output over time. The two properties, structural identifiability and observability, are completely determined by the model equations. Their analysis is of interest for modellers because it informs about the possibility of gaining insight into a model’s unmeasured variables. Here we cast the problem of analysing structural identifiability and observability as that of finding Lie symmetries. We build on previous results that showed that structural unidentifiability amounts to the existence of Lie symmetries. We consider nonlinear models described by ordinary differential equations and restrict ourselves to rational functions. We revisit a method for finding symmetries by transforming rational expressions into linear systems. We extend the method by enabling it to provide symmetry-breaking transformations, which allows for a semi-automatic model reformulation that renders a non-observable model observable. We provide a MATLAB implementation of the methodology as part of the STRIKE-GOLDD toolbox for observability and identifiability analysis. We illustrate the use of the methodology in the context of biological modelling by applying it to a set of problems taken from the literature.

Calibration, Selection and Identifiability Analysis of a Mathematical Model of the in vitro Erythropoiesis in Normal and Perturbed Contexts

The in vivo erythropoiesis, which is the generation of mature red blood cells in the bone marrow of whole organisms, has been described by a variety of mathematical models in the past decades. However, the in vitro erythropoiesis, which produces red blood cells in cultures, has received much less attention from the modelling community. In this paper, we propose the first mathematical model of in vitro erythropoiesis. We start by formulating different models and select the best one at fitting experimental data of in vitro erythropoietic differentiation obtained from chicken erythroid progenitor cells. It is based on a set of linear ODE, describing 3 hypothetical populations of cells at different stages of differentiation. We then compute confidence intervals for all of its parameters estimates, and conclude that our model is fully identifiable. Finally, we use this model to compute the effect of a chemical drug called Rapamycin, which affects all states of differentiation in the culture, and relate these effects to specific parameter variations. We provide the first model for the kinetics of in vitro cellular differentiation which is proven to be identifiable. It will serve as a basis for a model which will better account for the variability which is inherent to the experimental protocol used for the model calibration.

Identification of a time-varying intracellular signalling model through data clustering and parameter selection: application to NF-[inline-formula removed]B signalling pathway induced by LPS in the presence of BFA

Developing a model for a signalling pathway requires several iterations of experimentation and model refinement to obtain an accurate model. However, the implementation of such an approach to model a signalling pathway induced by a poorly-known stimulus can become labour intensive because only limited information on the pathway is available beforehand to formulate an initial model. Therefore, a large number of iterations are required since the initial model is likely to be erroneous. In this work, a numerical scheme is proposed to construct a time-varying model for a signalling pathway induced by a poorly-known stimulus when its nominal model is available in the literature. Here, the nominal model refers to one that describes the signalling dynamics under a well-characterised stimulus. First, global sensitivity analysis is implemented on the nominal model to identify the most important parameters, which are assumed to be piecewise constants. Second, measurement data are clustered to determine temporal subdomains where the parameters take different values. Finally, a least-squares problem is solved to estimate the parameter values in each temporal subdomain. The effectiveness of this approach is illustrated by developing a time-varying model for NF-[inline-formula removed]B signalling dynamics induced by lipopolysaccharide in the presence of brefeldin A.

Parameter identification for a stochastic SEIRS epidemic model: case study influenza

A recent parameter identification technique, the local lagged adapted generalized method of moments, is used to identify the time-dependent disease transmission rate and time-dependent noise for the stochastic susceptible, exposed, infectious, temporarily immune, susceptible disease model (SEIRS) with vital rates. The stochasticity appears in the model due to fluctuations in the time-dependent transmission rate of the disease. All other parameter values are assumed to be fixed, known constants. The method is demonstrated with US influenza data from the 2004-2005 through 2016-2017 influenza seasons. The transmission rate and noise intensity stochastically work together to generate the yearly peaks in infections. The local lagged adapted generalized method of moments is tested for forecasting ability. Forecasts are made for the 2016-2017 influenza season and for infection data in year 2017. The forecast method qualitatively matches a single influenza season. Confidence intervals are given for possible future infectious levels.

Recursive model identification for the analysis of the autonomic response to exercise testing in Brugada syndrome

This paper proposes the integration and analysis of a closed-loop model of the baroreflex and cardiovascular systems, focused on a time-varying estimation of the autonomic modulation of heart rate in Brugada syndrome (BS), during exercise and subsequent recovery. Patient-specific models of 44 BS patients at different levels of risk (symptomatic and asymptomatic) were identified through a recursive evolutionary algorithm. After parameter identification, a close match between experimental and simulated signals (mean error = 0.81%) was observed. The model-based estimation of vagal and sympathetic contributions were consistent with physiological knowledge, enabling to observe the expected autonomic changes induced by exercise testing. In particular, symptomatic patients presented a significantly higher parasympathetic activity during exercise, and an autonomic imbalance was observed in these patients at peak effort and during post-exercise recovery. A higher vagal modulation during exercise, as well as an increasing parasympathetic activity at peak effort and a decreasing vagal contribution during post-exercise recovery could be related with symptoms and, thus, with a worse prognosis in BS. This work proposes the first evaluation of the sympathetic and parasympathetic responses to exercise testing in patients suffering from BS, through the recursive identification of computational models; highlighting important trends of clinical relevance that provide new insights into the underlying autonomic mechanisms regulating the cardiovascular system in BS. The joint analysis of the extracted autonomic parameters and classic electrophysiological markers could improve BS risk stratification.

Extending existing structural identifiability analysis methods to mixed-effects models

The concept of structural identifiability for state-space models is expanded to cover mixed-effects state-space models. Two methods applicable for the analytical study of the structural identifiability of mixed-effects models are presented. The two methods are based on previously established techniques for non-mixed-effects models; namely the Taylor series expansion and the input-output form approach. By generating an exhaustive summary, and by assuming an infinite number of subjects, functions of random variables can be derived which in turn determine the distribution of the system's observation function(s). By considering the uniqueness of the analytical statistical moments of the derived functions of the random variables, the structural identifiability of the corresponding mixed-effects model can be determined. The two methods are applied to a set of examples of mixed-effects models to illustrate how they work in practice.

A confidence building exercise in data and identifiability: Modeling cancer chemotherapy as a case study

Mathematical modeling has a long history in the field of cancer therapeutics, and there is increasing recognition that it can help uncover the mechanisms that underlie tumor response to treatment. However, making quantitative predictions with such models often requires parameter estimation from data, raising questions of parameter identifiability and estimability. Even in the case of structural (theoretical) identifiability, imperfect data and the resulting practical unidentifiability of model parameters can make it difficult to infer the desired information, and in some cases, to yield biologically correct inferences and predictions. Here, we examine parameter identifiability and estimability using a case study of two compartmental, ordinary differential equation models of cancer treatment with drugs that are cell cycle-specific (taxol) as well as non-specific (oxaliplatin). We proceed through model building, structural identifiability analysis, parameter estimation, practical identifiability analysis and its biological implications, as well as alternative data collection protocols and experimental designs that render the model identifiable. We use the differential algebra/input-output relationship approach for structural identifiability, and primarily the profile likelihood approach for practical identifiability. Despite the models being structurally identifiable, we show that without consideration of practical identifiability, incorrect cell cycle distributions can be inferred, that would result in suboptimal therapeutic choices. We illustrate the usefulness of estimating practically identifiable combinations (in addition to the more typically considered structurally identifiable combinations) in generating biologically meaningful insights. We also use simulated data to evaluate how the practical identifiability of the model would change under alternative experimental designs. These results highlight the importance of understanding the underlying mechanisms rather than purely using parsimony or information criteria/goodness-of-fit to decide model selection questions. The overall roadmap for identifiability testing laid out here can be used to help provide mechanistic insight into complex biological phenomena, reduce experimental costs, and optimize model-driven experimentation.

A Systematic Approach to Determining the Identifiability of Multistage Carcinogenesis Models

Multistage clonal expansion (MSCE) models of carcinogenesis are continuous-time Markov process models often used to relate cancer incidence to biological mechanism. Identifiability analysis determines what model parameter combinations can, theoretically, be estimated from given data. We use a systematic approach, based on differential algebra methods traditionally used for deterministic ordinary differential equation (ODE) models, to determine identifiable combinations for a generalized subclass of MSCE models with any number of preinitation stages and one clonal expansion. Additionally, we determine the identifiable combinations of the generalized MSCE model with up to four clonal expansion stages, and conjecture the results for any number of clonal expansion stages. The results improve upon previous work in a number of ways and provide a framework to find the identifiable combinations for further variations on the MSCE models. Finally, our approach, which takes advantage of the Kolmogorov backward equations for the probability generating functions of the Markov process, demonstrates that identifiability methods used in engineering and mathematics for systems of ODEs can be applied to continuous-time Markov processes.

Parameter Identifiability in Statistical Machine Learning: A Review

This review examines the relevance of parameter identifiability for statistical models used in machine learning. In addition to defining main concepts, we address several issues of identifiability closely related to machine learning, showing the advantages and disadvantages of state-of-the-art research and demonstrating recent progress. First, we review criteria for determining the parameter structure of models from the literature. This has three related issues: parameter identifiability, parameter redundancy, and reparameterization. Second, we review the deep influence of identifiability on various aspects of machine learning from theoretical and application viewpoints. In addition to illustrating the utility and influence of identifiability, we emphasize the interplay among identifiability theory, machine learning, mathematical statistics, information theory, optimization theory, information geometry, Riemann geometry, symbolic computation, Bayesian inference, algebraic geometry, and others. Finally, we present a new perspective together with the associated challenges.

Delineating parameter unidentifiabilities in complex models

Scientists use mathematical modeling as a tool for understanding and predicting the properties of complex physical systems. In highly parametrized models there often exist relationships between parameters over which model predictions are identical, or nearly identical. These are known as structural or practical unidentifiabilities, respectively. They are hard to diagnose and make reliable parameter estimation from data impossible. They furthermore imply the existence of an underlying model simplification. We describe a scalable method for detecting unidentifiabilities, as well as the functional relations defining them, for generic models. This allows for model simplification, and appreciation of which parameters (or functions thereof) cannot be estimated from data. Our algorithm can identify features such as redundant mechanisms and fast time-scale subsystems, as well as the regimes in parameter space over which such approximations are valid. We base our algorithm on a quantification of regional parametric sensitivity that we call 'multiscale sloppiness'. Traditionally, the link between parametric sensitivity and the conditioning of the parameter estimation problem is made locally, through the Fisher information matrix. This is valid in the regime of infinitesimal measurement uncertainty. We demonstrate the duality between multiscale sloppiness and the geometry of confidence regions surrounding parameter estimates made where measurement uncertainty is non-negligible. Further theoretical relationships are provided linking multiscale sloppiness to the likelihood-ratio test. From this, we show that a local sensitivity analysis (as typically done) is insufficient for determining the reliability of parameter estimation, even with simple (non)linear systems. Our algorithm can provide a tractable alternative. We finally apply our methods to a large-scale, benchmark systems biology model of necrosis factor (NF)-κB, uncovering unidentifiabilities.

Structural and practical identifiability analysis of S-system

In the field of systems biology, biological reaction networks are usually modelled by ordinary differential equations. A sub-class, the S-systems representation, is a widely used form of modelling. Existing S-systems identification techniques assume that the system itself is always structurally identifiable. However, due to practical limitations, biological reaction networks are often only partially measured. In addition, the captured data only covers a limited trajectory, therefore data can only be considered as a local snapshot of the system responses with respect to the complete set of state trajectories over the entire state space. Hence the estimated model can only reflect partial system dynamics and may not be unique. To improve the identification quality, the structural and practical identifiablility of S-system are studied. The S-system is shown to be identifiable under a set of assumptions. Then, an application on yeast fermentation pathway was conducted. Two case studies were chosen; where the first case is based on a larger state trajectories and the second case is based on a smaller one. By expanding the dataset which span a relatively larger state space, the uncertainty of the estimated system can be reduced. The results indicated that initial concentration is related to the practical identifiablity.

Higher-order Lie symmetries in identifiability and predictability analysis of dynamic models

Parameter estimation in ordinary differential equations (ODEs) has manifold applications not only in physics but also in the life sciences. When estimating the ODE parameters from experimentally observed data, the modeler is frequently concerned with the question of parameter identifiability. The source of parameter nonidentifiability is tightly related to Lie group symmetries. In the present work, we establish a direct search algorithm for the determination of admitted Lie group symmetries. We clarify the relationship between admitted symmetries and parameter nonidentifiability. The proposed algorithm is applied to illustrative toy models as well as a data-based ODE model of the NFκB signaling pathway. We find that besides translations and scaling transformations also higher-order transformations play a role. Enabled by the knowledge about the explicit underlying symmetry transformations, we show how models with nonidentifiable parameters can still be employed to make reliable predictions.

A novel protocol for model calibration in biological wastewater treatment

Activated sludge models (ASMs) have been widely used for process design, operation and optimization in wastewater treatment plants. However, it is still a challenge to achieve an efficient calibration for reliable application by using the conventional approaches. Hereby, we propose a novel calibration protocol, i.e. Numerical Optimal Approaching Procedure (NOAP), for the systematic calibration of ASMs. The NOAP consists of three key steps in an iterative scheme flow: i) global factors sensitivity analysis for factors fixing; ii) pseudo-global parameter correlation analysis for non-identifiable factors detection; and iii) formation of a parameter subset through an estimation by using genetic algorithm. The validity and applicability are confirmed using experimental data obtained from two independent wastewater treatment systems, including a sequencing batch reactor and a continuous stirred-tank reactor. The results indicate that the NOAP can effectively determine the optimal parameter subset and successfully perform model calibration and validation for these two different systems. The proposed NOAP is expected to use for automatic calibration of ASMs and be applied potentially to other ordinary differential equations models.

A Parameter Estimation Method for Biological Systems modelled by ODE/DDE Models Using Spline Approximation and Differential Evolution Algorithm

The inverse problem of identifying unknown parameters of known structure dynamical biological systems, which are modelled by ordinary differential equations or delay differential equations, from experimental data is treated in this paper. A two stage approach is adopted: first, combine spline theory and Nonlinear Programming (NLP), the parameter estimation problem is formulated as an optimization problem with only algebraic constraints; then, a new differential evolution (DE) algorithm is proposed to find a feasible solution. The approach is designed to handle problem of realistic size with noisy observation data. Three cases are studied to evaluate the performance of the proposed algorithm: two are based on benchmark models with priori-determined structure and parameters; the other one is a particular biological system with unknown model structure. In the last case, only a set of observation data available and in this case a nominal model is adopted for the identification. All the test systems were successfully identified by using a reasonable amount of experimental data within an acceptable computation time. Experimental evaluation reveals that the proposed method is capable of fast estimation on the unknown parameters with good precision.

Extended space method for parameter identifiability of DAE systems

Mathematical models of physical systems often have parameters that must be identified from physical data. This makes the analysis of the parameter identifiability of the given model system an essential prerequisite. Thus far, several methods have been proposed for analyzing the parameter identifiability of ordinary differential equation (ODE) systems. But, to the best of our knowledge, the parameter identifiability of differential algebraic equation (DAE) systems has scarcely been analyzed as a specific topic. Traditional differential algebraic (DA) methods developed for ODE systems are often applied directly on DAE systems. These methods, however, are not always applicable, e.g., when the prime ideal condition is not satisfied by a DAE system. In this paper, we propose a novel method to analyze the identifiability of DAE systems, based on the concept of space extension, through which the algebraic and differential variables can be decoupled. Furthermore, an inherent, low-dimensional, regular ODE system can be obtained, which is the external equivalent of the original DAE system. Subsequently, the differential algebraic (DA) method can then be used to analyze the identifiability of the low-dimension ODE system. Theoretical analysis is also presented for the proposed method. Two examples, including a simplified interaction model and an isothermal reactor system, are presented to illustrate the detailed steps and effectiveness of the proposed method.

Determining identifiable parameter combinations using subset profiling

Identifiability is a necessary condition for successful parameter estimation of dynamic system models. A major component of identifiability analysis is determining the identifiable parameter combinations, the functional forms for the dependencies between unidentifiable parameters. Identifiable combinations can help in model reparameterization and also in determining which parameters may be experimentally measured to recover model identifiability. Several numerical approaches to determining identifiability of differential equation models have been developed, however the question of determining identifiable combinations remains incompletely addressed. In this paper, we present a new approach which uses parameter subset selection methods based on the Fisher Information Matrix, together with the profile likelihood, to effectively estimate identifiable combinations. We demonstrate this approach on several example models in pharmacokinetics, cellular biology, and physiology.

Structural identifiability analyses of candidate models for in vitro Pitavastatin hepatic uptake

In this paper a review of the application of four different techniques (a version of the similarity transformation approach for autonomous uncontrolled systems, a non-differential input/output observable normal form approach, the characteristic set differential algebra and a recent algebraic input/output relationship approach) to determine the structural identifiability of certain in vitro nonlinear pharmacokinetic models is provided. The Organic Anion Transporting Polypeptide (OATP) substrate, Pitavastatin, is used as a probe on freshly isolated animal and human hepatocytes. Candidate pharmacokinetic non-linear compartmental models have been derived to characterise the uptake process of Pitavastatin. As a prerequisite to parameter estimation, structural identifiability analyses are performed to establish that all unknown parameters can be identified from the experimental observations available.

Kinetic models in industrial biotechnology - Improving cell factory performance

An increasing number of industrial bioprocesses capitalize on living cells by using them as cell factories that convert sugars into chemicals. These processes range from the production of bulk chemicals in yeasts and bacteria to the synthesis of therapeutic proteins in mammalian cell lines. One of the tools in the continuous search for improved performance of such production systems is the development and application of mathematical models. To be of value for industrial biotechnology, mathematical models should be able to assist in the rational design of cell factory properties or in the production processes in which they are utilized. Kinetic models are particularly suitable towards this end because they are capable of representing the complex biochemistry of cells in a more complete way compared to most other types of models. They can, at least in principle, be used to in detail understand, predict, and evaluate the effects of adding, removing, or modifying molecular components of a cell factory and for supporting the design of the bioreactor or fermentation process. However, several challenges still remain before kinetic modeling will reach the degree of maturity required for routine application in industry. Here we review the current status of kinetic cell factory modeling. Emphasis is on modeling methodology concepts, including model network structure, kinetic rate expressions, parameter estimation, optimization methods, identifiability analysis, model reduction, and model validation, but several applications of kinetic models for the improvement of cell factories are also discussed.

An Identifiable State Model To Describe Light Intensity Influence on Microalgae Growth

Despite the high potential as feedstock for the production of fuels and chemicals, the industrial cultivation of microalgae still exhibits many issues. Yield in microalgae cultivation systems is limited by the solar energy that can be harvested. The availability of reliable models representing key phenomena affecting algae growth may help designing and optimizing effective production systems at an industrial level. In this work the complex influence of different light regimes on seawater alga Nannochloropsis salina growth is represented by first principles models. Experimental data such as in vivo fluorescence measurements are employed to develop the model. The proposed model allows description of all growth curves and fluorescence data in a reliable way. The model structure is assessed and modified in order to guarantee the model identifiability and the estimation of its parametric set in a robust and reliable way.

Statistical identifiability and convergence evaluation for nonlinear pharmacokinetic models with particle swarm optimization

The statistical identifiability of nonlinear pharmacokinetic (PK) models with the Michaelis-Menten (MM) kinetic equation is considered using a global optimization approach, which is particle swarm optimization (PSO). If a model is statistically non-identifiable, the conventional derivative-based estimation approach is often terminated earlier without converging, due to the singularity. To circumvent this difficulty, we develop a derivative-free global optimization algorithm by combining PSO with a derivative-free local optimization algorithm to improve the rate of convergence of PSO. We further propose an efficient approach to not only checking the convergence of estimation but also detecting the identifiability of nonlinear PK models. PK simulation studies demonstrate that the convergence and identifiability of the PK model can be detected efficiently through the proposed approach. The proposed approach is then applied to clinical PK data along with a two-compartmental model.

A general model-based design of experiments approach to achieve practical identifiability of pharmacokinetic and pharmacodynamic models

The use of pharmacokinetic (PK) and pharmacodynamic (PD) models is a common and widespread practice in the preliminary stages of drug development. However, PK-PD models may be affected by structural identifiability issues intrinsically related to their mathematical formulation. A preliminary structural identifiability analysis is usually carried out to check if the set of model parameters can be uniquely determined from experimental observations under the ideal assumptions of noise-free data and no model uncertainty. However, even for structurally identifiable models, real-life experimental conditions and model uncertainty may strongly affect the practical possibility to estimate the model parameters in a statistically sound way. A systematic procedure coupling the numerical assessment of structural identifiability with advanced model-based design of experiments formulations is presented in this paper. The objective is to propose a general approach to design experiments in an optimal way, detecting a proper set of experimental settings that ensure the practical identifiability of PK-PD models. Two simulated case studies based on in vitro bacterial growth and killing models are presented to demonstrate the applicability and generality of the methodology to tackle model identifiability issues effectively, through the design of feasible and highly informative experiments.

The input-output relationship approach to structural identifiability analysis

Analysis of the identifiability of a given model system is an essential prerequisite to the determination of model parameters from physical data. However, the tools available for the analysis of non-linear systems can be limited both in applicability and by computational intractability for any but the simplest of models. The input-output relation of a model summarises the input-output structure of the whole system and as such provides the potential for an alternative approach to this analysis. However for this approach to be valid it is necessary to determine whether the monomials of a differential polynomial are linearly independent. A simple test for this property is presented in this work. The derivation and analysis of this relation can be implemented symbolically within Maple. These techniques are applied to analyse classical models from biomedical systems modelling and those of enzyme catalysed reaction schemes.

Structural identifiability analysis and reparameterisation (parameter reduction) of a cardiovascular feedback model

Structural identifiability should be considered when developing mathematical models. A globally or at least locally identifiable model has to be obtained in order to have some chance of obtaining unique parameter estimates when real data are available. An indicator of structural unidentifiability may be that some unknown parameter estimates are found to be not well determined from parameter estimation of a model. An example is discussed in this paper to illustrate the procedures involved when such situations arise. Problems with parameter estimation were observed for a PKPD model for an α1A/1L-adrenoceptor partial agonist developed for the treatment of stress urinary incontinence The regulation of the side effects of the increased peripheral resistance, induced by the constriction of the blood vessels, was modelled by adapting a previous cardiovascular nonlinear PKPD model proposed by Franchetau and co-workers. Structural identifiability analysis confirmed that the model was unidentifiable. The model was then reparameterised (parameter list reduction) to obtain a globally identifiable model. Simulation studies confirm the superiority of the reduced parameterisation with respect to parameter estimation. The simulation study also confirms the models behave indistinguishably with respect to the input-output behaviour. The example demonstrates the importance of recognising an unidentifiable model and illustrates step by step identifiability analysis, reparameterisation and validation of reparameterised model against the original model.

Structural identifiability of systems biology models: a critical comparison of methods

Analysing the properties of a biological system through in silico experimentation requires a satisfactory mathematical representation of the system including accurate values of the model parameters. Fortunately, modern experimental techniques allow obtaining time-series data of appropriate quality which may then be used to estimate unknown parameters. However, in many cases, a subset of those parameters may not be uniquely estimated, independently of the experimental data available or the numerical techniques used for estimation. This lack of identifiability is related to the structure of the model, i.e. the system dynamics plus the observation function. Despite the interest in knowing a priori whether there is any chance of uniquely estimating all model unknown parameters, the structural identifiability analysis for general non-linear dynamic models is still an open question. There is no method amenable to every model, thus at some point we have to face the selection of one of the possibilities. This work presents a critical comparison of the currently available techniques. To this end, we perform the structural identifiability analysis of a collection of biological models. The results reveal that the generating series approach, in combination with identifiability tableaus, offers the most advantageous compromise among range of applicability, computational complexity and information provided.

Structural identifiability and indistinguishability analyses of the minimal model and a euglycemic hyperinsulinemic clamp model for glucose-insulin dynamics

Many mathematical models have been developed to describe glucose-insulin kinetics as a means of analysing the effective control of diabetes. This paper concentrates on the structural identifiability analysis of certain well-established mathematical models that have been developed to characterise glucose-insulin kinetics under different experimental scenarios. Such analysis is a pre-requisite to experiment design and parameter estimation and is applied for the first time to these models with the specific structures considered. The analysis is applied to a basic (original) form of the Minimal Model (MM) using the Taylor Series approach and a now well-accepted extended form of the MM by application of the Taylor Series approach and a form of the Similarity Transformation approach. Due to the established inappropriate nature of the MM with regard to glucose clamping experiments an alternative model describing the glucose-insulin dynamics during a Euglycemic Hyperinsulinemic Clamp (EIC) experiment was considered. Structural identifiability analysis of the EIC model is also performed using the Taylor Series approach and shows that, with glucose infusion as input alone, the model is structurally globally identifiable. Additional analysis demonstrates that the two different model forms are structurally distinguishable for observation of both glucose and insulin.

Indistinguishability and identifiability of kinetic models for the MurC reaction in peptidoglycan biosynthesis

An important question in Systems Biology is the design of experiments that enable discrimination between two (or more) competing chemical pathway models or biological mechanisms. In this paper analysis is performed between two different models describing the kinetic mechanism of a three-substrate three-product reaction, namely the MurC reaction in the cytoplasmic phase of peptidoglycan biosynthesis. One model involves ordered substrate binding and ordered release of the three products; the competing model also assumes ordered substrate binding, but with fast release of the three products. The two versions are shown to be distinguishable; however, if standard quasi-steady-state assumptions are made distinguishability cannot be determined. Once model structure uniqueness is ensured the experimenter must determine if it is possible to successfully recover rate constant values given the experiment observations, a process known as structural identifiability. Structural identifiability analysis is carried out for both models to determine which of the unknown reaction parameters can be determined uniquely, or otherwise, from the ideal system outputs. This structural analysis forms an integrated step towards the modelling of the full pathway of the cytoplasmic phase of peptidoglycan biosynthesis.

Compartmental modelling of the pharmacokinetics of a breast cancer resistance protein

A mathematical model for the pharmacokinetics of Hoechst 33342 following administration into a culture medium containing a population of transfected cells (HEK293 hBCRP) with a potent breast cancer resistance protein inhibitor, Fumitremorgin C (FTC), present is described. FTC is reported to almost completely annul resistance mediated by BCRP in vitro. This non-linear compartmental model has seven macroscopic sub-units, with 14 rate parameters. It describes the relationship between the concentration of Hoechst 33342 and FTC, initially spiked in the medium, and the observed change in fluorescence due to Hoechst 33342 binding to DNA. Structural identifiability analysis has been performed using two methods, one based on the similarity transformation/exhaustive modelling approach and the other based on the differential algebra approach. The analyses demonstrated that all models derived are uniquely identifiable for the experiments/observations available. A kinetic modelling software package, namely FACSIMILE (MPCA Software, UK), was used for parameter fitting and to obtain numerical solutions for the system equations. Model fits gave very good agreement with in vitro data provided by AstraZeneca across a variety of experimental scenarios.