1
|
Eilertsen J, Schnell S, Walcher S. The Michaelis-Menten Reaction at Low Substrate Concentrations: Pseudo-First-Order Kinetics and Conditions for Timescale Separation. Bull Math Biol 2024; 86:68. [PMID: 38703247 PMCID: PMC11069484 DOI: 10.1007/s11538-024-01295-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Accepted: 04/10/2024] [Indexed: 05/06/2024]
Abstract
We demonstrate that the Michaelis-Menten reaction mechanism can be accurately approximated by a linear system when the initial substrate concentration is low. This leads to pseudo-first-order kinetics, simplifying mathematical calculations and experimental analysis. Our proof utilizes a monotonicity property of the system and Kamke's comparison theorem. This linear approximation yields a closed-form solution, enabling accurate modeling and estimation of reaction rate constants even without timescale separation. Building on prior work, we establish that the sufficient condition for the validity of this approximation iss 0 ≪ K , where K = k 2 / k 1 is the Van Slyke-Cullen constant. This condition is independent of the initial enzyme concentration. Further, we investigate timescale separation within the linear system, identifying necessary and sufficient conditions and deriving the corresponding reduced one-dimensional equations.
Collapse
Affiliation(s)
- Justin Eilertsen
- Mathematical Reviews, American Mathematical Society, 416 4th Street, Ann Arbor, MI, 48103, USA
| | - Santiago Schnell
- Department of Biological Sciences and Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN, 46556, USA.
| | | |
Collapse
|
2
|
Chowell G, Bleichrodt A, Luo R. Parameter estimation and forecasting with quantified uncertainty for ordinary differential equation models using QuantDiffForecast: A MATLAB toolbox and tutorial. Stat Med 2024; 43:1826-1848. [PMID: 38378161 PMCID: PMC11031352 DOI: 10.1002/sim.10036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Revised: 01/17/2024] [Accepted: 01/31/2024] [Indexed: 02/22/2024]
Abstract
Mathematical models based on systems of ordinary differential equations (ODEs) are frequently applied in various scientific fields to assess hypotheses, estimate key model parameters, and generate predictions about the system's state. To support their application, we present a comprehensive, easy-to-use, and flexible MATLAB toolbox, QuantDiffForecast, and associated tutorial to estimate parameters and generate short-term forecasts with quantified uncertainty from dynamical models based on systems of ODEs. We provide software ( https://github.com/gchowell/paramEstimation_forecasting_ODEmodels/) and detailed guidance on estimating parameters and forecasting time-series trajectories that are characterized using ODEs with quantified uncertainty through a parametric bootstrapping approach. It includes functions that allow the user to infer model parameters and assess forecasting performance for different ODE models specified by the user, using different estimation methods and error structures in the data. The tutorial is intended for a diverse audience, including students training in dynamic systems, and will be broadly applicable to estimate parameters and generate forecasts from models based on ODEs. The functions included in the toolbox are illustrated using epidemic models with varying levels of complexity applied to data from the 1918 influenza pandemic in San Francisco. A tutorial video that demonstrates the functionality of the toolbox is included.
Collapse
Affiliation(s)
- Gerardo Chowell
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Amanda Bleichrodt
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| | - Ruiyan Luo
- Department of Population Health Sciences, School of Public Health, Georgia State University, Atlanta, GA, USA
| |
Collapse
|
3
|
Vicente-Martínez J, Bonmatí-Carrión MÁ, Madrid JA, Rol MA. Uncovering personal circadian responses to light through particle swarm optimization. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 243:107933. [PMID: 38006683 DOI: 10.1016/j.cmpb.2023.107933] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Revised: 10/02/2023] [Accepted: 11/17/2023] [Indexed: 11/27/2023]
Abstract
BACKGROUND AND OBJECTIVES Kronauer's oscillator model of the human central pacemaker is one of the most commonly used approaches to study the human circadian response to light. Two sources of error when applying it to a personal light exposure have been identified: (1) as a populational model, it does not consider inter-individual variability, and (2) the initial conditions needed to integrate the model are usually unknown, and thus subjectively estimated. In this work, we evaluate the ability of particle swarm optimization (PSO) algorithms to simultaneously uncover the optimal initial conditions and individual parameters of a pre-defined Kronauer's oscillator model. METHODS A Canonical PSO, a Dynamic Multi-Swarm PSO and a novel modification of the latter, namely Hierarchical Dynamic Multi-Swarm PSO, are evaluated. Two different target models (under a regular and an irregular schedule) are defined, and the same realistic light profile is fed to them. Based on their output, a fitness function is proposed, which is minimized by the algorithms to find the optimum set of parameters and initial conditions of the model. RESULTS We demonstrate that Dynamic Multi-Swarm and Hierarchical Dynamic Multi-Swarm algorithms can accurately uncover personal circadian parameters under both regular and irregular schedules, but as expected, optimization is easier under a regular schedule. Circadian parameters play the most important role in the optimization process and should be prioritized over initial conditions, although assessment of the impact of misestimating the latter is recommended. The log-log linear relationship between mean absolute error and computational cost shows that the number of particles to use is at the discretion of the user. CONCLUSIONS The robustness and low errors achieved by the algorithms support their further testing, validation and systematic application to empirical data under a regular or irregular schedule. Uncovering personal circadian parameters can improve the assessment of the circadian status of a person and the applicability of personalized light therapies, as well as help to discover other factors that may lie behind the interindividual variability in the circadian response to light.
Collapse
Affiliation(s)
- Jesús Vicente-Martínez
- Chronobiology Laboratory, Department of Physiology, College of Biology, University of Murcia, Mare Nostrum Campus, IUIE, IMIB-Arrixaca, Murcia 30100, Spain; Ciber Fragilidad y Envejecimiento Saludable, Instituto de Salud Carlos III, Madrid 28029, Spain
| | - María Ángeles Bonmatí-Carrión
- Chronobiology Laboratory, Department of Physiology, College of Biology, University of Murcia, Mare Nostrum Campus, IUIE, IMIB-Arrixaca, Murcia 30100, Spain; Ciber Fragilidad y Envejecimiento Saludable, Instituto de Salud Carlos III, Madrid 28029, Spain.
| | - Juan Antonio Madrid
- Chronobiology Laboratory, Department of Physiology, College of Biology, University of Murcia, Mare Nostrum Campus, IUIE, IMIB-Arrixaca, Murcia 30100, Spain; Ciber Fragilidad y Envejecimiento Saludable, Instituto de Salud Carlos III, Madrid 28029, Spain
| | - Maria Angeles Rol
- Chronobiology Laboratory, Department of Physiology, College of Biology, University of Murcia, Mare Nostrum Campus, IUIE, IMIB-Arrixaca, Murcia 30100, Spain; Ciber Fragilidad y Envejecimiento Saludable, Instituto de Salud Carlos III, Madrid 28029, Spain
| |
Collapse
|
4
|
Bortz DM, Messenger DA, Dukic V. Direct Estimation of Parameters in ODE Models Using WENDy: Weak-Form Estimation of Nonlinear Dynamics. Bull Math Biol 2023; 85:110. [PMID: 37796411 PMCID: PMC10556152 DOI: 10.1007/s11538-023-01208-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2023] [Accepted: 08/30/2023] [Indexed: 10/06/2023]
Abstract
We introduce the Weak-form Estimation of Nonlinear Dynamics (WENDy) method for estimating model parameters for non-linear systems of ODEs. Without relying on any numerical differential equation solvers, WENDy computes accurate estimates and is robust to large (biologically relevant) levels of measurement noise. For low dimensional systems with modest amounts of data, WENDy is competitive with conventional forward solver-based nonlinear least squares methods in terms of speed and accuracy. For both higher dimensional systems and stiff systems, WENDy is typically both faster (often by orders of magnitude) and more accurate than forward solver-based approaches. The core mathematical idea involves an efficient conversion of the strong form representation of a model to its weak form, and then solving a regression problem to perform parameter inference. The core statistical idea rests on the Errors-In-Variables framework, which necessitates the use of the iteratively reweighted least squares algorithm. Further improvements are obtained by using orthonormal test functions, created from a set of [Formula: see text] bump functions of varying support sizes.We demonstrate the high robustness and computational efficiency by applying WENDy to estimate parameters in some common models from population biology, neuroscience, and biochemistry, including logistic growth, Lotka-Volterra, FitzHugh-Nagumo, Hindmarsh-Rose, and a Protein Transduction Benchmark model. Software and code for reproducing the examples is available at https://github.com/MathBioCU/WENDy .
Collapse
Affiliation(s)
- David M Bortz
- Department of Applied Mathematics, University of Colorado, Boulder, CO 80309-0526, USA.
| | - Daniel A Messenger
- Department of Applied Mathematics, University of Colorado, Boulder, CO 80309-0526, USA
| | - Vanja Dukic
- Department of Applied Mathematics, University of Colorado, Boulder, CO 80309-0526, USA
| |
Collapse
|
5
|
Wang B, Wang L, Peng J, Hong M, Xu W. The identification of piecewise non-linear dynamical system without understanding the mechanism. CHAOS (WOODBURY, N.Y.) 2023; 33:2894488. [PMID: 37276571 DOI: 10.1063/5.0147847] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Accepted: 05/16/2023] [Indexed: 06/07/2023]
Abstract
This paper designs an algorithm to distill the piecewise non-linear dynamical system from the data without prior knowledge. The system to be identified does not have to be written as a known model term or be thoroughly understood. We exploit the fact that an unknown piecewise non-linear system can be decomposed into the Fourier series as long as its equations of motion are Riemann integrable. Based on this property, we reduce the challenge of finding the correct model to discovering the Fourier series approximation. However, the Fourier series approximation of the piecewise function is inaccurate. The new method takes advantage of this weakness to determine whether the model has piecewise features and to find a way to discover the discontinuity set. Then, the dynamical system on each segment is identified as a pure Fourier series. Identification of intricate models can be achieved in simple steps. The results show that the method can accurately discover the equation of motion and precisely capture the non-smooth characteristic. Next, the prediction and further detailed analysis can be carried out.
Collapse
Affiliation(s)
- Bochen Wang
- Department of Applied Probability and Statistics, School of Mathematics and Statistics, Northwestern Polytechnical University, Xi'an 710129, Peoples Republic of China
| | - Liang Wang
- Department of Applied Probability and Statistics, School of Mathematics and Statistics, Northwestern Polytechnical University, Xi'an 710129, Peoples Republic of China
| | - Jiahui Peng
- Department of Applied Probability and Statistics, School of Mathematics and Statistics, Northwestern Polytechnical University, Xi'an 710129, Peoples Republic of China
| | - Mingyue Hong
- Department of Applied Probability and Statistics, School of Mathematics and Statistics, Northwestern Polytechnical University, Xi'an 710129, Peoples Republic of China
| | - Wei Xu
- Department of Applied Probability and Statistics, School of Mathematics and Statistics, Northwestern Polytechnical University, Xi'an 710129, Peoples Republic of China
| |
Collapse
|
6
|
Bortz DM, Messenger DA, Dukic V. Direct Estimation of Parameters in ODE Models Using WENDy: Weak-form Estimation of Nonlinear Dynamics. ARXIV 2023:arXiv:2302.13271v3. [PMID: 36911272 PMCID: PMC10002818] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 04/13/2023]
Abstract
We introduce the Weak-form Estimation of Nonlinear Dynamics (WENDy) method for estimating model parameters for non-linear systems of ODEs. Without relying on any numerical differential equation solvers, WENDy computes accurate estimates and is robust to large (biologically relevant) levels of measurement noise. For low dimensional systems with modest amounts of data, WENDy is competitive with conventional forward solver-based nonlinear least squares methods in terms of speed and accuracy. For both higher dimensional systems and stiff systems, WENDy is typically both faster (often by orders of magnitude) and more accurate than forward solver-based approaches. The core mathematical idea involves an efficient conversion of the strong form representation of a model to its weak form, and then solving a regression problem to perform parameter inference. The core statistical idea rests on the Errors-In-Variables framework, which necessitates the use of the iteratively reweighted least squares algorithm. Further improvements are obtained by using orthonormal test functions, created from a set of C-infinity bump functions of varying support sizes. We demonstrate the high robustness and computational efficiency by applying WENDy to estimate parameters in some common models from population biology, neuroscience, and biochemistry, including logistic growth, Lotka-Volterra, FitzHugh-Nagumo, Hindmarsh-Rose, and a Protein Transduction Benchmark model. Software and code for reproducing the examples is available at (https://github.com/MathBioCU/WENDy).
Collapse
Affiliation(s)
- David M Bortz
- Department of Applied Mathematics, University of Colorado, Boulder, CO 80309-0526
| | | | | |
Collapse
|
7
|
Son D, Kim J. Estimation of Ordinary Differential Equation Models for Gene Regulatory Networks Through Data Cloning. J Comput Biol 2023; 30:609-618. [PMID: 36898058 DOI: 10.1089/cmb.2022.0201] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/12/2023] Open
Abstract
Ordinary differential equations (ODEs) are widely used for elucidating dynamic processes in various fields. One of the applications of ODEs is to describe dynamics of gene regulatory networks (GRNs), which is a critical step in understanding disease mechanisms. However, estimation of ODE models for GRNs is challenging because of inflexibility of the model and noisy data with complex error structures such as heteroscedasticity, correlations between genes, and time dependency. In addition, either a likelihood or Bayesian approach is commonly used for estimation of ODE models, but both approaches have benefits and drawbacks in their own right. Data cloning is a maximum likelihood (ML) estimation method through the Bayesian framework. Since it works in the Bayesian framework, it is free from local optimum problems that are common drawbacks of ML methods. Also, its inference is invariant for the selection of prior distributions, which is a major issue in Bayesian methods. This study proposes an estimation method of ODE models for GRNs through data cloning. The proposed method is demonstrated through simulation and it is applied to real gene expression time-course data.
Collapse
Affiliation(s)
- Donghui Son
- Department of Statistics, Sungkyunkwan University, Seoul, South Korea
| | - Jaejik Kim
- Department of Statistics, Sungkyunkwan University, Seoul, South Korea
| |
Collapse
|
8
|
Browning AP, Drovandi C, Turner IW, Jenner AL, Simpson MJ. Efficient inference and identifiability analysis for differential equation models with random parameters. PLoS Comput Biol 2022; 18:e1010734. [PMID: 36441811 PMCID: PMC9731444 DOI: 10.1371/journal.pcbi.1010734] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Revised: 12/08/2022] [Accepted: 11/14/2022] [Indexed: 11/29/2022] Open
Abstract
Heterogeneity is a dominant factor in the behaviour of many biological processes. Despite this, it is common for mathematical and statistical analyses to ignore biological heterogeneity as a source of variability in experimental data. Therefore, methods for exploring the identifiability of models that explicitly incorporate heterogeneity through variability in model parameters are relatively underdeveloped. We develop a new likelihood-based framework, based on moment matching, for inference and identifiability analysis of differential equation models that capture biological heterogeneity through parameters that vary according to probability distributions. As our novel method is based on an approximate likelihood function, it is highly flexible; we demonstrate identifiability analysis using both a frequentist approach based on profile likelihood, and a Bayesian approach based on Markov-chain Monte Carlo. Through three case studies, we demonstrate our method by providing a didactic guide to inference and identifiability analysis of hyperparameters that relate to the statistical moments of model parameters from independent observed data. Our approach has a computational cost comparable to analysis of models that neglect heterogeneity, a significant improvement over many existing alternatives. We demonstrate how analysis of random parameter models can aid better understanding of the sources of heterogeneity from biological data.
Collapse
Affiliation(s)
- Alexander P. Browning
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Australia
- QUT Centre for Data Science, Queensland University of Technology, Brisbane, Australia
- Mathematical Institute, University of Oxford, Oxford, United Kingdom
| | - Christopher Drovandi
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Australia
- QUT Centre for Data Science, Queensland University of Technology, Brisbane, Australia
| | - Ian W. Turner
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Australia
| | - Adrianne L. Jenner
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Australia
- QUT Centre for Data Science, Queensland University of Technology, Brisbane, Australia
| | - Matthew J. Simpson
- School of Mathematical Sciences, Queensland University of Technology, Brisbane, Australia
- QUT Centre for Data Science, Queensland University of Technology, Brisbane, Australia
- * E-mail:
| |
Collapse
|
9
|
Huang H. Bayesian multi‐level mixed‐effects model for influenza dynamics. J R Stat Soc Ser C Appl Stat 2022. [DOI: 10.1111/rssc.12603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Affiliation(s)
- Hanwen Huang
- Department of Epidemiology and Biostatistics University of Georgia Athens Georgia USA
| |
Collapse
|
10
|
Dubey P, Chen Y, Gajardo Á, Bhattacharjee S, Carroll C, Zhou Y, Chen H, Müller HG. Learning delay dynamics for multivariate stochastic processes, with application to the prediction of the growth rate of COVID-19 cases in the United States. JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS 2022; 514:125677. [PMID: 34642503 PMCID: PMC8494512 DOI: 10.1016/j.jmaa.2021.125677] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/11/2020] [Indexed: 06/13/2023]
Abstract
Delay differential equations form the underpinning of many complex dynamical systems. The forward problem of solving random differential equations with delay has received increasing attention in recent years. Motivated by the challenge to predict the COVID-19 caseload trajectories for individual states in the U.S., we target here the inverse problem. Given a sample of observed random trajectories obeying an unknown random differential equation model with delay, we use a functional data analysis framework to learn the model parameters that govern the underlying dynamics from the data. We show the existence and uniqueness of the analytical solutions of the population delay random differential equation model when one has discrete time delays in the functional concurrent regression model and also for a second scenario where one has a delay continuum or distributed delay. The latter involves a functional linear regression model with history index. The derivative of the process of interest is modeled using the process itself as predictor and also other functional predictors with predictor-specific delayed impacts. This dynamics learning approach is shown to be well suited to model the growth rate of COVID-19 for the states that are part of the U.S., by pooling information from the individual states, using the case process and concurrently observed economic and mobility data as predictors.
Collapse
Affiliation(s)
- Paromita Dubey
- Department of Statistics, Stanford University, United States of America
| | - Yaqing Chen
- Department of Statistics, University of California, Davis, United States of America
| | - Álvaro Gajardo
- Department of Statistics, University of California, Davis, United States of America
| | | | - Cody Carroll
- Department of Statistics, University of California, Davis, United States of America
| | - Yidong Zhou
- Department of Statistics, University of California, Davis, United States of America
| | - Han Chen
- Department of Statistics, University of California, Davis, United States of America
| | - Hans-Georg Müller
- Department of Statistics, University of California, Davis, United States of America
| |
Collapse
|
11
|
Dynamical modeling for non-Gaussian data with high-dimensional sparse ordinary differential equations. Comput Stat Data Anal 2022. [DOI: 10.1016/j.csda.2022.107483] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
|
12
|
Zhang N, Nanshan M, Cao J. A Joint estimation approach to sparse additive ordinary differential equations. STATISTICS AND COMPUTING 2022; 32:69. [PMID: 36033975 PMCID: PMC9395913 DOI: 10.1007/s11222-022-10117-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Accepted: 06/13/2022] [Indexed: 06/15/2023]
Abstract
Ordinary differential equations (ODEs) are widely used to characterize the dynamics of complex systems in real applications. In this article, we propose a novel joint estimation approach for generalized sparse additive ODEs where observations are allowed to be non-Gaussian. The new method is unified with existing collocation methods by considering the likelihood, ODE fidelity and sparse regularization simultaneously. We design a block coordinate descent algorithm for optimizing the non-convex and non-differentiable objective function. The global convergence of the algorithm is established. The simulation study and two applications demonstrate the superior performance of the proposed method in estimation and improved performance of identifying the sparse structure.
Collapse
Affiliation(s)
- Nan Zhang
- School of Data Science, Fudan University, Shanghai, China
| | - Muye Nanshan
- School of Data Science, Fudan University, Shanghai, China
| | - Jiguo Cao
- Department of Statistics and Actuarial Science, Simon Fraser University, Burnaby, Canada
| |
Collapse
|
13
|
Abstract
Ordinary differential equation (ODE) is widely used in modeling biological and physical processes in science. In this article, we propose a new reproducing kernel-based approach for estimation and inference of ODE given noisy observations. We do not assume the functional forms in ODE to be known, or restrict them to be linear or additive, and we allow pairwise interactions. We perform sparse estimation to select individual functionals, and construct confidence intervals for the estimated signal trajectories. We establish the estimation optimality and selection consistency of kernel ODE under both the low-dimensional and high-dimensional settings, where the number of unknown functionals can be smaller or larger than the sample size. Our proposal builds upon the smoothing spline analysis of variance (SS-ANOVA) framework, but tackles several important problems that are not yet fully addressed, and thus extends the scope of existing SS-ANOVA as well. We demonstrate the efficacy of our method through numerous ODE examples.
Collapse
Affiliation(s)
- Xiaowu Dai
- Department of Economics and Simons Institute for the Theory of Computing, the University of California, Berkeley, Berkeley, CA
| | - Lexin Li
- Department of Economics and Simons Institute for the Theory of Computing, the University of California, Berkeley, Berkeley, CA
- Department of Biostatistics and Epidemiology, the University of California, Berkeley, Berkeley, CA
| |
Collapse
|
14
|
Bradley W, Boukouvala F. Two-Stage Approach to Parameter Estimation of Differential Equations Using Neural ODEs. Ind Eng Chem Res 2021. [DOI: 10.1021/acs.iecr.1c00552] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Affiliation(s)
- William Bradley
- School of Chemical & Biomolecular Engineering, Georgia Institute of Technology, 311 Ferst Dr., N.W., Atlanta, Georgia 30332-0100, United States
| | - Fani Boukouvala
- School of Chemical & Biomolecular Engineering, Georgia Institute of Technology, 311 Ferst Dr., N.W., Atlanta, Georgia 30332-0100, United States
| |
Collapse
|
15
|
Semiparametric Mixed-Effects Ordinary Differential Equation Models with Heavy-Tailed Distributions. JOURNAL OF AGRICULTURAL, BIOLOGICAL AND ENVIRONMENTAL STATISTICS 2021; 26:428-445. [PMID: 33840991 PMCID: PMC8020077 DOI: 10.1007/s13253-021-00446-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/03/2019] [Revised: 02/12/2021] [Accepted: 02/24/2021] [Indexed: 11/01/2022]
Abstract
Ordinary differential equation (ODE) models are popularly used to describe complex dynamical systems. When estimating ODE parameters from noisy data, a common distribution assumption is using the Gaussian distribution. It is known that the Gaussian distribution is not robust when abnormal data exist. In this article, we develop a hierarchical semiparametric mixed-effects ODE model for longitudinal data under the Bayesian framework. For robust inference on ODE parameters, we consider a class of heavy-tailed distributions to model the random effects of ODE parameters and observations errors. An MCMC method is proposed to sample ODE parameters from the posterior distributions. Our proposed method is illustrated by studying a gene regulation experiment. Simulation studies show that our proposed method provides satisfactory results for the semiparametric mixed-effects ODE models with finite samples. Supplementary materials accompanying this paper appear online.
Collapse
|
16
|
Roesch E, Rackauckas C, Stumpf MPH. Collocation based training of neural ordinary differential equations. Stat Appl Genet Mol Biol 2021; 20:37-49. [PMID: 34237805 DOI: 10.1515/sagmb-2020-0025] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2020] [Accepted: 05/04/2021] [Indexed: 11/15/2022]
Abstract
The predictive power of machine learning models often exceeds that of mechanistic modeling approaches. However, the interpretability of purely data-driven models, without any mechanistic basis is often complicated, and predictive power by itself can be a poor metric by which we might want to judge different methods. In this work, we focus on the relatively new modeling techniques of neural ordinary differential equations. We discuss how they relate to machine learning and mechanistic models, with the potential to narrow the gulf between these two frameworks: they constitute a class of hybrid model that integrates ideas from data-driven and dynamical systems approaches. Training neural ODEs as representations of dynamical systems data has its own specific demands, and we here propose a collocation scheme as a fast and efficient training strategy. This alleviates the need for costly ODE solvers. We illustrate the advantages that collocation approaches offer, as well as their robustness to qualitative features of a dynamical system, and the quantity and quality of observational data. We focus on systems that exemplify some of the hallmarks of complex dynamical systems encountered in systems biology, and we map out how these methods can be used in the analysis of mathematical models of cellular and physiological processes.
Collapse
Affiliation(s)
- Elisabeth Roesch
- Melbourne Integrative Genomics, University of Melbourne, 30 Royal Parade, Parkville, VIC3052, Australia.,School of Mathematics and Statistics, University of Melbourne, 813 Swanston Street, Parkville, VIC3010, Australia
| | - Christopher Rackauckas
- Department of Mathematics, Massachusetts Institute of Technology, 182 Memorial Dr, Cambridge, MA02142, USA.,Julia Computing, 240 Elm Street, 2nd Floor, Somerville, Massachusetts02144, USA.,Pumas-AI, 14711 Kamputa Drive, Centerville, VA20120, USA
| | - Michael P H Stumpf
- Melbourne Integrative Genomics, University of Melbourne, 30 Royal Parade, Parkville, VIC3052, Australia.,School of Mathematics and Statistics, University of Melbourne, 813 Swanston Street, Parkville, VIC3010, Australia
| |
Collapse
|
17
|
Integral matching-based nonlinear grey Bernoulli model for forecasting the coal consumption in China. Soft comput 2021. [DOI: 10.1007/s00500-020-05521-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
18
|
|
19
|
Zhang H, Zhao X, Yin K, Yan Y, Qian W, Chen B, Fang X. Dynamic estimation of epidemiological parameters of COVID-19 outbreak and effects of interventions on its spread. J Public Health Res 2021; 10:1906. [PMID: 33849254 PMCID: PMC7967497 DOI: 10.4081/jphr.2021.1906] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2020] [Accepted: 02/25/2021] [Indexed: 01/08/2023] Open
Abstract
BACKGROUND A key challenge in estimating epidemiological parameters for a pandemic such as the initial COVID-19 outbreak in Wuhan is the discrepancy between the officially reported number of infections and the true number of infections. A common approach to tackling the challenge is to use the number of infections exported from the originating city to infer the true number. This approach can only provide a static estimate of the epidemiological parameters before city lockdown because there are almost no exported cases thereafter. METHODS We propose a Bayesian estimation method that dynamically estimates the epidemiological parameters by recovering true numbers of infections from day-to-day official numbers. To illustrate the use of this method, we provide a comprehensive retrospection on how the COVID-19 had progressed in Wuhan from January 19 to March 5, 2020. Particularly, we estimate that the outbreak sizes by January 23 and March 5 were 11,239 [95% CI 4,794-22,372] and 124,506 [95% CI 69,526-265,113], respectively. RESULTS The effective reproduction number attained its maximum on January 24 (3.42 [95% CI 3.34-3.50]) and became less than 1 from February 7 (0.76 [95% CI 0.65-0.92]). We also estimate the effects of two major government interventions on the spread of COVID-19 in Wuhan. CONCLUSIONS This case study by our proposed method affirms the believed importance and effectiveness of imposing tight non-essential travel restrictions and affirm the importance and effectiveness of government interventions (e.g., transportation suspension and large scale hospitalization) for effective mitigation of COVID-19 community spread.
Collapse
Affiliation(s)
- Hongzhe Zhang
- Institute for Financial Services Analytics, University of Delaware, Newark, DE.
| | - Xiaohang Zhao
- Institute for Financial Services Analytics, University of Delaware, Newark, DE.
| | - Kexin Yin
- Institute for Financial Services Analytics, University of Delaware, Newark, DE.
| | - Yiren Yan
- Institute for Financial Services Analytics, University of Delaware, Newark, DE.
| | - Wei Qian
- Institute for Financial Services Analytics, and Department of Applied Economics and Statistics, University of Delaware, Newark, DE.
| | - Bintong Chen
- Institute for Financial Services Analytics Lerner College of Business and Economics, University of Delaware, Newark, DE.
| | - Xiao Fang
- Institute for Financial Services Analytics Lerner College of Business and Economics, University of Delaware, Newark, DE.
| |
Collapse
|
20
|
Meng L, Zhang J, Zhang X, Feng G. Bayesian estimation of time-varying parameters in ordinary differential equation models with noisy time-varying covariates. COMMUN STAT-SIMUL C 2021. [DOI: 10.1080/03610918.2019.1565584] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Affiliation(s)
- Lixin Meng
- School of Statistics, Jilin University of Finance and Economics, Changchun, P R China
| | - Jiwei Zhang
- School of Mathematics and Statistics, Yunnan University, Kunming, P R China
| | - Xue Zhang
- China Institute of Rural Education Development, Northeast Normal University, Changchun, P R China
| | - Guozhong Feng
- School of Information Science and Technology, Northeast Normal University, Changchun, P R China
| |
Collapse
|
21
|
Yin M, Zheng X, Humphrey JD, Em Karniadakis G. Non-invasive Inference of Thrombus Material Properties with Physics-Informed Neural Networks. COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING 2021; 375:113603. [PMID: 33414569 PMCID: PMC7785048 DOI: 10.1016/j.cma.2020.113603] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/17/2023]
Abstract
We employ physics-informed neural networks (PINNs) to infer properties of biological materials using synthetic data. In particular, we successfully apply PINNs on inferring permeability and viscoelastic modulus from thrombus deformation data, which can be described by the fourth-order Cahn-Hilliard and Navier-Stokes Equations. In PINNs, the partial differential equations are encoded into a loss function, where partial derivatives can be obtained through automatic differentiation (AD). In addition to tackling the challenge of calculating the fourth-order derivative in the Cahn-Hilliard equation with AD, we introduce an auxiliary network along with the main neural network to approximate the second-derivative of the energy potential term. Our model can simultaneously predict unknown material parameters and velocity, pressure, and deformation gradient fields by merely training with partial information among all data, i.e., phase field and pressure measurements, while remaining highly flexible in sampling within the spatio-temporal domain for data acquisition. We validate our model by numerical solutions from the spectral/hp element method (SEM) and demonstrate its robustness by training it with noisy measurements. Our results show that PINNs can infer the material properties from noisy synthetic data, and thus they have great potential for inferring these properties from experimental multi-modality and multi-fidelity data.
Collapse
Affiliation(s)
- Minglang Yin
- Center for Biomedical Engineering, Brown University, Providence, RI 02912
- School of Engineering, Brown University, Providence, RI 02912
| | - Xiaoning Zheng
- Division of Applied Mathematics, Brown University, Providence, RI 02912
| | - Jay D. Humphrey
- Department of Biomedical Engineering, Yale University, New Haven, CT 06520
| | - George Em Karniadakis
- Division of Applied Mathematics, Brown University, Providence, RI 02912
- Corresponding author:
| |
Collapse
|
22
|
Niu M, Wandy J, Daly R, Rogers S, Husmeier D. R package for statistical inference in dynamical systems using kernel based gradient matching: KGode. Comput Stat 2020. [DOI: 10.1007/s00180-020-01014-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
AbstractMany processes in science and engineering can be described by dynamical systems based on nonlinear ordinary differential equations (ODEs). Often ODE parameters are unknown and not directly measurable. Since nonlinear ODEs typically have no closed form solution, standard iterative inference procedures require a computationally expensive numerical integration of the ODEs every time the parameters are adapted, which in practice restricts statistical inference to rather small systems. To overcome this computational bottleneck, approximate methods based on gradient matching have recently gained much attention. The idea is to circumvent the numerical integration step by using a surrogate cost function that quantifies the discrepancy between the derivatives obtained from a smooth interpolant to the data and the derivatives predicted by the ODEs. The present article describes the software implementation of a recent method that is based on the framework of reproducing kernel Hilbert spaces. We provide an overview of the methods available, illustrate them on a series of widely used benchmark problems, and discuss the accuracy–efficiency trade-off of various regularization methods.
Collapse
|
23
|
Hu X, Hu Y, Wu F, Leung RWT, Qin J. Integration of single-cell multi-omics for gene regulatory network inference. Comput Struct Biotechnol J 2020; 18:1925-1938. [PMID: 32774787 PMCID: PMC7385034 DOI: 10.1016/j.csbj.2020.06.033] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2020] [Revised: 06/17/2020] [Accepted: 06/20/2020] [Indexed: 12/20/2022] Open
Abstract
The advancement of single-cell sequencing technology in recent years has provided an opportunity to reconstruct gene regulatory networks (GRNs) with the data from thousands of single cells in one sample. This uncovers regulatory interactions in cells and speeds up the discoveries of regulatory mechanisms in diseases and biological processes. Therefore, more methods have been proposed to reconstruct GRNs using single-cell sequencing data. In this review, we introduce technologies for sequencing single-cell genome, transcriptome, and epigenome. At the same time, we present an overview of current GRN reconstruction strategies utilizing different single-cell sequencing data. Bioinformatics tools were grouped by their input data type and mathematical principles for reader's convenience, and the fundamental mathematics inherent in each group will be discussed. Furthermore, the adaptabilities and limitations of these different methods will also be summarized and compared, with the hope to facilitate researchers recognizing the most suitable tools for them.
Collapse
Affiliation(s)
- Xinlin Hu
- Shenzhen Key Laboratory of Advanced Machine Learning and Applications, College of Mathematics and Statistics, Shenzhen University, Shenzhen 518060, China
| | - Yaohua Hu
- Shenzhen Key Laboratory of Advanced Machine Learning and Applications, College of Mathematics and Statistics, Shenzhen University, Shenzhen 518060, China
| | - Fanjie Wu
- School of Pharmaceutical Sciences (Shenzhen), Sun Yat-sen University, Shenzhen 518107, China
| | - Ricky Wai Tak Leung
- School of Pharmaceutical Sciences (Shenzhen), Sun Yat-sen University, Shenzhen 518107, China
| | - Jing Qin
- School of Pharmaceutical Sciences (Shenzhen), Sun Yat-sen University, Shenzhen 518107, China
| |
Collapse
|
24
|
Maroufy V, Shah P, Asghari A, Deng N, Le RNU, Ramirez JC, Yaseen A, Zheng WJ, Umetani M, Wu H. Gene expression dynamic analysis reveals co-activation of Sonic Hedgehog and epidermal growth factor followed by dynamic silencing. Oncotarget 2020; 11:1358-1372. [PMID: 32341755 PMCID: PMC7170495 DOI: 10.18632/oncotarget.27547] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2019] [Accepted: 03/14/2020] [Indexed: 12/02/2022] Open
Abstract
Aberrant activation of the Sonic Hedgehog (SHH) gene is observed in various cancers. Previous studies have shown a “cross-talk” effect between the canonical Hedgehog signaling pathway and the Epidermal Growth Factor (EGF) pathway when SHH is active in the presence of EGF. However, the precise mechanism of the cross-talk effect on the entire gene population has not been investigated. Here, we re-analyzed publicly available data to study how SHH and EGF cooperate to affect the dynamic activity of the gene population. We used genome dynamic analysis to explore the expression profiles under different conditions in a human medulloblastoma cell line. Ordinary differential equations, equipped with solid statistical and computational tools, were exploited to extract the information hidden in the dynamic behavior of the gene population. Our results revealed that EGF stimulation plays a dominant role, overshadowing most of the SHH effects. We also identified cross-talk genes that exhibited expression profiles dissimilar to that seen under SHH or EGF stimulation alone. These unique cross-talk patterns were validated in a cell culture model. These cross-talk genes identified here may serve as valuable markers to study or test for EGF co-stimulatory effects in an SHH+ environment. Furthermore, these cross-talk genes may play roles in cancer progression, thus they may be further explored as cancer treatment targets.
Collapse
Affiliation(s)
- Vahed Maroufy
- Department of Biostatistics and Data Science, School of Public Heath, University of Texas Health Science Center at Houston, Houston, TX, USA.,These authors contributed equally to this work
| | - Pankil Shah
- Department of Epidemiology, Human Genetics and Environmental Sciences, School of Public Health, University of Texas Health Science Center at Houston, Houston, TX, USA.,These authors contributed equally to this work
| | - Arvand Asghari
- Center for Nuclear Receptors and Cell Signaling, Department of Biology and Biochemistry, University of Houston, Houston, TX, USA
| | - Nan Deng
- Department of Biostatistics and Data Science, School of Public Heath, University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Rosemarie N U Le
- Center for Nuclear Receptors and Cell Signaling, Department of Biology and Biochemistry, University of Houston, Houston, TX, USA
| | - Juan C Ramirez
- Facultad de Ingeniería de Sistemas, Universidad Antonio Nariño, Bogota, Colombia
| | - Ashraf Yaseen
- Department of Biostatistics and Data Science, School of Public Heath, University of Texas Health Science Center at Houston, Houston, TX, USA
| | - W Jim Zheng
- School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Michihisa Umetani
- Center for Nuclear Receptors and Cell Signaling, Department of Biology and Biochemistry, University of Houston, Houston, TX, USA.,HEALTH Research Institute, University of Houston, Houston, TX, USA
| | - Hulin Wu
- Department of Biostatistics and Data Science, School of Public Heath, University of Texas Health Science Center at Houston, Houston, TX, USA
| |
Collapse
|
25
|
Soltanalizadeh B, Gonzalez Rodriguez E, Maroufy V, Zheng WJ, Wu H. Modelling of hypoxia gene expression for three different cancer cell lines. ACTA ACUST UNITED AC 2020; 13:124-143. [PMID: 32153660 DOI: 10.1504/ijcbdd.2020.10026794] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
Abstract
Gene dynamic analysis is essential in identifying target genes involved pathogenesis of various diseases, including cancer. Cancer prognosis is often influenced by hypoxia. We apply a multi-step pipeline to study dynamic gene expressions in response to hypoxia in three cancer cell lines: prostate (DU145), colon (HT29), and breast (MCF7) cancers. We identified 26 distinct temporal expression patterns for prostate cell line, and 29 patterns for colon and breast cell lines. The module-based dynamic networks have been developed for all three cell lines. Our analyses improve the existing results in multiple ways. It exploits the time-dependence nature of gene expression values in identifying the dynamically significant genes; hence, more key significant genes and transcription factors have been identified. Our gene network returns significant information regarding biologically important modules of genes. Furthermore, the network has potential in learning the regulatory path between transcription factors and the downstream genes. In addition, our findings suggest that changes in genes BMP6 and ARSJ expression might have a key role in the time-dependent response to hypoxia in breast cancer.
Collapse
Affiliation(s)
- Babak Soltanalizadeh
- Department of Biostatistics & Data Science, University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Erika Gonzalez Rodriguez
- Center for translational Injury Research, Department of Surgery, McGovern Medical School, UT Houston, Houston, TX, USA
| | - Vahed Maroufy
- Department of Biostatistics & Data Science, University of Texas Health Science Center at Houston, Houston, TX, USA
| | - W Jim Zheng
- School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Hulin Wu
- Department of Biostatistics & Data Science, University of Texas Health Science Center at Houston, Houston, TX, USA
| |
Collapse
|
26
|
Huang H, Handel A, Song X. A Bayesian approach to estimate parameters of ordinary differential equation. Comput Stat 2020. [DOI: 10.1007/s00180-020-00962-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
27
|
Carey M, Ramírez JC, Wu S, Wu H. A big data pipeline: Identifying dynamic gene regulatory networks from time-course Gene Expression Omnibus data with applications to influenza infection. Stat Methods Med Res 2019; 27:1930-1955. [PMID: 29846143 DOI: 10.1177/0962280217746719] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
A biological host response to an external stimulus or intervention such as a disease or infection is a dynamic process, which is regulated by an intricate network of many genes and their products. Understanding the dynamics of this gene regulatory network allows us to infer the mechanisms involved in a host response to an external stimulus, and hence aids the discovery of biomarkers of phenotype and biological function. In this article, we propose a modeling/analysis pipeline for dynamic gene expression data, called Pipeline4DGEData, which consists of a series of statistical modeling techniques to construct dynamic gene regulatory networks from the large volumes of high-dimensional time-course gene expression data that are freely available in the Gene Expression Omnibus repository. This pipeline has a consistent and scalable structure that allows it to simultaneously analyze a large number of time-course gene expression data sets, and then integrate the results across different studies. We apply the proposed pipeline to influenza infection data from nine studies and demonstrate that interesting biological findings can be discovered with its implementation.
Collapse
Affiliation(s)
- Michelle Carey
- 1 School of Mathematics and Statistics, University College Dublin, Dublin, Ireland
| | - Juan Camilo Ramírez
- 2 Department of Biostatistics, School of Public Health, University of Texas Health Science Center at Houston, Houston, TX, USA
| | | | - Hulin Wu
- 2 Department of Biostatistics, School of Public Health, University of Texas Health Science Center at Houston, Houston, TX, USA
| |
Collapse
|
28
|
Liu B, Wang L, Nie Y, Cao J. Bayesian inference of mixed-effects ordinary differential equations models using heavy-tailed distributions. Comput Stat Data Anal 2019. [DOI: 10.1016/j.csda.2019.03.001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
29
|
Nonparametric inference of interaction laws in systems of agents from trajectory data. Proc Natl Acad Sci U S A 2019; 116:14424-14433. [PMID: 31253708 DOI: 10.1073/pnas.1822012116] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
Inferring the laws of interaction in agent-based systems from observational data is a fundamental challenge in a wide variety of disciplines. We propose a nonparametric statistical learning approach for distance-based interactions, with no reference or assumption on their analytical form, given data consisting of sampled trajectories of interacting agents. We demonstrate the effectiveness of our estimators both by providing theoretical guarantees that avoid the curse of dimensionality and by testing them on a variety of prototypical systems used in various disciplines. These systems include homogeneous and heterogeneous agent systems, ranging from particle systems in fundamental physics to agent-based systems that model opinion dynamics under the social influence, prey-predator dynamics, flocking and swarming, and phototaxis in cell dynamics.
Collapse
|
30
|
Jonoska Stojkova B, Campbell DA. Incremental Mixture Importance Sampling With Shotgun Optimization. J Comput Graph Stat 2019. [DOI: 10.1080/10618600.2019.1592756] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Affiliation(s)
| | - David A. Campbell
- Department of Statistics and Actuarial Science, Simon Fraser University, Burnaby, Canada
| |
Collapse
|
31
|
Dony L, He F, Stumpf MPH. Parametric and non-parametric gradient matching for network inference: a comparison. BMC Bioinformatics 2019; 20:52. [PMID: 30683048 PMCID: PMC6346534 DOI: 10.1186/s12859-018-2590-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2018] [Accepted: 12/21/2018] [Indexed: 11/24/2022] Open
Abstract
Background Reverse engineering of gene regulatory networks from time series gene-expression data is a challenging problem, not only because of the vast sets of candidate interactions but also due to the stochastic nature of gene expression. We limit our analysis to nonlinear differential equation based inference methods. In order to avoid the computational cost of large-scale simulations, a two-step Gaussian process interpolation based gradient matching approach has been proposed to solve differential equations approximately. Results We apply a gradient matching inference approach to a large number of candidate models, including parametric differential equations or their corresponding non-parametric representations, we evaluate the network inference performance under various settings for different inference objectives. We use model averaging, based on the Bayesian Information Criterion (BIC), to combine the different inferences. The performance of different inference approaches is evaluated using area under the precision-recall curves. Conclusions We found that parametric methods can provide comparable, and often improved inference compared to non-parametric methods; the latter, however, require no kinetic information and are computationally more efficient. Electronic supplementary material The online version of this article (10.1186/s12859-018-2590-7) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Leander Dony
- Centre for Integrative Systems Biology and Bioinformatics, Department of Life Sciences, Imperial College London, London, SW7 2AZ, UK.,Institute of Computational Biology, Helmholtz Center Munich, German Research Center for Environmental Health, Neuherberg, 85764, Germany.,Max Planck Institute of Psychiatry, Kraepelinstr. 2-10, Munich, 80804, Germany
| | - Fei He
- Centre for Integrative Systems Biology and Bioinformatics, Department of Life Sciences, Imperial College London, London, SW7 2AZ, UK.,School of Computing, Electronics, and Mathematics, Coventry University, Coventry, CV1 2JH, UK
| | - Michael P H Stumpf
- Centre for Integrative Systems Biology and Bioinformatics, Department of Life Sciences, Imperial College London, London, SW7 2AZ, UK. .,Melbourne Integrative Genomics, School of BioScience & School of Mathematics and Statistics, University of Melbourne, Parkville Melbourne, 3010, Australia.
| |
Collapse
|
32
|
Wu L, Qiu X, Yuan YX, Wu H. Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach. J Am Stat Assoc 2019; 114:657-667. [PMID: 34385718 DOI: 10.1080/01621459.2017.1423074] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
Ordinary differential equations (ODEs) are widely used to model the dynamic behavior of a complex system. Parameter estimation and variable selection for a "Big System" with linear ODEs are very challenging due to the need of nonlinear optimization in an ultra-high dimensional parameter space. In this article, we develop a parameter estimation and variable selection method based on the ideas of similarity transformation and separable least squares (SLS). Simulation studies demonstrate that the proposed matrix-based SLS method could be used to estimate the coefficient matrix more accurately and perform variable selection for a linear ODE system with thousands of dimensions and millions of parameters much better than the direct least squares (LS) method and the vector-based two-stage method that are currently available. We applied this new method to two real data sets: a yeast cell cycle gene expression data set with 30 dimensions and 930 unknown parameters and the Standard & Poor 1500 index stock price data with 1250 dimensions and 1,563,750 unknown parameters, to illustrate the utility and numerical performance of the proposed parameter estimation and variable selection method for big systems in practice.
Collapse
Affiliation(s)
- Leqin Wu
- Department of Mathematics, Jinan University, Guangzhou, China
| | - Xing Qiu
- Department of Biostatistics and Computational Biology University of Rochester, Rochester, New York, U.S.A
| | - Ya-Xiang Yuan
- Academy of Mathematics and System Sciences Chinese Academy of Sciences, Beijing, China
| | - Hulin Wu
- Department of Biostatistics, University of Texas Health Science Center at Houston, Houston, TX, U.S.A
| |
Collapse
|
33
|
Xue H, Kumar A, Wu H. Parameter Estimation for Semiparametric Ordinary Differential Equation Models. COMMUN STAT-THEOR M 2019; 48:5985-6004. [PMID: 32952273 DOI: 10.1080/03610926.2018.1523433] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
We propose a new class of two-stage parameter estimation methods for semiparametric ordinary differential equation (ODE) models. In the first stage, state variables are estimated using a penalized spline approach; In the second stage, form of numerical discretization algorithms for an ODE solver is used to formulate estimating equations. Estimated state variables from the first stage are used to obtain more data points for the second stage. Asymptotic properties for the proposed estimators are established. Simulation studies show that the method performs well, especially for small sample. Real life use of the method is illustrated using Influenza specific cell-trafficking study.
Collapse
Affiliation(s)
- Hongqi Xue
- iCardiac Technologies, 150 Allens Creek Rd, Rochester, NY 14618
| | - Arun Kumar
- Livanova, 100 Cyberonics Blvd, Houston, TX 77058
| | - Hulin Wu
- The university of texas health science center of Houston, 7000 Fannin St 1200, Houston, TX 77030
| |
Collapse
|
34
|
Xue H, Wu S, Wu Y, Idarraga JCR, Wu H. Independence screening for high dimensional nonlinear additive ODE models with applications to dynamic gene regulatory networks. Stat Med 2018; 37:2630-2644. [PMID: 29722041 PMCID: PMC6940146 DOI: 10.1002/sim.7669] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2016] [Revised: 01/18/2018] [Accepted: 03/08/2018] [Indexed: 11/12/2022]
Abstract
Mechanism-driven low-dimensional ordinary differential equation (ODE) models are often used to model viral dynamics at cellular levels and epidemics of infectious diseases. However, low-dimensional mechanism-based ODE models are limited for modeling infectious diseases at molecular levels such as transcriptomic or proteomic levels, which is critical to understand pathogenesis of diseases. Although linear ODE models have been proposed for gene regulatory networks (GRNs), nonlinear regulations are common in GRNs. The reconstruction of large-scale nonlinear networks from time-course gene expression data remains an unresolved issue. Here, we use high-dimensional nonlinear additive ODEs to model GRNs and propose a 4-step procedure to efficiently perform variable selection for nonlinear ODEs. To tackle the challenge of high dimensionality, we couple the 2-stage smoothing-based estimation method for ODEs and a nonlinear independence screening method to perform variable selection for the nonlinear ODE models. We have shown that our method possesses the sure screening property and it can handle problems with non-polynomial dimensionality. Numerical performance of the proposed method is illustrated with simulated data and a real data example for identifying the dynamic GRN of Saccharomyces cerevisiae.
Collapse
Affiliation(s)
- Hongqi Xue
- iCardiac Technologies, 150 Allens Creek Road, Rochester, NY 14618, USA
| | - Shuang Wu
- Biogen, 300 Binney Street, Cambridge, MA 02142, USA
| | - Yichao Wu
- Department of Mathematics, Statistics and Computer Science, University of Illinois at Chicago, Chicago, IL 60607-7045, USA
| | | | - Hulin Wu
- Department of Biostatistics and Data Science, School of Public Health, University of Texas Health Science Center at Houston, 1200 Pressler Street, RAS E833, Houston, TX 77030, USA
| |
Collapse
|
35
|
Kim Y, Kim J. Estimation of Dynamic Systems for Gene Regulatory Networks from Dependent Time-Course Data. J Comput Biol 2018; 25:987-996. [PMID: 29905491 DOI: 10.1089/cmb.2018.0062] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/01/2023] Open
Abstract
Dynamic system consisting of ordinary differential equations (ODEs) is a well-known tool for describing dynamic nature of gene regulatory networks (GRNs), and the dynamic features of GRNs are usually captured through time-course gene expression data. Owing to high-throughput technologies, time-course gene expression data have complex structures such as heteroscedasticity, correlations between genes, and time dependence. Since gene experiments typically yield highly noisy data with small sample size, for a more accurate prediction of the dynamics, the complex structures should be taken into account in ODE models. Hence, this study proposes an ODE model considering such data structures and a fast and stable estimation method for the ODE parameters based on the generalized profiling approach with data smoothing techniques. The proposed method also provides statistical inference for the ODE estimator and it is applied to a zebrafish retina cell network.
Collapse
Affiliation(s)
- Yoonji Kim
- Department of Statistics, Sungkyunkwan University , Seoul, Korea
| | - Jaejik Kim
- Department of Statistics, Sungkyunkwan University , Seoul, Korea
| |
Collapse
|
36
|
Zhang Q, Yu Y, Zhang J, Liang H. Using single-index ODEs to study dynamic gene regulatory network. PLoS One 2018; 13:e0192833. [PMID: 29474376 PMCID: PMC5825071 DOI: 10.1371/journal.pone.0192833] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2017] [Accepted: 01/31/2018] [Indexed: 12/18/2022] Open
Abstract
With the development of biotechnology, high-throughput studies on protein-protein, protein-gene, and gene-gene interactions become possible and attract remarkable attention. To explore the interactions in dynamic gene regulatory networks, we propose a single-index ordinary differential equation (ODE) model and develop a variable selection procedure. We employ the smoothly clipped absolute deviation penalty (SCAD) penalized function for variable selection. We analyze a yeast cell cycle gene expression data set to illustrate the usefulness of the single-index ODE model. In real data analysis, we group genes into functional modules using the smoothing spline clustering approach. We estimate state functions and their first derivatives for functional modules using penalized spline-based nonparametric mixed-effects models and the spline method. We substitute the estimates into the single-index ODE models, and then use the penalized profile least-squares procedure to identify network structures among the models. The results indicate that our model fits the data better than linear ODE models and our variable selection procedure identifies the interactions that may be missed by linear ODE models but confirmed in biological studies. In addition, Monte Carlo simulation studies are used to evaluate and compare the methods.
Collapse
Affiliation(s)
- Qi Zhang
- Department of Statistics, Qingdao University, Qingdao, China
| | - Yao Yu
- Department of Biostatistics and Computational Biology, University of Rochester School of Medicine and Dentistry, Rochester, New York, United States of America
| | - Jun Zhang
- Institute of Statistical Sciences at Shenzhen University, Shenzhen University, Shenzhen, China
| | - Hua Liang
- Department of Statistics, George Washington University, Washington, D.C., United States of America
- * E-mail:
| |
Collapse
|
37
|
Niu M, Macdonald B, Rogers S, Filippone M, Husmeier D. Statistical inference in mechanistic models: time warping for improved gradient matching. Comput Stat 2018; 33:1091-1123. [PMID: 31258254 PMCID: PMC6560940 DOI: 10.1007/s00180-017-0753-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2016] [Accepted: 07/19/2017] [Indexed: 10/31/2022]
Abstract
Inference in mechanistic models of non-linear differential equations is a challenging problem in current computational statistics. Due to the high computational costs of numerically solving the differential equations in every step of an iterative parameter adaptation scheme, approximate methods based on gradient matching have become popular. However, these methods critically depend on the smoothing scheme for function interpolation. The present article adapts an idea from manifold learning and demonstrates that a time warping approach aiming to homogenize intrinsic length scales can lead to a significant improvement in parameter estimation accuracy. We demonstrate the effectiveness of this scheme on noisy data from two dynamical systems with periodic limit cycle, a biopathway, and an application from soft-tissue mechanics. Our study also provides a comparative evaluation on a wide range of signal-to-noise ratios.
Collapse
Affiliation(s)
- Mu Niu
- School of Mathematics and Statistics, University of Glasgow, Glasgow, UK
| | - Benn Macdonald
- School of Mathematics and Statistics, University of Glasgow, Glasgow, UK
| | - Simon Rogers
- Department of Computer Science, University of Glasgow, Glasgow, UK
| | | | - Dirk Husmeier
- School of Mathematics and Statistics, University of Glasgow, Glasgow, UK
| |
Collapse
|
38
|
Song J, Carey M, Zhu H, Miao H, Ramírez JC, Wu H. Identifying the dynamic gene regulatory network during latent HIV-1 reactivation using high-dimensional ordinary differential equations. ACTA ACUST UNITED AC 2018; 11:135-153. [PMID: 34531927 DOI: 10.1504/ijcbdd.2018.10011910] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
Abstract
Reactivation of latently infected cells has emerged as an important strategy for eradication of HIV. However, genetic mechanisms of regulation after reactivation remain unclear. We describe a five-step pipeline to study the dynamics of the gene regulatory network following a viral reactivation using high-dimensional ordinary differential equations. Our pipeline implements a combination of five different methods, by detecting temporally differentially expressed genes (step 1), clustering genes with similar temporal expression patterns into a small number of response modules (step2), performing a functional enrichment analysis within each gene response module (step 3), identifying a network structure based on the gene response modules using ordinary differential equations (ODE) and a high-dimensional variable selection technique (step 4), and obtaining a gene regulatory model based on refined parameter estimates using nonlinear least squares (step 5). We applied our pipeline to a time course gene expression data of latently infected T-cells following a latency-reversion.
Collapse
Affiliation(s)
- Jaejoon Song
- Department of Biostatistics, The University of Texas MD, Anderson Cancer Center, 1400 Pressler Street, Houston, TX, 77030, USA
| | - Michelle Carey
- Department of Mathematics and Statistics, McGill University, 805 Sherbrooke Street West, Montreal, Canada, H3A 0B9
| | - Hongjian Zhu
- Department of Biostatistics, The University of Texas School of Public Health, 1200 Pressler Street, Houston, TX, 77030, USA
| | - Hongyu Miao
- Department of Biostatistics, The University of Texas School of Public Health, 1200 Pressler Street, Houston, TX, 77030, USA
| | - Juan Camilo Ramírez
- Faculty of Computer Engineering, Universidad Antonio Nariño, Cl. 58a Bis 3794, Bogotá, Cundinamarca, Colombia
| | - Hulin Wu
- Department of Biostatistics, The University of Texas School of Public Health, 1200 Pressler Street, Houston, TX, USA, 77030, USA
| |
Collapse
|
39
|
Dattner I, Miller E, Petrenko M, Kadouri DE, Jurkevitch E, Huppert A. Modelling and parameter inference of predator-prey dynamics in heterogeneous environments using the direct integral approach. J R Soc Interface 2017; 14:rsif.2016.0525. [PMID: 28053112 DOI: 10.1098/rsif.2016.0525] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2016] [Accepted: 11/28/2016] [Indexed: 11/12/2022] Open
Abstract
Most bacterial habitats are topographically complex in the micro scale. Important examples include the gastrointestinal and tracheal tracts, and the soil. Although there are myriad theoretical studies that explore the role of spatial structures on antagonistic interactions (predation, competition) among animals, there are many fewer experimental studies that have explored, validated and quantified their predictions. In this study, we experimentally monitored the temporal dynamic of the predatory bacterium Bdellovibrio bacteriovorus, and its prey, the bacterium Burkholderia stabilis in a structured habitat consisting of sand under various regimes of wetness. We constructed a dynamic model, and estimated its parameters by further developing the direct integral method, a novel estimation procedure that exploits the separability of the states and parameters in the model. We also verified that one of our parameter estimates was consistent with its known, directly measured value from the literature. The ability of the model to fit the data combined with realistic parameter estimates indicate that bacterial predation in the sand can be described by a relatively simple model, and stress the importance of prey refuge on predation dynamics in heterogeneous environments.
Collapse
Affiliation(s)
- Itai Dattner
- Department of Statistics, University of Haifa, 199 Abba Khoushy Avenue, Mount Carmel, Haifa 3498838, Israel
| | - Ezer Miller
- Bio-statistical Unit, The Gertner Institute for Epidemiology and Health Policy Research, Chaim Sheba Medical Center, Tel Hashomer 52621, Israel
| | - Margarita Petrenko
- Department of Agroecology and Plant Health, The Robert H. Smith Faculty of Agriculture, Food and Environment, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Daniel E Kadouri
- Department of Oral Biology, Rutgers School of Dental Medicine, Newark, NJ, USA
| | - Edouard Jurkevitch
- Department of Agroecology and Plant Health, The Robert H. Smith Faculty of Agriculture, Food and Environment, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Amit Huppert
- Bio-statistical Unit, The Gertner Institute for Epidemiology and Health Policy Research, Chaim Sheba Medical Center, Tel Hashomer 52621, Israel.,Department of Epidemiology and Preventive Medicine at the School of Public Health, the Sackler Faculty of Medicine, Tel-Aviv University, Israel
| |
Collapse
|
40
|
Anelone AJN, Spurgeon SK. Prediction of the containment of HIV infection by antiretroviral therapy - a variable structure control approach. IET Syst Biol 2017; 11:44-53. [PMID: 28303793 DOI: 10.1049/iet-syb.2016.0028] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/24/2023] Open
Abstract
It is demonstrated that the reachability paradigm from variable structure control theory is a suitable framework to monitor and predict the progression of the human immunodeficiency virus (HIV) infection following initiation of antiretroviral therapy (ART). A manifold is selected which characterises the infection-free steady-state. A model of HIV infection together with an associated reachability analysis is used to formulate a dynamical condition for the containment of HIV infection on the manifold. This condition is tested using data from two different HIV clinical trials which contain measurements of the CD4+ T cell count and HIV load in the peripheral blood collected from HIV infected individuals for the six month period following initiation of ART. The biological rates of the model are estimated using the multi-point identification method and data points collected in the initial period of the trial. Using the parameter estimates and the numerical solutions of the model, the predictions of the reachability analysis are shown to be consistent with the clinical diagnosis at the conclusion of the trial. The methodology captures the dynamical characteristics of eventual successful, failed and marginal outcomes. The findings evidence that the reachability analysis is an appropriate tool to monitor and develop personalised antiretroviral treatment.
Collapse
Affiliation(s)
- Anet J N Anelone
- School of Engineering and Digital Arts, The University of Kent, U.K
| | - Sarah K Spurgeon
- School of Engineering and Digital Arts, The University of Kent, U.K.
| |
Collapse
|
41
|
Carey M, Gath EG, Hayes K. A Generalized Smoother for Linear Ordinary Differential Equations. J Comput Graph Stat 2017. [DOI: 10.1080/10618600.2016.1265526] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Michelle Carey
- Department of Mathematics and Statistics, College of Science and Engineering, University of Limerick, Limerick, Ireland
| | - Eugene G. Gath
- Department of Mathematics and Statistics, College of Science and Engineering, University of Limerick, Limerick, Ireland
| | - Kevin Hayes
- Department of Mathematics and Statistics, College of Science and Engineering, University of Limerick, Limerick, Ireland
| |
Collapse
|
42
|
Abstract
Self-regulation can be conceptualized in terms of dynamic tension between highly probable reactions (prepotent responses) and use of strategies that can modulate those reactions (executive processes). This study investigated the value of a dynamical systems approach to the study of early childhood self-regulation. Specifically, ordinary differential equations (ODEs) were used to model the interactive influences of 115 36-month-olds' executive processes (strategy use) and prepotent responses to waiting to open a gift (desire for the gift and frustration about waiting to open it). Using a pair of coupled second-order ODEs in a nonlinear mixed effects framework, the study tested predictions for specific within- and between-child patterns of prepotent response-executive process coupling. Dynamic modeling results articulated the limits of 36-month olds' strategic efforts. They engaged executive processes when their prepotent responding levels were high, which delayed the resurgence of prepotent responses, but ultimately did not damp prepotent responding over the course of the wait. There was, however, preliminary evidence that the effectiveness of 36-month-olds' self-regulation depended upon child characteristics. Externalizing behavior problems were associated with more regulatory interference. Temperamental negative affectivity was marginally associated with more regulatory inefficiency. Compared with conventional methods of studying self-regulation, dynamic modeling yielded complementary and unique findings, suggesting its potential. (PsycINFO Database Record
Collapse
Affiliation(s)
| | | | - Nilam Ram
- The Pennsylvania State University
- German Institute for Economic Research (DIW), Berlin
| | | |
Collapse
|
43
|
Chen S, Shojaie A, Witten DM. Network Reconstruction From High-Dimensional Ordinary Differential Equations. J Am Stat Assoc 2017; 112:1697-1707. [PMID: 29618851 DOI: 10.1080/01621459.2016.1229197] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
We consider the task of learning a dynamical system from high-dimensional time-course data. For instance, we might wish to estimate a gene regulatory network from gene expression data measured at discrete time points. We model the dynamical system nonparametrically as a system of additive ordinary differential equations. Most existing methods for parameter estimation in ordinary differential equations estimate the derivatives from noisy observations. This is known to be challenging and inefficient. We propose a novel approach that does not involve derivative estimation. We show that the proposed method can consistently recover the true network structure even in high dimensions, and we demonstrate empirical improvement over competing approaches. Supplementary materials for this article are available online.
Collapse
Affiliation(s)
- Shizhe Chen
- Department of Biostatistics, University of Washington, WA
| | - Ali Shojaie
- Departments of Biostatistics and Statistics, University of Washington, WA
| | - Daniela M Witten
- Departments of Biostatistics and Statistics, University of Washington, WA
| |
Collapse
|
44
|
Nguyen VK, Klawonn F, Mikolajczyk R, Hernandez-Vargas EA. Analysis of Practical Identifiability of a Viral Infection Model. PLoS One 2016; 11:e0167568. [PMID: 28036339 PMCID: PMC5201286 DOI: 10.1371/journal.pone.0167568] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2016] [Accepted: 11/16/2016] [Indexed: 11/27/2022] Open
Abstract
Mathematical modelling approaches have granted a significant contribution to life sciences and beyond to understand experimental results. However, incomplete and inadequate assessments in parameter estimation practices hamper the parameter reliability, and consequently the insights that ultimately could arise from a mathematical model. To keep the diligent works in modelling biological systems from being mistrusted, potential sources of error must be acknowledged. Employing a popular mathematical model in viral infection research, existing means and practices in parameter estimation are exemplified. Numerical results show that poor experimental data is a main source that can lead to erroneous parameter estimates despite the use of innovative parameter estimation algorithms. Arbitrary choices of initial conditions as well as data asynchrony distort the parameter estimates but are often overlooked in modelling studies. This work stresses the existence of several sources of error buried in reports of modelling biological systems, voicing the need for assessing the sources of error, consolidating efforts in solving the immediate difficulties, and possibly reconsidering the use of mathematical modelling to quantify experimental data.
Collapse
Affiliation(s)
- Van Kinh Nguyen
- Systems Medicine of Infectious Diseases, Department of Systems Immunology and Braunschweig Integrated Centre of Systems Biology, Helmholtz Centre for Infection Research, Braunschweig, Germany
- Epidemiology Department, Ho Chi Minh University of Medicine and Pharmacy, Ho Chi Minh, Vietnam
- PhD Programme “Epidemiology”, Braunschweig-Hannover, Germany
| | - Frank Klawonn
- Biostatistics, Helmholtz Centre for Infection Research, Braunschweig, Germany
- Department of Computer Science, Ostfalia University, Wolfenbüttel, Germany
| | - Rafael Mikolajczyk
- Epidemiological and Statistical Methods, Helmholtz Centre for Infection Research, Braunschweig, Germany
- German Centre for Infection Research, site Hannover-Braunschweig, Germany
- Hannover Medical School, Hannover, Germany
- [Institute of] Medical Epidemiology, Biometry and Informatics, Martin-Luther University Halle-Wittenberg, Germany
| | - Esteban A. Hernandez-Vargas
- Systems Medicine of Infectious Diseases, Department of Systems Immunology and Braunschweig Integrated Centre of Systems Biology, Helmholtz Centre for Infection Research, Braunschweig, Germany
- * E-mail:
| |
Collapse
|
45
|
Zhou J, Feng HL. Consistent estimation of ordinary differential equation when the transformation parameter is unknown. Stat Probab Lett 2016. [DOI: 10.1016/j.spl.2016.02.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
46
|
Macdonald B, Niu M, Rogers S, Filippone M, Husmeier D. Approximate parameter inference in systems biology using gradient matching: a comparative evaluation. Biomed Eng Online 2016; 15 Suppl 1:80. [PMID: 27454253 PMCID: PMC4959362 DOI: 10.1186/s12938-016-0186-x] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/03/2022] Open
Abstract
BACKGROUND A challenging problem in current systems biology is that of parameter inference in biological pathways expressed as coupled ordinary differential equations (ODEs). Conventional methods that repeatedly numerically solve the ODEs have large associated computational costs. Aimed at reducing this cost, new concepts using gradient matching have been proposed, which bypass the need for numerical integration. This paper presents a recently established adaptive gradient matching approach, using Gaussian processes (GPs), combined with a parallel tempering scheme, and conducts a comparative evaluation with current state-of-the-art methods used for parameter inference in ODEs. Among these contemporary methods is a technique based on reproducing kernel Hilbert spaces (RKHS). This has previously shown promising results for parameter estimation, but under lax experimental settings. We look at a range of scenarios to test the robustness of this method. We also change the approach of inferring the penalty parameter from AIC to cross validation to improve the stability of the method. METHODS Methodology for the recently proposed adaptive gradient matching method using GPs, upon which we build our new method, is provided. Details of a competing method using RKHS are also described here. RESULTS We conduct a comparative analysis for the methods described in this paper, using two benchmark ODE systems. The analyses are repeated under different experimental settings, to observe the sensitivity of the techniques. CONCLUSIONS Our study reveals that for known noise variance, our proposed method based on GPs and parallel tempering achieves overall the best performance. When the noise variance is unknown, the RKHS method proves to be more robust.
Collapse
Affiliation(s)
- Benn Macdonald
- School of Mathematics and Statistics, University of Glasgow, Glasgow, G12 8QW Scotland
| | - Mu Niu
- School of Mathematics and Statistics, University of Glasgow, Glasgow, G12 8QW Scotland
| | - Simon Rogers
- School of Computing Science, University of Glasgow, Glasgow, G12 8RZ Scotland
| | | | - Dirk Husmeier
- School of Mathematics and Statistics, University of Glasgow, Glasgow, G12 8QW Scotland
| |
Collapse
|
47
|
Conrad PR, Girolami M, Särkkä S, Stuart A, Zygalakis K. Statistical analysis of differential equations: introducing probability measures on numerical solutions. STATISTICS AND COMPUTING 2016; 27:1065-1082. [PMID: 32226237 PMCID: PMC7089645 DOI: 10.1007/s11222-016-9671-0] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/05/2015] [Accepted: 05/05/2016] [Indexed: 06/10/2023]
Abstract
In this paper, we present a formal quantification of uncertainty induced by numerical solutions of ordinary and partial differential equation models. Numerical solutions of differential equations contain inherent uncertainties due to the finite-dimensional approximation of an unknown and implicitly defined function. When statistically analysing models based on differential equations describing physical, or other naturally occurring, phenomena, it can be important to explicitly account for the uncertainty introduced by the numerical method. Doing so enables objective determination of this source of uncertainty, relative to other uncertainties, such as those caused by data contaminated with noise or model error induced by missing physical or inadequate descriptors. As ever larger scale mathematical models are being used in the sciences, often sacrificing complete resolution of the differential equation on the grids used, formally accounting for the uncertainty in the numerical method is becoming increasingly more important. This paper provides the formal means to incorporate this uncertainty in a statistical model and its subsequent analysis. We show that a wide variety of existing solvers can be randomised, inducing a probability measure over the solutions of such differential equations. These measures exhibit contraction to a Dirac measure around the true unknown solution, where the rates of convergence are consistent with the underlying deterministic numerical method. Furthermore, we employ the method of modified equations to demonstrate enhanced rates of convergence to stochastic perturbations of the original deterministic problem. Ordinary differential equations and elliptic partial differential equations are used to illustrate the approach to quantify uncertainty in both the statistical analysis of the forward and inverse problems.
Collapse
Affiliation(s)
| | - Mark Girolami
- Department of Statistics, University of Warwick, Coventry, UK
- Present Address: Alan Turing Institute, London, UK
| | - Simo Särkkä
- Department of Electrical Engineering and Automation, Aalto University, Espoo, Finland
| | - Andrew Stuart
- Department of Mathematics, University of Warwick, Coventry, UK
| | | |
Collapse
|
48
|
|
49
|
Chow SM, Bendezú JJ, Cole PM, Ram N. A Comparison of Two-Stage Approaches for Fitting Nonlinear Ordinary Differential Equation Models with Mixed Effects. MULTIVARIATE BEHAVIORAL RESEARCH 2016; 51:154-84. [PMID: 27391255 PMCID: PMC4940142 DOI: 10.1080/00273171.2015.1123138] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/25/2023]
Abstract
Several approaches exist for estimating the derivatives of observed data for model exploration purposes, including functional data analysis (FDA; Ramsay & Silverman, 2005 ), generalized local linear approximation (GLLA; Boker, Deboeck, Edler, & Peel, 2010 ), and generalized orthogonal local derivative approximation (GOLD; Deboeck, 2010 ). These derivative estimation procedures can be used in a two-stage process to fit mixed effects ordinary differential equation (ODE) models. While the performance and utility of these routines for estimating linear ODEs have been established, they have not yet been evaluated in the context of nonlinear ODEs with mixed effects. We compared properties of the GLLA and GOLD to an FDA-based two-stage approach denoted herein as functional ordinary differential equation with mixed effects (FODEmixed) in a Monte Carlo (MC) study using a nonlinear coupled oscillators model with mixed effects. Simulation results showed that overall, the FODEmixed outperformed both the GLLA and GOLD across all the embedding dimensions considered, but a novel use of a fourth-order GLLA approach combined with very high embedding dimensions yielded estimation results that almost paralleled those from the FODEmixed. We discuss the strengths and limitations of each approach and demonstrate how output from each stage of FODEmixed may be used to inform empirical modeling of young children's self-regulation.
Collapse
|
50
|
Zhou J. Detection of influential measurement for ordinary differential equation with application to HIV dynamics. Stat Probab Lett 2015. [DOI: 10.1016/j.spl.2015.09.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|