1
|
Fronk C, Yun J, Singh P, Petzold L. Bayesian polynomial neural networks and polynomial neural ordinary differential equations. PLoS Comput Biol 2024; 20:e1012414. [PMID: 39388392 DOI: 10.1371/journal.pcbi.1012414] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Accepted: 08/14/2024] [Indexed: 10/12/2024] Open
Abstract
Symbolic regression with polynomial neural networks and polynomial neural ordinary differential equations (ODEs) are two recent and powerful approaches for equation recovery of many science and engineering problems. However, these methods provide point estimates for the model parameters and are currently unable to accommodate noisy data. We address this challenge by developing and validating the following Bayesian inference methods: the Laplace approximation, Markov Chain Monte Carlo (MCMC) sampling methods, and variational inference. We have found the Laplace approximation to be the best method for this class of problems. Our work can be easily extended to the broader class of symbolic neural networks to which the polynomial neural network belongs.
Collapse
Affiliation(s)
- Colby Fronk
- Department of Chemical Engineering, University of California, Santa Barbara, California; United States of America
| | - Jaewoong Yun
- Department of Statistics and Applied Probability, University of California, Santa Barbara, California; United States of America
- Department of Geography, University of California, Santa Barbara, California; United States of America
| | - Prashant Singh
- Science for Life Laboratory, Department of Information Technology, Uppsala University, Uppsala, Sweden
| | - Linda Petzold
- Department of Mechanical Engineering, University of California, Santa Barbara, California; United States of America
- Department of Computer Science, University of California, Santa Barbara, California; United States of America
| |
Collapse
|
2
|
Kuwahara B, Bauch CT. Predicting Covid-19 pandemic waves with biologically and behaviorally informed universal differential equations. Heliyon 2024; 10:e25363. [PMID: 38370214 PMCID: PMC10869765 DOI: 10.1016/j.heliyon.2024.e25363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Revised: 12/29/2023] [Accepted: 01/25/2024] [Indexed: 02/20/2024] Open
Abstract
During the COVID-19 pandemic, it became clear that pandemic waves and population responses were locked in a mutual feedback loop in a classic example of a coupled behavior-disease system. We demonstrate for the first time that universal differential equation (UDE) models are able to extract this interplay from data. We develop a UDE model for COVID-19 and test its ability to make predictions of second pandemic waves. We find that UDEs are capable of learning coupled behavior-disease dynamics and predicting second waves in a variety of populations, provided they are supplied with learning biases describing simple assumptions about disease transmission and population response. Though not yet suitable for deployment as a policy-guiding tool, our results demonstrate potential benefits, drawbacks, and useful techniques when applying universal differential equations to coupled systems.
Collapse
Affiliation(s)
- Bruce Kuwahara
- Department of Applied Mathematics, University of Waterloo, 200 University Ave West, Waterloo, Ontario, Canada
| | - Chris T. Bauch
- Department of Applied Mathematics, University of Waterloo, 200 University Ave West, Waterloo, Ontario, Canada
| |
Collapse
|
3
|
Singh FA, Afzal N, Smithline SJ, Thalhauser CJ. Assessing the performance of QSP models: biology as the driver for validation. J Pharmacokinet Pharmacodyn 2023:10.1007/s10928-023-09871-x. [PMID: 37386340 DOI: 10.1007/s10928-023-09871-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Accepted: 06/15/2023] [Indexed: 07/01/2023]
Abstract
Validation of a quantitative model is a critical step in establishing confidence in the model's suitability for whatever analysis it was designed. While processes for validation are well-established in the statistical sciences, the field of quantitative systems pharmacology (QSP) has taken a more piecemeal approach to defining and demonstrating validation. Although classical statistical methods can be used in a QSP context, proper validation of a mechanistic systems model requires a more nuanced approach to what precisely is being validated, and what role said validation plays in the larger context of the analysis. In this review, we summarize current thoughts of QSP validation in the scientific community, contrast the aims of statistical validation from several contexts (including inference, pharmacometrics analysis, and machine learning) with the challenges faced in QSP analysis, and use examples from published QSP models to define different stages or levels of validation, any of which may be sufficient depending on the context at hand.
Collapse
Affiliation(s)
- Fulya Akpinar Singh
- Genmab US, Inc., 777 Scudders Mill Rd Bldg 2 4th Floor, Plainsboro, NJ, 08536, USA
| | - Nasrin Afzal
- Genmab US, Inc., 777 Scudders Mill Rd Bldg 2 4th Floor, Plainsboro, NJ, 08536, USA
| | - Shepard J Smithline
- Genmab US, Inc., 777 Scudders Mill Rd Bldg 2 4th Floor, Plainsboro, NJ, 08536, USA
| | - Craig J Thalhauser
- Genmab US, Inc., 777 Scudders Mill Rd Bldg 2 4th Floor, Plainsboro, NJ, 08536, USA.
| |
Collapse
|
4
|
Gyllingberg L, Birhane A, Sumpter DJT. The lost art of mathematical modelling. Math Biosci 2023:109033. [PMID: 37257641 DOI: 10.1016/j.mbs.2023.109033] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 05/24/2023] [Accepted: 05/24/2023] [Indexed: 06/02/2023]
Abstract
We provide a critique of mathematical biology in light of rapid developments in modern machine learning. We argue that out of the three modelling activities - (1) formulating models; (2) analysing models; and (3) fitting or comparing models to data - inherent to mathematical biology, researchers currently focus too much on activity (2) at the cost of (1). This trend, we propose, can be reversed by realising that any given biological phenomenon can be modelled in an infinite number of different ways, through the adoption of an pluralistic approach, where we view a system from multiple, different points of view. We explain this pluralistic approach using fish locomotion as a case study and illustrate some of the pitfalls - universalism, creating models of models, etc. - that hinder mathematical biology. We then ask how we might rediscover a lost art: that of creative mathematical modelling.
Collapse
Affiliation(s)
| | - Abeba Birhane
- Mozilla Foundation, 2 Harrison Street, Suite 175, San Francisco, CA 94105, USA
| | - David J T Sumpter
- Department of Information Technology, Uppsala University, Box 337, Uppsala, SE-751 05, Sweden.
| |
Collapse
|
5
|
Fronk C, Petzold L. Interpretable polynomial neural ordinary differential equations. CHAOS (WOODBURY, N.Y.) 2023; 33:043101. [PMID: 37097945 PMCID: PMC10076068 DOI: 10.1063/5.0130803] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Accepted: 03/09/2023] [Indexed: 06/19/2023]
Abstract
Neural networks have the ability to serve as universal function approximators, but they are not interpretable and do not generalize well outside of their training region. Both of these issues are problematic when trying to apply standard neural ordinary differential equations (ODEs) to dynamical systems. We introduce the polynomial neural ODE, which is a deep polynomial neural network inside of the neural ODE framework. We demonstrate the capability of polynomial neural ODEs to predict outside of the training region, as well as to perform direct symbolic regression without using additional tools such as SINDy.
Collapse
Affiliation(s)
- Colby Fronk
- Department of Chemical Engineering, University of California, Santa Barbara, California 93106, USA
| | - Linda Petzold
- Authors to whom correspondence should be addressed: and
| |
Collapse
|
6
|
Vittadello ST, Stumpf MPH. Open problems in mathematical biology. Math Biosci 2022; 354:108926. [PMID: 36377100 DOI: 10.1016/j.mbs.2022.108926] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Revised: 10/21/2022] [Accepted: 10/21/2022] [Indexed: 11/06/2022]
Abstract
Biology is data-rich, and it is equally rich in concepts and hypotheses. Part of trying to understand biological processes and systems is therefore to confront our ideas and hypotheses with data using statistical methods to determine the extent to which our hypotheses agree with reality. But doing so in a systematic way is becoming increasingly challenging as our hypotheses become more detailed, and our data becomes more complex. Mathematical methods are therefore gaining in importance across the life- and biomedical sciences. Mathematical models allow us to test our understanding, make testable predictions about future behaviour, and gain insights into how we can control the behaviour of biological systems. It has been argued that mathematical methods can be of great benefit to biologists to make sense of data. But mathematics and mathematicians are set to benefit equally from considering the often bewildering complexity inherent to living systems. Here we present a small selection of open problems and challenges in mathematical biology. We have chosen these open problems because they are of both biological and mathematical interest.
Collapse
Affiliation(s)
- Sean T Vittadello
- Melbourne Integrative Genomics, University of Melbourne, Australia; School of BioSciences, University of Melbourne, Australia
| | - Michael P H Stumpf
- Melbourne Integrative Genomics, University of Melbourne, Australia; School of BioSciences, University of Melbourne, Australia; School of Mathematics and Statistics, University of Melbourne, Australia.
| |
Collapse
|
7
|
Chen F, Li C. Inferring structural and dynamical properties of gene networks from data with deep learning. NAR Genom Bioinform 2022; 4:lqac068. [PMID: 36110897 PMCID: PMC9469930 DOI: 10.1093/nargab/lqac068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Revised: 07/22/2022] [Accepted: 08/24/2022] [Indexed: 11/29/2022] Open
Abstract
The reconstruction of gene regulatory networks (GRNs) from data is vital in systems biology. Although different approaches have been proposed to infer causality from data, some challenges remain, such as how to accurately infer the direction and type of interactions, how to deal with complex network involving multiple feedbacks, as well as how to infer causality between variables from real-world data, especially single cell data. Here, we tackle these problems by deep neural networks (DNNs). The underlying regulatory network for different systems (gene regulations, ecology, diseases, development) can be successfully reconstructed from trained DNN models. We show that DNN is superior to existing approaches including Boolean network, Random Forest and partial cross mapping for network inference. Further, by interrogating the ensemble DNN model trained from single cell data from dynamical system perspective, we are able to unravel complex cell fate dynamics during preimplantation development. We also propose a data-driven approach to quantify the energy landscape for gene regulatory systems, by combining DNN with the partial self-consistent mean field approximation (PSCA) approach. We anticipate the proposed method can be applied to other fields to decipher the underlying dynamical mechanisms of systems from data.
Collapse
Affiliation(s)
- Feng Chen
- Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai 200433, China
- Shanghai Center for Mathematical Sciences, Fudan University, Shanghai 200433, China
| | - Chunhe Li
- Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai 200433, China
- Shanghai Center for Mathematical Sciences, Fudan University, Shanghai 200433, China
- School of Mathematical Sciences, Fudan University, Shanghai 200433, China
| |
Collapse
|