Systems biology informed deep learning for inferring parameters and hidden dynamics

Deep fuzzy physics-informed neural networks for forward and inverse PDE problems

As a grid-independent approach for solving partial differential equations (PDEs), Physics-Informed Neural Networks (PINNs) have garnered significant attention due to their unique capability to simultaneously learn from both data and the governing physical equations. Existing PINNs methods always assume that the data is stable and reliable, but data obtained from commercial simulation software often inevitably have ambiguous and inaccurate problems. Obviously, this will have a negative impact on the use of PINNs to solve forward and inverse PDE problems. To overcome the above problems, this paper proposes a Deep Fuzzy Physics-Informed Neural Networks (FPINNs) that explores the uncertainty in data. Specifically, to capture the uncertainty behind the data, FPINNs learns fuzzy representation through the fuzzy membership function layer and fuzzy rule layer. Afterward, we use deep neural networks to learn neural representation. Subsequently, the fuzzy representation is integrated with the neural representation. Finally, the residual of the physical equation and the data error are considered as the two components of the loss function, guiding the network to optimize towards adherence to the physical laws for accurate prediction of the physical field. Extensive experiment results show that FPINNs outperforms these comparative methods in solving forward and inverse PDE problems on four widely used datasets. The demo code will be released at https://github.com/siyuancncd/FPINNs.

Identifying effective evolutionary strategies-based protocol for uncovering reaction kinetic parameters under the effect of measurement noises

BACKGROUND

The transition from explanative modeling of fitted data to the predictive modeling of unseen data for systems biology endeavors necessitates the effective recovery of reaction parameters. Yet, the relative efficacy of optimization algorithms in doing so remains under-studied, as to the specific reaction kinetics and the effect of measurement noises. To this end, we simulate the reactions of an artificial pathway using 4 kinetic formulations: generalized mass action (GMA), Michaelis-Menten, linear-logarithmic, and convenience kinetics. We then compare the effectiveness of 5 evolutionary algorithms (CMAES, DE, SRES, ISRES, G3PCX) for objective function optimization in kinetic parameter hyperspace to determine the corresponding estimated parameters.

RESULTS

We quickly dropped the DE algorithm due to its poor performance. Baring measurement noise, we find the CMAES algorithm to only require a fraction of the computational cost incurred by other EAs for both GMA and linear-logarithmic kinetics yet performing as well by other criteria. However, with increasing noise, SRES and ISRES perform more reliably for GMA kinetics, but at considerably higher computational cost. Conversely, G3PCX is among the most efficacious for estimating Michaelis-Menten parameters regardless of noise, while achieving numerous folds saving in computational cost. Cost aside, we find SRES to be versatilely applicable across GMA, Michaelis-Menten, and linear-logarithmic kinetics, with good resilience to noise. Nonetheless, we could not identify the parameters of convenience kinetics using any algorithm.

CONCLUSIONS

Altogether, we identify a protocol for predicting reaction parameters under marked measurement noise, as a step towards predictive modeling for systems biology endeavors.

Mathematical Modeling and Inference of Epidermal Growth Factor-Induced Mitogen-Activated Protein Kinase Cell Signaling Pathways

The mitogen-activated protein kinase (MAPK) pathway is an important intracellular signaling cascade that plays a key role in various cellular processes. Understanding the regulatory mechanisms of this pathway is essential for developing effective interventions and targeted therapies for related diseases. Recent advances in single-cell proteomic technologies have provided unprecedented opportunities to investigate the heterogeneity and noise within complex, multi-signaling networks across diverse cells and cell types. Mathematical modeling has become a powerful interdisciplinary tool that bridges mathematics and experimental biology, providing valuable insights into these intricate cellular processes. In addition, statistical methods have been developed to infer pathway topologies and estimate unknown parameters within dynamic models. This review presents a comprehensive analysis of how mathematical modeling of the MAPK pathway deepens our understanding of its regulatory mechanisms, enhances the prediction of system behavior, and informs experimental research, with a particular focus on recent advances in modeling and inference using single-cell proteomic data.

Kolmogorov n-widths for multitask physics-informed machine learning (PIML) methods: Towards robust metrics

Physics-informed machine learning (PIML) as a means of solving partial differential equations (PDEs) has garnered much attention in the Computational Science and Engineering (CS&E) world. This topic encompasses a broad array of methods and models aimed at solving a single or a collection of PDE problems, called multitask learning. PIML is characterized by the incorporation of physical laws into the training process of machine learning models in lieu of large data when solving PDE problems. Despite the overall success of this collection of methods, it remains incredibly difficult to analyze, benchmark, and generally compare one approach to another. Using Kolmogorov n-widths as a measure of effectiveness of approximating functions, we judiciously apply this metric in the comparison of various multitask PIML architectures. We compute lower accuracy bounds and analyze the model's learned basis functions on various PDE problems. This is the first objective metric for comparing multitask PIML architectures and helps remove uncertainty in model validation from selective sampling and overfitting. We also identify avenues of improvement for model architectures, such as the choice of activation function, which can drastically affect model generalization to "worst-case" scenarios, which is not observed when reporting task-specific errors. We also incorporate this metric into the optimization process through regularization, which improves the models' generalizability over the multitask PDE problem.

Learning metabolic dynamics from irregular observations by Bidirectional Time-Series State Transfer Network

Modeling microbial metabolic dynamics is important for the rational optimization of both biosynthetic systems and industrial processes to facilitate green and efficient biomanufacturing. Classical approaches utilize explicit equation systems to represent metabolic networks, enabling the quantification of pathway fluxes to identify metabolic bottlenecks. However, these white-box models, despite their diverse applications, have limitations in simulating metabolic dynamics and are intrinsically inaccurate for industrial strains that lack information on network structures and kinetic parameters. On the other hand, black-box models do not rely on prior mechanistic knowledge of strains but are built upon observed time-series trajectories of biosynthetic systems in action. In practice, these observations are typically irregular, with discontinuously observed time points across multiple independent batches, each time point potentially containing missing measurements. Learning from such irregular data remains challenging for existing approaches. To address this issue, we present the Bidirectional Time-Series State Transfer Network (BTSTN) for modeling metabolic dynamics directly from irregular observations. Using evaluation data sets derived from both ideal dynamic systems and a real-world fermentation process, we demonstrate that BTSTN accurately reconstructs dynamic behaviors and predicts future trajectories. This approach exhibits enhanced robustness against missing measurements and noise, as compared to the state-of-the-art methods.IMPORTANCEIndustrial biosynthetic systems often involve strains with unclear genetic backgrounds, posing challenges in modeling their distinct metabolic dynamics. In such scenarios, white-box models, which commonly rely on inferred networks, are thereby of limited applicability and accuracy. In contrast, black-box models, such as statistical models and neural networks, are directly fitted or learned from observed time-series trajectories of biosynthetic systems in action. These methods typically assume regular observations without missing time points or measurements. If the observations are irregular, a pre-processing step becomes necessary to obtain a fully filled data set for subsequent model training, which, at the same time, inevitably introduces errors into the resulting models. BTSTN is a novel approach that natively learns from irregular observations. This distinctive feature makes it a unique addition to the current arsenal of technologies modeling metabolic dynamics.

Identifying Heterogeneous Micromechanical Properties of Biological Tissues via Physics-Informed Neural Networks

The heterogeneous micromechanical properties of biological tissues have profound implications across diverse medical and engineering domains. However, identifying full-field heterogeneous elastic properties of soft materials using traditional engineering approaches is fundamentally challenging due to difficulties in estimating local stress fields. Recently, there has been a growing interest in data-driven models for learning full-field mechanical responses, such as displacement and strain, from experimental or synthetic data. However, research studies on inferring full-field elastic properties of materials, a more challenging problem, are scarce, particularly for large deformation, hyperelastic materials. Here, a physics-informed machine learning approach is proposed to identify the elasticity map in nonlinear, large deformation hyperelastic materials. This study reports the prediction accuracies and computational efficiency of physics-informed neural networks (PINNs) in inferring the heterogeneous elasticity maps across materials with structural complexity that closely resemble real tissue microstructure, such as brain, tricuspid valve, and breast cancer tissues. Further, the improved architecture is applied to three hyperelastic constitutive models: Neo-Hookean, Mooney Rivlin, and Gent. The improved network architecture consistently produces accurate estimations of heterogeneous elasticity maps, even when there is up to 10% noise present in the training data.

Identifying heterogeneous micromechanical properties of biological tissues via physics-informed neural networks

The heterogeneous micromechanical properties of biological tissues have profound implications across diverse medical and engineering domains. However, identifying full-field heterogeneous elastic properties of soft materials using traditional engineering approaches is fundamentally challenging due to difficulties in estimating local stress fields. Recently, there has been a growing interest in using data-driven models to learn full-field mechanical responses such as displacement and strain from experimental or synthetic data. However, research studies on inferring full-field elastic properties of materials, a more challenging problem, are scarce, particularly for large deformation, hyperelastic materials. Here, we propose a physics-informed machine learning approach to identify the elasticity map in nonlinear, large deformation hyperelastic materials. We evaluate the prediction accuracies and computational efficiency of physics-informed neural networks (PINNs) by inferring the heterogeneous elasticity maps across three materials with structural complexity that closely resemble real tissue patterns, such as brain tissue and tricuspid valve tissue. We further applied our improved architecture to three additional examples of breast cancer tissue and extended our analysis to three hyperelastic constitutive models: Neo-Hookean, Mooney Rivlin, and Gent. Our selected network architecture consistently produced highly accurate estimations of heterogeneous elasticity maps, even when there was up to 10% noise present in the training data.

Exploring gene regulation and biological processes in insects: Insights from omics data using gene regulatory network models

Gene regulatory network (GRN) comprises complicated yet intertwined gene-regulator relationships. Understanding the GRN dynamics will unravel the complexity behind the observed gene expressions. Insect gene regulation is often complicated due to their complex life cycles and diverse ecological adaptations. The main interest of this review is to have an update on the current mathematical modelling methods of GRNs to explain insect science. Several popular GRN architecture models are discussed, together with examples of applications in insect science. In the last part of this review, each model is compared from different aspects, including network scalability, computation complexity, robustness to noise and biological relevancy.

Bridging systems biology and tissue engineering: Unleashing the full potential of complex 3D in vitro tissue models of disease

Rapid advances in tissue engineering have resulted in more complex and physiologically relevant 3D in vitro tissue models with applications in fundamental biology and therapeutic development. However, the complexity provided by these models is often not leveraged fully due to the reductionist methods used to analyze them. Computational and mathematical models developed in the field of systems biology can address this issue. Yet, traditional systems biology has been mostly applied to simpler in vitro models with little physiological relevance and limited cellular complexity. Therefore, integrating these two inherently interdisciplinary fields can result in new insights and move both disciplines forward. In this review, we provide a systematic overview of how systems biology has been integrated with 3D in vitro tissue models and discuss key application areas where the synergies between both fields have led to important advances with potential translational impact. We then outline key directions for future research and discuss a framework for further integration between fields.

A review of mechanistic learning in mathematical oncology

Mechanistic learning refers to the synergistic combination of mechanistic mathematical modeling and data-driven machine or deep learning. This emerging field finds increasing applications in (mathematical) oncology. This review aims to capture the current state of the field and provides a perspective on how mechanistic learning may progress in the oncology domain. We highlight the synergistic potential of mechanistic learning and point out similarities and differences between purely data-driven and mechanistic approaches concerning model complexity, data requirements, outputs generated, and interpretability of the algorithms and their results. Four categories of mechanistic learning (sequential, parallel, extrinsic, intrinsic) of mechanistic learning are presented with specific examples. We discuss a range of techniques including physics-informed neural networks, surrogate model learning, and digital twins. Example applications address complex problems predominantly from the domain of oncology research such as longitudinal tumor response predictions or time-to-event modeling. As the field of mechanistic learning advances, we aim for this review and proposed categorization framework to foster additional collaboration between the data- and knowledge-driven modeling fields. Further collaboration will help address difficult issues in oncology such as limited data availability, requirements of model transparency, and complex input data which are embraced in a mechanistic learning framework.

AI-Aristotle: A physics-informed framework for systems biology gray-box identification

Discovering mathematical equations that govern physical and biological systems from observed data is a fundamental challenge in scientific research. We present a new physics-informed framework for parameter estimation and missing physics identification (gray-box) in the field of Systems Biology. The proposed framework-named AI-Aristotle-combines the eXtreme Theory of Functional Connections (X-TFC) domain-decomposition and Physics-Informed Neural Networks (PINNs) with symbolic regression (SR) techniques for parameter discovery and gray-box identification. We test the accuracy, speed, flexibility, and robustness of AI-Aristotle based on two benchmark problems in Systems Biology: a pharmacokinetics drug absorption model and an ultradian endocrine model for glucose-insulin interactions. We compare the two machine learning methods (X-TFC and PINNs), and moreover, we employ two different symbolic regression techniques to cross-verify our results. To test the performance of AI-Aristotle, we use sparse synthetic data perturbed by uniformly distributed noise. More broadly, our work provides insights into the accuracy, cost, scalability, and robustness of integrating neural networks with symbolic regressors, offering a comprehensive guide for researchers tackling gray-box identification challenges in complex dynamical systems in biomedicine and beyond.

Inferring biophysical properties of membranes during endocytosis using machine learning

Endocytosis is a fundamental cellular process in eukaryotic cells that facilitates the transport of molecules into the cell. With the help of fluorescence microscopy and electron tomography, researchers have accumulated extensive geometric data of membrane shapes during endocytosis. These data contain rich information about the mechanical properties of membranes, which are hard to access via experiments due to the small dimensions of the endocytic patch. In this study, we propose an approach that combines machine learning with the Helfrich theory of membranes to infer the mechanical properties of membranes during endocytosis from a dataset of membrane shapes extracted from electron tomography. Our results demonstrate that machine learning can output solutions that both match the experimental profile and satisfy the membrane shape equations derived from Helfrich theory. The learning results show that during the early stage of endocytosis, the inferred membrane tension is negative, indicating the presence of strong compressive forces at the boundary of the endocytic invagination. Our method presents a generic framework for extracting membrane information from super-resolution imaging.

Data-driven discovery of sparse dynamical model of cardiovascular system for model predictive control

Cardiovascular diseases remain the leading cause of death globally. In recent years, vagal nerve stimulation (VNS) has shown promising results in the treatment of a number of cardiovascular diseases. In this approach, mild electrical pulses are sent to the brain via the vagus nerve. This open-loop neurostimulation, however, leads to various side effects due to physiological and inter-patient variability and therefore a closed-loop delivery strategy of electrical pulses that accounts for this variability is desired. In this context, we envision data-driven sparse dynamical model parameterized by patient-specific data as appropriate for use in closed loop controller design. In this work, we build a dynamical model for mean arterial pressure and heart rate using the method sparse identification of nonlinear dynamics (SINDy). As a proxy for real datasets or measurements from a patient, we simulate a mechanistic model from the literature and then discover a data-driven model for predicting mean arterial pressure and heart rate in response to neural stimulus. This discovered model is then used to design a controller to be implemented in closed-loop via model predictive control. We observe that this data-driven model is interpretable, consistent with experiments, provides insights on the sensitivity of different stimulation locations and simplifies the formulation of the optimal control problem. Noting the set-point tracking performance of this closed-loop model-based controller that uses this discovered model, we conclude that the model is adequate in capturing the dynamics of a highly nonlinear cardiovascular system for the purpose of optimal predictive controller design.

A multi-resolution physics-informed recurrent neural network: formulation and application to musculoskeletal systems

This work presents a multi-resolution physics-informed recurrent neural network (MR PI-RNN), for simultaneous prediction of musculoskeletal (MSK) motion and parameter identification of the MSK systems. The MSK application was selected as the model problem due to its challenging nature in mapping the high-frequency surface electromyography (sEMG) signals to the low-frequency body joint motion controlled by the MSK and muscle contraction dynamics. The proposed method utilizes the fast wavelet transform to decompose the mixed frequency input sEMG and output joint motion signals into nested multi-resolution signals. The prediction model is subsequently trained on coarser-scale input-output signals using a gated recurrent unit (GRU), and then the trained parameters are transferred to the next level of training with finer-scale signals. These training processes are repeated recursively under a transfer-learning fashion until the full-scale training (i.e., with unfiltered signals) is achieved, while satisfying the underlying dynamic equilibrium. Numerical examples on recorded subject data demonstrate the effectiveness of the proposed framework in generating a physics-informed forward-dynamics surrogate, which yields higher accuracy in motion predictions of elbow flexion-extension of an MSK system compared to the case with single-scale training. The framework is also capable of identifying muscle parameters that are physiologically consistent with the subject's kinematics data.

A topological classifier to characterize brain states: When shape matters more than variance

Despite the remarkable accuracies attained by machine learning classifiers to separate complex datasets in a supervised fashion, most of their operation falls short to provide an informed intuition about the structure of data, and, what is more important, about the phenomena being characterized by the given datasets. By contrast, topological data analysis (TDA) is devoted to study the shape of data clouds by means of persistence descriptors and provides a quantitative characterization of specific topological features of the dataset under scrutiny. Here we introduce a novel TDA-based classifier that works on the principle of assessing quantifiable changes on topological metrics caused by the addition of new input to a subset of data. We used this classifier with a high-dimensional electro-encephalographic (EEG) dataset recorded from eleven participants during a previous decision-making experiment in which three motivational states were induced through a manipulation of social pressure. We calculated silhouettes from persistence diagrams associated with each motivated state with a ready-made band-pass filtered version of these signals, and classified unlabeled signals according to their impact on each reference silhouette. Our results show that in addition to providing accuracies within the range of those of a nearest neighbour classifier, the TDA classifier provides formal intuition of the structure of the dataset as well as an estimate of its intrinsic dimension. Towards this end, we incorporated variance-based dimensionality reduction methods to our dataset and found that in most cases the accuracy of our TDA classifier remains essentially invariant beyond a certain dimension.

Transmission dynamics informed neural network with application to COVID-19 infections

Since the end of 2019 the COVID-19 repeatedly surges with most countries/territories experiencing multiple waves, and mechanism-based epidemic models played important roles in understanding the transmission mechanism of multiple epidemic waves. However, capturing temporal changes of the transmissibility of COVID-19 during the multiple waves keeps ill-posed problem for traditional mechanism-based epidemic compartment models, because that the transmission rate is usually assumed to be specific piecewise functions and more parameters are added to the model once multiple epidemic waves involved, which poses a huge challenge to parameter estimation. Meanwhile, data-driven deep neural networks fail to discover the driving factors of repeated outbreaks and lack interpretability. In this study, aiming at developing a data-driven method to project time-dependent parameters but also merging the advantage of mechanism-based models, we propose a transmission dynamics informed neural network (TDINN) by encoding the SEIRD compartment model into deep neural networks. We show that the proposed TDINN algorithm performs very well when fitting the COVID-19 epidemic data with multiple waves, where the epidemics in the United States, Italy, South Africa, and Kenya, and several outbreaks the Omicron variant in China are taken as examples. In addition, the numerical simulation shows that the trained TDINN can also perform as a predictive model to capture the future development of COVID-19 epidemic. We find that the transmission rate inferred by the TDINN frequently fluctuates, and a feedback loop between the epidemic shifting and the changes of transmissibility drives the occurrence of multiple waves. We observe a long response delay to the implementation of control interventions in the four countries, while the decline of the transmission rate in the outbreaks in China usually happens once the implementation of control interventions. The further simulation show that 17 days' delay of the response to the implementation of control interventions lead to a roughly four-fold increase in daily reported cases in one epidemic wave in Italy, which suggest that a rapid response to policies that strengthen control interventions can be effective in flattening the epidemic curve or avoiding subsequent epidemic waves. We observe that the transmission rate in the outbreaks in China is already decreasing before enhancing control interventions, providing the evidence that the increasing of the epidemics can drive self-conscious behavioural changes to protect against infections.

Physics-informed neural network-based serial hybrid model capturing the hidden kinetics for sulfur-driven autotrophic denitrification process

Sulfur-driven autotrophic denitrification (SdAD) is a biological process that can remove nitrate from low carbon/nitrogen (C/N) ratio wastewater. Although this process has been intensively researched, the mechanism whereby its intermediates (i.e., elemental sulfur and nitrite ions) are generated and accumulated remains elusive. Existing mathematical models developed for SdAD cannot accurately predict the intermediates in SdAD because of the incomplete knowledge of process kinetic resulting from changes in the environmental conditions and electron competition during SdAD. To address this limitation, we proposed a novel serial hybrid model structure based on a physics-informed neural network (PINN) to capture the dynamics of the process kinetics and predict the substrate concentrations in SdAD. In this study, we evaluated the model through numerical experiments and applied it to real case studies involving batch and continuous-flow reactor scenarios. By leveraging the PINN approach, the hybrid model yielded accurate predictions at both the state (i.e. substrate concentration) and kinetic levels in the numerical experiments and performed better than both mechanistic and purely data-driven models in the case studies. Furthermore, we used the trained hybrid model to design control strategies for SdAD and a novel integrated process involving SdAD and anammox for energy-efficient nitrogen removal. Finally, we discuss the advantages and application scope of the PINN-based hybrid model.

Integrating a tailored recurrent neural network with Bayesian experimental design to optimize microbial community functions

Microbiomes interact dynamically with their environment to perform exploitable functions such as production of valuable metabolites and degradation of toxic metabolites for a wide range of applications in human health, agriculture, and environmental cleanup. Developing computational models to predict the key bacterial species and environmental factors to build and optimize such functions are crucial to accelerate microbial community engineering. However, there is an unknown web of interactions that determine the highly complex and dynamic behavior of these systems, which precludes the development of models based on known mechanisms. By contrast, entirely data-driven machine learning models can produce physically unrealistic predictions and often require significant amounts of experimental data to learn system behavior. We develop a physically-constrained recurrent neural network that preserves model flexibility but is constrained to produce physically consistent predictions and show that it can outperform existing machine learning methods in the prediction of certain experimentally measured species abundance and metabolite concentrations. Further, we present a closed-loop, Bayesian experimental design algorithm to guide data collection by selecting experimental conditions that simultaneously maximize information gain and target microbial community functions. Using a bioreactor case study, we demonstrate how the proposed framework can be used to efficiently navigate a large design space to identify optimal operating conditions. The proposed methodology offers a flexible machine learning approach specifically tailored to optimize microbiome target functions through the sequential design of informative experiments that seek to explore and exploit community functions.

Recent Advances in Deep Learning for Protein-Protein Interaction Analysis: A Comprehensive Review

Deep learning, a potent branch of artificial intelligence, is steadily leaving its transformative imprint across multiple disciplines. Within computational biology, it is expediting progress in the understanding of Protein-Protein Interactions (PPIs), key components governing a wide array of biological functionalities. Hence, an in-depth exploration of PPIs is crucial for decoding the intricate biological system dynamics and unveiling potential avenues for therapeutic interventions. As the deployment of deep learning techniques in PPI analysis proliferates at an accelerated pace, there exists an immediate demand for an exhaustive review that encapsulates and critically assesses these novel developments. Addressing this requirement, this review offers a detailed analysis of the literature from 2021 to 2023, highlighting the cutting-edge deep learning methodologies harnessed for PPI analysis. Thus, this review stands as a crucial reference for researchers in the discipline, presenting an overview of the recent studies in the field. This consolidation helps elucidate the dynamic paradigm of PPI analysis, the evolution of deep learning techniques, and their interdependent dynamics. This scrutiny is expected to serve as a vital aid for researchers, both well-established and newcomers, assisting them in maneuvering the rapidly shifting terrain of deep learning applications in PPI analysis.

Effective data sampling strategies and boundary condition constraints of physics-informed neural networks for identifying material properties in solid mechanics

Material identification is critical for understanding the relationship between mechanical properties and the associated mechanical functions. However, material identification is a challenging task, especially when the characteristic of the material is highly nonlinear in nature, as is common in biological tissue. In this work, we identify unknown material properties in continuum solid mechanics via physics-informed neural networks (PINNs). To improve the accuracy and efficiency of PINNs, we develop efficient strategies to nonuniformly sample observational data. We also investigate different approaches to enforce Dirichlet-type boundary conditions (BCs) as soft or hard constraints. Finally, we apply the proposed methods to a diverse set of time-dependent and time-independent solid mechanic examples that span linear elastic and hyperelastic material space. The estimated material parameters achieve relative errors of less than 1%. As such, this work is relevant to diverse applications, including optimizing structural integrity and developing novel materials.

Parameter Inference for an Astrocyte Model using Machine Learning Approaches

Astrocytes are the largest subset of glial cells and perform structural, metabolic, and regulatory functions. They are directly involved in the communication at neuronal synapses and the maintenance of brain homeostasis. Several disorders, such as Alzheimer's, epilepsy, and schizophrenia, have been associated with astrocyte dysfunction. Computational models on various spatial levels have been proposed to aid in the understanding and research of astrocytes. The difficulty of computational astrocyte models is to fastly and precisely infer parameters. Physics informed neural networks (PINNs) use the underlying physics to infer parameters and, if necessary, dynamics that can not be observed. We have applied PINNs to estimate parameters for a computational model of an astrocytic compartment. The addition of two techniques helped with the gradient pathologies of the PINNS, the dynamic weighting of various loss components and the addition of Transformers. To overcome the issue that the neural network only learned the time dependence but did not know about eventual changes of the input stimulation to the astrocyte model, we followed an adaptation of PINNs from control theory (PINCs). In the end, we were able to infer parameters from artificial, noisy data, with stable results for the computational astrocyte model.

Integration of a multi-omics stem cell differentiation dataset using a dynamical model

Stem cell differentiation is a highly dynamic process involving pervasive changes in gene expression. The large majority of existing studies has characterized differentiation at the level of individual molecular profiles, such as the transcriptome or the proteome. To obtain a more comprehensive view, we measured protein, mRNA and microRNA abundance during retinoic acid-driven differentiation of mouse embryonic stem cells. We found that mRNA and protein abundance are typically only weakly correlated across time. To understand this finding, we developed a hierarchical dynamical model that allowed us to integrate all data sets. This model was able to explain mRNA-protein discordance for most genes and identified instances of potential microRNA-mediated regulation. Overexpression or depletion of microRNAs identified by the model, followed by RNA sequencing and protein quantification, were used to follow up on the predictions of the model. Overall, our study shows how multi-omics integration by a dynamical model could be used to nominate candidate regulators.

Modal wavenumber estimation by combining physical informed neural network

Estimation of modal wavenumbers is important for inference of geoacoustic properties and data-driven matched field processing in shallow water waveguides. This paper introduces a deep neural network called combining physical informed neural network (CPINN) for modal wavenumber estimation using a vertical linear array (VLA). Note that the sound field recorded by a VLA can be expressed as a linear superposition of finite modal depth functions, and the differential equations satisfied by the modal depth functions are related to the corresponding modal wavenumbers. CPINN can estimate the modal wavenumbers by introducing the proxies of the modal depth functions and constraining them to satisfy the corresponding differential equations. The performance of the CPINN is evaluated by simulated data in a noisy shallow water environment. Numerical results show that, when compared with the previous methods, CPINN does not need to know the exact horizontal distance between the sound source and the VLA. Moreover, CPINN can estimate the modal wavenumbers at the VLA position in the case where the range segment traversed by the source, i.e., the aperture in the range direction, is smaller than the maximum modal cycle distance and in a range-dependent waveguide.

Physics-informed neural entangled-ladder network for inhalation impedance of the respiratory system

BACKGROUND AND OBJECTIVES

The use of machine learning methods for modelling bio-systems is becoming prominent which can further improve bio-medical technologies. Physics-informed neural networks (PINNs) can embed the knowledge of physical laws that govern a system during the model training process. PINNs utilise differential equations in the model which traditionally used numerical methods that are computationally complex.

METHODS

We integrate PINNs with an entangled ladder network for modelling respiratory systems by considering a lungs conduction zone to evaluate the respiratory impedance for different initial conditions. We evaluate the respiratory impedance for the inhalation phase of breathing for a symmetric model of the human lungs using entanglement and continued fractions.

RESULTS

We obtain the impedance of the conduction zone of the lungs pulmonary airways using PINNs for nine different combinations of velocity and pressure of inhalation. We compare the results from PINNs with the finite element method using the mean absolute error and root mean square error. The results show that the impedance obtained with PINNs contrasts with the conventional forced oscillation test used for deducing the respiratory impedance. The results show similarity with the impedance plots for different respiratory diseases.

CONCLUSION

We find a decrease in impedance when the velocity of breathing is lowered gradually by 20%. Hence, the methodology can be used to design smart ventilators to the improve flow of breathing.

Uncertainty quantification in mechanistic epidemic models via cross-entropy approximate Bayesian computation

This paper proposes a data-driven approximate Bayesian computation framework for parameter estimation and uncertainty quantification of epidemic models, which incorporates two novelties: (i) the identification of the initial conditions by using plausible dynamic states that are compatible with observational data; (ii) learning of an informative prior distribution for the model parameters via the cross-entropy method. The new methodology's effectiveness is illustrated with the aid of actual data from the COVID-19 epidemic in Rio de Janeiro city in Brazil, employing an ordinary differential equation-based model with a generalized SEIR mechanistic structure that includes time-dependent transmission rate, asymptomatics, and hospitalizations. A minimization problem with two cost terms (number of hospitalizations and deaths) is formulated, and twelve parameters are identified. The calibrated model provides a consistent description of the available data, able to extrapolate forecasts over a few weeks, making the proposed methodology very appealing for real-time epidemic modeling.

Crosstalk between biochemical signalling network architecture and trafficking governs AMPAR dynamics in synaptic plasticity

Synaptic plasticity involves modification of both biochemical and structural components of neurons. Many studies have revealed that the change in the number density of the glutamatergic receptor AMPAR at the synapse is proportional to synaptic weight update; an increase in AMPAR corresponds to strengthening of synapses while a decrease in AMPAR density weakens synaptic connections. The dynamics of AMPAR are thought to be regulated by upstream signalling, primarily the calcium-CaMKII pathway, trafficking to and from the synapse, and influx from extrasynaptic sources. Previous work in the field of deterministic modelling of CaMKII dynamics has assumed bistable kinetics, while experiments and rule-based modelling have revealed that CaMKII dynamics can be either monostable or ultrasensitive. This raises the following question: how does the choice of model assumptions involving CaMKII dynamics influence AMPAR dynamics at the synapse? To answer this question, we have developed a set of models using compartmental ordinary differential equations to systematically investigate contributions of different signalling and trafficking variations, along with their coupled effects, on AMPAR dynamics at the synaptic site. We find that the properties of the model including network architecture describing different stability features of CaMKII and parameters that capture the endocytosis and exocytosis of AMPAR significantly affect the integration of fast upstream species by slower downstream species. Furthermore, we predict that the model outcome, as determined by bound AMPAR at the synaptic site, depends on (1) the choice of signalling model (bistable CaMKII or monostable CaMKII dynamics), (2) trafficking versus influx contributions and (3) frequency of stimulus. KEY POINTS: The density of AMPA receptors (AMPARs) at the postsynaptic density of the synapse provides a readout of synaptic plasticity, which involves crosstalk between complex biochemical signalling networks including CaMKII dynamics and trafficking pathways including exocytosis and endocytosis. Here we build a model that integrates CaMKII dynamics and AMPAR trafficking to explore this crosstalk. We compare different models of CaMKII that result in monostable or bistable kinetics and their impact on AMPAR dynamics. Our results show that AMPAR density depends on the coupling between aspects of biochemical signalling and trafficking. Specifically, assumptions regarding CaMKII dynamics and its stability features can alter AMPAR density at the synapse. Our model also predicts that the kinetics of trafficking versus influx of AMPAR from the extrasynaptic space can further impact AMPAR density. Thus, the contributions of both signalling and trafficking should be considered in computational models.

Systems Biology: Identifiability Analysis and Parameter Identification via Systems-Biology-Informed Neural Networks

The dynamics of systems biological processes are usually modeled by a system of ordinary differential equations (ODEs) with many unknown parameters that need to be inferred from noisy and sparse measurements. Here, we introduce systems-biology-informed neural networks for parameter estimation by incorporating the system of ODEs into the neural networks. To complete the workflow of system identification, we also describe structural and practical identifiability analysis to analyze the identifiability of parameters. We use the ultradian endocrine model for glucose-insulin interaction as the example to demonstrate all these methods and their implementation.

Theranostic digital twins for personalized radiopharmaceutical therapies: Reimagining theranostics via computational nuclear oncology

This work emphasizes that patient data, including images, are not operable (clinically), but that digital twins are. Based on the former, the latter can be created. Subsequently, virtual clinical operations can be performed towards selection of optimal therapies. Digital twins are beginning to emerge in the field of medicine. We suggest that theranostic digital twins (TDTs) are amongst the most natural and feasible flavors of digitals twins. We elaborate on the importance of TDTs in a future where 'one-size-fits-all' therapeutic schemes, as prevalent nowadays, are transcended in radiopharmaceutical therapies (RPTs). Personalized RPTs will be deployed, including optimized intervention parameters. Examples include optimization of injected radioactivities, sites of injection, injection intervals and profiles, and combination therapies. Multi-modal multi-scale images, combined with other data and aided by artificial intelligence (AI) techniques, will be utilized towards routine digital twinning of our patients, and will enable improved deliveries of RPTs and overall healthcare.

A Feature-Encoded Physics-Informed Parameter Identification Neural Network for Musculoskeletal Systems

Identification of muscle-tendon force generation properties and muscle activities from physiological measurements, e.g., motion data and raw surface electromyography (sEMG), offers opportunities to construct a subject-specific musculoskeletal (MSK) digital twin system for health condition assessment and motion prediction. While machine learning approaches with capabilities in extracting complex features and patterns from a large amount of data have been applied to motion prediction given sEMG signals, the learned data-driven mapping is black-box and may not satisfy the underlying physics and has reduced generality. In this work, we propose a feature-encoded physics-informed parameter identification neural network (FEPI-PINN) for simultaneous prediction of motion and parameter identification of human MSK systems. In this approach, features of high-dimensional noisy sEMG signals are projected onto a low-dimensional noise-filtered embedding space for the enhancement of forwarding dynamics prediction. This FEPI-PINN model can be trained to relate sEMG signals to joint motion and simultaneously identify key MSK parameters. The numerical examples demonstrate that the proposed framework can effectively identify subject-specific muscle parameters and the trained physics-informed forward-dynamics surrogate yields accurate motion predictions of elbow flexion-extension motion that are in good agreement with the measured joint motion data.

Toward predictive engineering of gene circuits

Many synthetic biology applications rely on programming living cells using gene circuits - the assembly and wiring of genetic elements to control cellular behaviors. Extensive progress has been made in constructing gene circuits with diverse functions and applications. For many circuit functions, however, it remains challenging to ensure that the circuits operate in a predictable manner. Although the notion of predictability may appear intuitive, close inspection suggests that it is not always clear what constitutes predictability. We dissect this concept and how it can be confounded by the complexity of a circuit, the complexity of the context, and the interplay between the two. We discuss circuit engineering strategies, in both computation and experiment, that have been used to improve the predictability of gene circuits.

Bayesian parameter estimation for dynamical models in systems biology

Dynamical systems modeling, particularly via systems of ordinary differential equations, has been used to effectively capture the temporal behavior of different biochemical components in signal transduction networks. Despite the recent advances in experimental measurements, including sensor development and '-omics' studies that have helped populate protein-protein interaction networks in great detail, modeling in systems biology lacks systematic methods to estimate kinetic parameters and quantify associated uncertainties. This is because of multiple reasons, including sparse and noisy experimental measurements, lack of detailed molecular mechanisms underlying the reactions, and missing biochemical interactions. Additionally, the inherent nonlinearities with respect to the states and parameters associated with the system of differential equations further compound the challenges of parameter estimation. In this study, we propose a comprehensive framework for Bayesian parameter estimation and complete quantification of the effects of uncertainties in the data and models. We apply these methods to a series of signaling models of increasing mathematical complexity. Systematic analysis of these dynamical systems showed that parameter estimation depends on data sparsity, noise level, and model structure, including the existence of multiple steady states. These results highlight how focused uncertainty quantification can enrich systems biology modeling and enable additional quantitative analyses for parameter estimation.

Machine learning alternative to systems biology should not solely depend on data

In recent years, artificial intelligence (AI)/machine learning has emerged as a plausible alternative to systems biology for the elucidation of biological phenomena and in attaining specified design objective in synthetic biology. Although considered highly disruptive with numerous notable successes so far, we seek to bring attention to both the fundamental and practical pitfalls of their usage, especially in illuminating emergent behaviors from chaotic or stochastic systems in biology. Without deliberating on their suitability and the required data qualities and pre-processing approaches beforehand, the research and development community could experience similar 'AI winters' that had plagued other fields. Instead, we anticipate the integration or combination of the two approaches, where appropriate, moving forward.

Investigating molecular transport in the human brain from MRI with physics-informed neural networks

In recent years, a plethora of methods combining neural networks and partial differential equations have been developed. A widely known example are physics-informed neural networks, which solve problems involving partial differential equations by training a neural network. We apply physics-informed neural networks and the finite element method to estimate the diffusion coefficient governing the long term spread of molecules in the human brain from magnetic resonance images. Synthetic testcases are created to demonstrate that the standard formulation of the physics-informed neural network faces challenges with noisy measurements in our application. Our numerical results demonstrate that the residual of the partial differential equation after training needs to be small for accurate parameter recovery. To achieve this, we tune the weights and the norms used in the loss function and use residual based adaptive refinement of training points. We find that the diffusion coefficient estimated from magnetic resonance images with physics-informed neural networks becomes consistent with results from a finite element based approach when the residuum after training becomes small. The observations presented here are an important first step towards solving inverse problems on cohorts of patients in a semi-automated fashion with physics-informed neural networks.

Process Model Inversion in the Data-Driven Engineering Context for Improved Parameter Sensitivities

Industry 4.0 has embraced process models in recent years, and the use of model-based digital twins has become even more critical in process systems engineering, monitoring, and control. However, the reliability of these models depends on the model parameters available. The accuracy of the estimated parameters is, in turn, determined by the amount and quality of the measurement data and the algorithm used for parameter identification. For the definition of the parameter identification problem, the ordinary least squares framework is still state-of-the-art in the literature, and better parameter estimates are only possible with additional data. In this work, we present an alternative strategy to identify model parameters by incorporating differential flatness for model inversion and neural ordinary differential equations for surrogate modeling. The novel concept results in an input-least-squares-based parameter identification problem with significant parameter sensitivity changes. To study these sensitivity effects, we use a classic one-dimensional diffusion-type problem, i.e., an omnipresent equation in process systems engineering and transport phenomena. As shown, the proposed concept ensures higher parameter sensitivities for two relevant scenarios. Based on the results derived, we also discuss general implications for data-driven engineering concepts used to identify process model parameters in the recent literature.

Modelling membrane curvature generation using mechanics and machine learning

The deformation of cellular membranes regulates trafficking processes, such as exocytosis and endocytosis. Classically, the Helfrich continuum model is used to characterize the forces and mechanical parameters that cells tune to accomplish membrane shape changes. While this classical model effectively captures curvature generation, one of the core challenges in using it to approximate a biological process is selecting a set of mechanical parameters (including bending modulus and membrane tension) from a large set of reasonable values. We used the Helfrich model to generate a large synthetic dataset from a random sampling of realistic mechanical parameters and used this dataset to train machine-learning models. These models produced promising results, accurately classifying model behaviour and predicting membrane shape from mechanical parameters. We also note emerging methods in machine learning that can leverage the physical insight of the Helfrich model to improve performance and draw greater insight into how cells control membrane shape change.

Gaussian processes meet NeuralODEs: a Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data

We present a machine learning framework (GP-NODE) for Bayesian model discovery from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. The use of the Finnish Horseshoe as a sparsity-promoting prior for free model parameters also enables the discovery of parsimonious representations for the latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator-prey systems, systems biology and a 50-dimensional human motion dynamical system. This article is part of the theme issue 'Data-driven prediction in dynamical systems'.

Promoting Sustainability through Next-Generation Biologics Drug Development

The fourth industrial revolution in 2011 aimed to transform the traditional manufacturing processes. As part of this revolution, disruptive innovations in drug development and data science approaches have the potential to optimize CMC (chemistry, manufacture, and control). The real-time simulation of processes using “digital twins” can maximize efficiency while improving sustainability. As part of this review, we investigate how the World Health Organization’s 17 sustainability goals can apply toward next-generation drug development. We analyze the state-of-the-art laboratory leadership, inclusive personnel recruiting, the latest therapy approaches, and intelligent process automation. We also outline how modern data science techniques and machine tools for CMC help to shorten drug development time, reduce failure rates, and minimize resource usage. Finally, we systematically analyze and compare existing approaches to our experiences with the high-throughput laboratory KIWI-biolab at the TU Berlin. We describe a sustainable business model that accelerates scientific innovations and supports global action toward a sustainable future.

Inverse Dirichlet weighting enables reliable training of physics informed neural networks

We characterize and remedy a failure mode that may arise from multi-scale dynamics with scale imbalances during training of deep neural networks, such as physics informed neural networks (PINNs). PINNs are popular machine-learning templates that allow for seamless integration of physical equation models with data. Their training amounts to solving an optimization problem over a weighted sum of data-fidelity and equation-fidelity objectives. Conflicts between objectives can arise from scale imbalances, heteroscedasticity in the data, stiffness of the physical equation, or from catastrophic interference during sequential training. We explain the training pathology arising from this and propose a simple yet effective inverse Dirichlet weighting strategy to alleviate the issue. We compare with Sobolev training of neural networks, providing the baseline of analytically ε-optimal training. We demonstrate the effectiveness of inverse Dirichlet weighting in various applications, including a multi-scale model of active turbulence, where we show orders of magnitude improvement in accuracy and convergence over conventional PINN training. For inverse modeling using sequential training, we find that inverse Dirichlet weighting protects a PINN against catastrophic forgetting.

EP-PINNs: Cardiac Electrophysiology Characterisation Using Physics-Informed Neural Networks

Accurately inferring underlying electrophysiological (EP) tissue properties from action potential recordings is expected to be clinically useful in the diagnosis and treatment of arrhythmias such as atrial fibrillation. It is, however, notoriously difficult to perform. We present EP-PINNs (Physics Informed Neural Networks), a novel tool for accurate action potential simulation and EP parameter estimation from sparse amounts of EP data. We demonstrate, using 1D and 2D in silico data, how EP-PINNs are able to reconstruct the spatio-temporal evolution of action potentials, whilst predicting parameters related to action potential duration (APD), excitability and diffusion coefficients. EP-PINNs are additionally able to identify heterogeneities in EP properties, making them potentially useful for the detection of fibrosis and other localised pathology linked to arrhythmias. Finally, we show EP-PINNs effectiveness on biological in vitro preparations, by characterising the effect of anti-arrhythmic drugs on APD using optical mapping data. EP-PINNs are a promising clinical tool for the characterisation and potential treatment guidance of arrhythmias.

Review of applications and challenges of quantitative systems pharmacology modeling and machine learning for heart failure

Quantitative systems pharmacology (QSP) is an important approach in pharmaceutical research and development that facilitates in silico generation of quantitative mechanistic hypotheses and enables in silico trials. As demonstrated by applications from numerous industry groups and interest from regulatory authorities, QSP is becoming an increasingly critical component in clinical drug development. With rapidly evolving computational tools and methods, QSP modeling has achieved important progress in pharmaceutical research and development, including for heart failure (HF). However, various challenges exist in the QSP modeling and clinical characterization of HF. Machine/deep learning (ML/DL) methods have had success in a wide variety of fields and disciplines. They provide data-driven approaches in HF diagnosis and modeling, and offer a novel strategy to inform QSP model development and calibration. The combination of ML/DL and QSP modeling becomes an emergent direction in the understanding of HF and clinical development new therapies. In this work, we review the current status and achievement in QSP and ML/DL for HF, and discuss remaining challenges and future perspectives in the field.

Online Machine Learning for Accelerating Molecular Dynamics Modeling of Cells

We developed a biomechanics-informed online learning framework to learn the dynamics with ground truth generated with multiscale modeling simulation. It was built on Summit-like supercomputers, which were also used to benchmark and validate our framework on one physiologically significant modeling of deformable biological cells. We generalized the century-old equation of Jeffery orbits to a new equation of motion with additional parameters to account for the flow conditions and the cell deformability. Using simulation data at particle-based resolutions for flowing cells and the learned parameters from our framework, we validated the new equation by the motions, mostly rotations, of a human platelet in shear blood flow at various shear stresses and platelet deformability. Our online framework, which surrogates redundant computations in the conventional multiscale modeling by solutions of our learned equation, accelerates the conventional modeling by three orders of magnitude without visible loss of accuracy.

Teaching computational systems biology with an eye on quantitative systems pharmacology at the undergraduate level: Why do it, who would take it, and what should we teach?

Computational systems biology (CSB) is a field that emerged primarily as the product of research activities. As such, it grew in several directions in a distributed and uncoordinated manner making the area appealing and fascinating. The idea of not having to follow a specific path but instead creating one fueled innovation. As the field matured, several interdisciplinary graduate programs emerged attempting to educate future generations of computational systems biologists. These educational initiatives coordinated the dissemination of information across student populations that had already decided to specialize in this field. However, we are now entering an era where CSB, having established itself as a valuable research discipline, is attempting the next major step: Entering undergraduate curricula. As interesting as this endeavor may sound, it has several difficulties, mainly because the field is not uniformly defined. In this manuscript, we argue that this diversity is a significant advantage and that several incarnations of an undergraduate-level CSB biology course could, and should, be developed tailored to programmatic needs. In this manuscript, we share our experiences creating a course as part of a Biomedical Engineering program.

Expert-enhanced machine learning for cardiac arrhythmia classification

We propose a new method for the classification task of distinguishing atrial fibrillation (AFib) from regular atrial tachycardias including atrial flutter (AFlu) based on a surface electrocardiogram (ECG). Recently, many approaches for an automatic classification of cardiac arrhythmia were proposed and to our knowledge none of them can distinguish between these two. We discuss reasons why deep learning may not yield satisfactory results for this task. We generate new and clinically interpretable features using mathematical optimization for subsequent use within a machine learning (ML) model. These features are generated from the same input data by solving an additional regression problem with complicated combinatorial substructures. The resultant can be seen as a novel machine learning model that incorporates expert knowledge on the pathophysiology of atrial flutter. Our approach achieves an unprecedented accuracy of 82.84% and an area under the receiver operating characteristic (ROC) curve of 0.9, which classifies as "excellent" according to the classification indicator of diagnostic tests. One additional advantage of our approach is the inherent interpretability of the classification results. Our features give insight into a possibly occurring multilevel atrioventricular blocking mechanism, which may improve treatment decisions beyond the classification itself. Our research ideally complements existing textbook cardiac arrhythmia classification methods, which cannot provide a classification for the important case of AFib↔AFlu. The main contribution is the successful use of a novel mathematical model for multilevel atrioventricular block and optimization-driven inverse simulation to enhance machine learning for classification of the arguably most difficult cases in cardiac arrhythmia. A tailored Branch-and-Bound algorithm was implemented for the domain knowledge part, while standard algorithms such as Adam could be used for training.

The future of research in hematology: Integration of conventional studies with real-world data and artificial intelligence

Most national health-care systems approve new drugs based on data of safety and efficacy from large randomized clinical trials (RCTs). Strict selection biases and study-entry criteria of subjects included in RCTs often do not reflect those of the population where a therapy is intended to be used. Compliance to treatment in RCTs also differs considerably from real world settings and the relatively small size of most RCTs make them unlikely to detect rare but important safety signals. These and other considerations may explain the gap between evidence generated in RCTs and translating conclusions to health-care policies in the real world. Real-world evidence (RWE) derived from real-world data (RWD) is receiving increasing attention from scientists, clinicians, and health-care policy decision-makers - especially when it is processed by artificial intelligence (AI). We describe the potential of using RWD and AI in Hematology to support research and health-care decisions.