Mathematical Modeling and Parameter Estimation of Intracellular Signaling Pathway: Application to LPS-induced NFκB Activation and TNFα Production in Macrophages

Hybrid modeling in bioprocess dynamics: Structural variabilities, implementation strategies, and practical challenges

Hybrid modeling, with an appropriate blend of the mechanistic and data-driven framework, is increasingly being adopted in bioprocess modeling, model-based experimental design (digital-twin), identification of critical process parameters, and optimization. However, the development of a hybrid model from experimental data is an inherently complex workflow, involving designed experiments, selection of the data-driven process, identification of model parameters, assessment fitness, and generalization capability. Depending on the complexity of the process system and purpose, each piece of these modules can flexibly be incorporated into the puzzle. However, this extra flexibility can be a cause of concern to trace an "optimal" model structure. In this paper, the development of hybrid models in a common bioprocess system, selection of data-driven components and their mapping to states, choice of parameter identification techniques, and model quality assurance are revisited. The challenges associated with hybrid-model development, and corrective actions have also been reviewed. The review also suggests the lack of data, and code sharing in communal repositories can be a hurdle in the exploration, and expansion of those tools in a bioprocess system.

Identifying cell-to-cell variability in internalization using flow cytometry

Biological heterogeneity is a primary contributor to the variation observed in experiments that probe dynamical processes, such as the internalization of material by cells. Given that internalization is a critical process by which many therapeutics and viruses reach their intracellular site of action, quantifying cell-to-cell variability in internalization is of high biological interest. Yet, it is common for studies of internalization to neglect cell-to-cell variability. We develop a simple mathematical model of internalization that captures the dynamical behaviour, cell-to-cell variation, and extrinsic noise introduced by flow cytometry. We calibrate our model through a novel distribution-matching approximate Bayesian computation algorithm to flow cytometry data of internalization of anti-transferrin receptor antibody in a human B-cell lymphoblastoid cell line. This approach provides information relating to the region of the parameter space, and consequentially the nature of cell-to-cell variability, that produces model realizations consistent with the experimental data. Given that our approach is agnostic to sample size and signal-to-noise ratio, our modelling framework is broadly applicable to identify biological variability in single-cell data from internalization assays and similar experiments that probe cellular dynamical processes.

Dynamic Modeling of the Impact of Temperature Changes on CO2 Production during Milk Fermentation in Batch Bioreactors

Knowledge of the mathematical models of the fermentation processes is indispensable for their simulation and optimization and for the design and synthesis of the applicable control systems. The paper focuses on determining a dynamic mathematical model of the milk fermentation process taking place in a batch bioreactor. Models in the literature describe milk fermentation in batch bioreactors as an autonomous system. They do not enable the analysis of the effect of temperature changes on the metabolism during fermentation. In the presented extensive multidisciplinary study, we have developed a new mathematical model that considers the impact of temperature changes on the dynamics of the CO₂ produced during fermentation in the batch bioreactor. Based on laboratory tests and theoretical analysis, the appropriate structure of the temperature-considered dynamic model was first determined. Next, the model parameters of the fermentation process in the laboratory bioreactor were identified by means of particle swarm optimization. Finally, the experiments with the laboratory batch bioreactor were compared with the simulations to verify the derived mathematical model. The developed model proved to be very suitable for simulations, and, above all, it enables the design and synthesis of a control system for batch bioreactors.

Model Parameterization with Quantitative Proteomics: Case Study with Trehalose Metabolism in Saccharomyces cerevisiae

When Saccharomyces cerevisiae undergoes heat stress it stimulates several changes that are necessary for its survival, notably in carbon metabolism. Notable changes include increase in trehalose production and glycolytic flux. The increase in glycolytic flux has been postulated to be due to the regulatory effects in upper glycolysis, but this has not been confirmed. Additionally, trehalose is a useful industrial compound for its protective properties. A model of trehalose metabolism in S. cerevisiae was constructed using Convenient Modeller, a software that uses a combination of convenience kinetics and a genetic algorithm. The model was parameterized with quantitative omics under standard conditions and validated using data collected under heat stress conditions. The completed model was used to show that feedforward activation of pyruvate kinase by fructose 1,6-bisphosphate during heat stress contributes to the increase in metabolic flux. We were also able to demonstrate in silico that overexpression of enzymes involved in production and degradation of trehalose can lead to higher trehalose yield in the cell. By integrating quantitative proteomics with metabolic modelling, we were able to confirm that the flux increase in trehalose metabolic pathways during heat stress is due to regulatory effects and not purely changes in enzyme expression. The overexpression of enzymes involved in trehalose metabolism is a potential approach to be exploited for trehalose production without need for increasing temperature.

Development of a hybrid model for a partially known intracellular signaling pathway through correction term estimation and neural network modeling

Developing an accurate first-principle model is an important step in employing systems biology approaches to analyze an intracellular signaling pathway. However, an accurate first-principle model is difficult to be developed since it requires in-depth mechanistic understandings of the signaling pathway. Since underlying mechanisms such as the reaction network structure are not fully understood, significant discrepancy exists between predicted and actual signaling dynamics. Motivated by these considerations, this work proposes a hybrid modeling approach that combines a first-principle model and an artificial neural network (ANN) model so that predictions of the hybrid model surpass those of the original model. First, the proposed approach determines an optimal subset of model states whose dynamics should be corrected by the ANN by examining the correlation between each state and outputs through relative order. Second, an L2-regularized least-squares problem is solved to infer values of the correction terms that are necessary to minimize the discrepancy between the model predictions and available measurements. Third, an ANN is developed to generalize relationships between the values of the correction terms and the system dynamics. Lastly, the original first-principle model is coupled with the developed ANN to finalize the hybrid model development so that the model will possess generalized prediction capabilities while retaining the model interpretability. We have successfully validated the proposed methodology with two case studies, simplified apoptosis and lipopolysaccharide-induced NFκB signaling pathways, to develop hybrid models with in silico and in vitro measurements, respectively.

An intracellular signaling pathway is often represented by a set of nonlinear ordinary differential equations, which translate our current knowledge about the signaling pathway into a testable mathematical model. However, predictions from such models are often subject to high uncertainty since many signaling pathways are only partially known beforehand. In this study, we propose a systematic approach to develop a hybrid model to improve model accuracy by combining machine learning and the first-principle modeling. Specifically, model correction terms are learned from discrepancy between model predictions and measurements, and these terms are added to the first-principle model to enhance the prediction accuracy. Once these correction terms are learned from the data, an artificial neural network (ANN) model is developed to find an empirical relation between the model and the correction terms so that the developed ANN can be used to posses improved predictive capabilities even in new operating conditions (i.e., generalizability). The final hybrid model is then constructed by coupling the first-principle model with the developed ANN.

Combining model structure identification and hybrid modelling for photo-production process predictive simulation and optimisation

Integrating physical knowledge and machine learning is a critical aspect of developing industrially focused digital twins for monitoring, optimisation, and design of microalgal and cyanobacterial photo-production processes. However, identifying the correct model structure to quantify the complex biological mechanism poses a severe challenge for the construction of kinetic models, while the lack of data due to the time-consuming experiments greatly impedes applications of most data-driven models. This study proposes the use of an innovative hybrid modelling approach that consists of a simple kinetic model to govern the overall process dynamic trajectory and a data-driven model to estimate mismatch between the kinetic equations and the real process. An advanced automatic model structure identification strategy is adopted to simultaneously identify the most physically probable kinetic model structure and minimum number of data-driven model parameters that can accurately represent multiple data sets over a broad spectrum of process operating conditions. Through this hybrid modelling and automatic structure identification framework, a highly accurate mathematical model was constructed to simulate and optimise an algal lutein production process. Performance of this hybrid model for long-term predictive modelling, optimisation, and online self-calibration is demonstrated and thoroughly discussed, indicating its significant potential for future industrial application.

Identification of a time-varying intracellular signalling model through data clustering and parameter selection: application to NF-[inline-formula removed]B signalling pathway induced by LPS in the presence of BFA

Developing a model for a signalling pathway requires several iterations of experimentation and model refinement to obtain an accurate model. However, the implementation of such an approach to model a signalling pathway induced by a poorly-known stimulus can become labour intensive because only limited information on the pathway is available beforehand to formulate an initial model. Therefore, a large number of iterations are required since the initial model is likely to be erroneous. In this work, a numerical scheme is proposed to construct a time-varying model for a signalling pathway induced by a poorly-known stimulus when its nominal model is available in the literature. Here, the nominal model refers to one that describes the signalling dynamics under a well-characterised stimulus. First, global sensitivity analysis is implemented on the nominal model to identify the most important parameters, which are assumed to be piecewise constants. Second, measurement data are clustered to determine temporal subdomains where the parameters take different values. Finally, a least-squares problem is solved to estimate the parameter values in each temporal subdomain. The effectiveness of this approach is illustrated by developing a time-varying model for NF-[inline-formula removed]B signalling dynamics induced by lipopolysaccharide in the presence of brefeldin A.

Learning-accelerated discovery of immune-tumour interactions

We present an integrated framework for enabling dynamic exploration of design spaces for cancer immunotherapies with detailed dynamical simulation models on high-performance computing resources. Our framework combines PhysiCell, an open source agent-based simulation platform for cancer and other multicellular systems, and EMEWS, an open source platform for extreme-scale model exploration. We build an agent-based model of immunosurveillance against heterogeneous tumours, which includes spatial dynamics of stochastic tumour-immune contact interactions. We implement active learning and genetic algorithms using high-performance computing workflows to adaptively sample the model parameter space and iteratively discover optimal cancer regression regions within biological and clinical constraints.

Special Issue on “Biological Networks”

Networks of coordinated interactions among biological entities govern a myriad of biological functions that span a wide range of both length and time scales—from ecosystems to individual cells, and from years (e.g., the life cycle of periodical cicadas) to milliseconds (e.g., allosteric enzymeregulation[...]

Successive complementary model-based experimental designs for parameter estimation of fed-batch bioreactors

When a dynamic model is used for the description of (fed-)batch bioreactors, it is typical that the model parameters are highly correlated to each other. In this case, it is important to keep the parameter correlation as small as possible to obtain a reliable set of parameter estimates. In this study, we propose an anticorrelation parameter estimation scheme that can be best utilized when a number of different batch experiments are sequentially processed. The scheme iteratively performs parameter estimation and model-based design of experiment (MBDOE) at the beginning and between the batches. The important difference from the existing approaches is that the MBDOE objective is defined according to the system analysis performed a priori, so that each new batch supplements what is lacking from the previous batches combined, in terms of information. The use of the scheme is illustrated on a fed-batch bioreactor model.

Sequential Parameter Estimation for Mammalian Cell Model Based on In Silico Design of Experiments

Due to the complicated metabolism of mammalian cells, the corresponding dynamic mathematical models usually consist of large sets of differential and algebraic equations with a large number of parameters to be estimated. On the other hand, the measured data for estimating the model parameters are limited. Consequently, the parameter estimates may converge to a local minimum far from the optimal ones, especially when the initial guesses of the parameter values are poor. The methodology presented in this paper provides a systematic way for estimating parameters sequentially that generates better initial guesses for parameter estimation and improves the accuracy of the obtained metabolic model. The model parameters are first classified into four subsets of decreasing importance, based on the sensitivity of the model’s predictions on the parameters’ assumed values. The parameters in the most sensitive subset, typically a small fraction of the total, are estimated first. When estimating the remaining parameters with next most sensitive subset, the subsets of parameters with higher sensitivities are estimated again using their previously obtained optimal values as the initial guesses. The power of this sequential estimation approach is illustrated through a case study on the estimation of parameters in a dynamic model of CHO cell metabolism in fed-batch culture. We show that the sequential parameter estimation approach improves model accuracy and that using limited data to estimate low-sensitivity parameters can worsen model performance.