Optimal design of stimulus experiments for robust discrimination of biochemical reaction networks

Multi-Level Computational Modeling of Anti-Cancer Dendritic Cell Vaccination Utilized to Select Molecular Targets for Therapy Optimization

Dendritic cells (DCs) can be used for therapeutic vaccination against cancer. The success of this therapy depends on efficient tumor-antigen presentation to cytotoxic T lymphocytes (CTLs) and the induction of durable CTL responses by the DCs. Therefore, simulation of such a biological system by computational modeling is appealing because it can improve our understanding of the molecular mechanisms underlying CTL induction by DCs and help identify new strategies to improve therapeutic DC vaccination for cancer. Here, we developed a multi-level model accounting for the life cycle of DCs during anti-cancer immunotherapy. Specifically, the model is composed of three parts representing different stages of DC immunotherapy – the spreading and bio-distribution of intravenously injected DCs in human organs, the biochemical reactions regulating the DCs’ maturation and activation, and DC-mediated activation of CTLs. We calibrated the model using quantitative experimental data that account for the activation of key molecular circuits within DCs, the bio-distribution of DCs in the body, and the interaction between DCs and T cells. We showed how such a data-driven model can be exploited in combination with sensitivity analysis and model simulations to identify targets for enhancing anti-cancer DC vaccination. Since other previous works show how modeling improves therapy schedules and DC dosage, we here focused on the molecular optimization of the therapy. In line with this, we simulated the effect in DC vaccination of the concerted modulation of combined intracellular regulatory processes and proposed several possibilities that can enhance DC-mediated immunogenicity. Taken together, we present a comprehensive time-resolved multi-level model for studying DC vaccination in melanoma. Although the model is not intended for personalized patient therapy, it could be used as a tool for identifying molecular targets for optimizing DC-based therapy for cancer, which ultimately should be tested in in vitro and in vivo experiments.

Kinetic analysis of the partial synthesis of artemisinin: Photooxygenation to the intermediate hydroperoxide

The price of the currently best available antimalarial treatment is driven in large part by the limited availability of its base drug compound artemisinin. One approach to reduce the artemisinin cost is to efficiently integrate the partial synthesis of artemisinin starting from its biological precursor dihydroartemisinic acid (DHAA) into the production process. The optimal design of such an integrated process is a complex task that is easier to solve through simulations studies and process modelling. In this article, we present a quantitative kinetic model for the photooxygenation of DHAA to an hydroperoxide, the essential initial step of the partial synthesis to artemisinin. The photooxygenation reactions were studied in a two-phase photo-flow reactor utilizing Taylor flow for enhanced mixing and fast gas-liquid mass transfer. A good agreement of the model and the experimental data was achieved for all combinations of photosensitizer concentration, photon flux, fluid velocity and both liquid and gas phase compositions. Deviations between simulated predictions and measurements for the amount of hydroperoxide formed are 7.1 % on average. Consequently, the identified and parameterized kinetic model is exploited to investigate different behaviors of the reactor under study. In a final step, the kinetic model is utilized to suggest attractive operating windows for future applications of the photooxygenation of DHAA exploiting reaction rates that are not affected by mass transfer limitations.

Optimally Designed Model Selection for Synthetic Biology

Modeling parts and circuits represents a significant roadblock to automating the Design-Build-Test-Learn cycle in synthetic biology. Once models are developed, discriminating among them requires informative data, computational resources, and skills that might not be readily available. The high cost entailed in model discrimination frequently leads to subjective choices on the selected structures and, in turn, to suboptimal models. Here, we outline frequentist and Bayesian approaches to model discrimination. We ranked three candidate models of a genetic toggle switch, which was adopted as a test case, according to the support from in vivo data. We show that, in each framework, efficient model discrimination can be achieved via optimally designed experiments. We offer a dynamical-systems interpretation of our optimization results and investigate their sensitivity to key parameters in the characterization of synthetic circuits. Our approach suggests that optimal experimental design is an effective strategy to discriminate between competing models of a gene regulatory network. Independent of the adopted framework, optimally designed perturbations exploit regions in the input space that maximally distinguish predictions from the competing models.

Multi-experiment nonlinear mixed effect modeling of single-cell translation kinetics after transfection

Single-cell time-lapse studies have advanced the quantitative understanding of cellular pathways and their inherent cell-to-cell variability. However, parameters retrieved from individual experiments are model dependent and their estimation is limited, if based on solely one kind of experiment. Hence, methods to integrate data collected under different conditions are expected to improve model validation and information content. Here we present a multi-experiment nonlinear mixed effect modeling approach for mechanistic pathway models, which allows the integration of multiple single-cell perturbation experiments. We apply this approach to the translation of green fluorescent protein after transfection using a massively parallel read-out of micropatterned single-cell arrays. We demonstrate that the integration of data from perturbation experiments allows the robust reconstruction of cell-to-cell variability, i.e., parameter densities, while each individual experiment provides insufficient information. Indeed, we show that the integration of the datasets on the population level also improves the estimates for individual cells by breaking symmetries, although each of them is only measured in one experiment. Moreover, we confirmed that the suggested approach is robust with respect to batch effects across experimental replicates and can provide mechanistic insights into the nature of batch effects. We anticipate that the proposed multi-experiment nonlinear mixed effect modeling approach will serve as a basis for the analysis of cellular heterogeneity in single-cell dynamics.

Efficient simulation of intrinsic, extrinsic and external noise in biochemical systems

Motivation

Biological cells operate in a noisy regime influenced by intrinsic, extrinsic and external noise, which leads to large differences of individual cell states. Stochastic effects must be taken into account to characterize biochemical kinetics accurately. Since the exact solution of the chemical master equation, which governs the underlying stochastic process, cannot be derived for most biochemical systems, approximate methods are used to obtain a solution.

Results

In this study, a method to efficiently simulate the various sources of noise simultaneously is proposed and benchmarked on several examples. The method relies on the combination of the sigma point approach to describe extrinsic and external variability and the τ-leaping algorithm to account for the stochasticity due to probabilistic reactions. The comparison of our method to extensive Monte Carlo calculations demonstrates an immense computational advantage while losing an acceptable amount of accuracy. Additionally, the application to parameter optimization problems in stochastic biochemical reaction networks is shown, which is rarely applied due to its huge computational burden. To give further insight, a MATLAB script is provided including the proposed method applied to a simple toy example of gene expression.

Availability and implementation

MATLAB code is available at Bioinformatics online.

Supplementary information

Supplementary data are available at Bioinformatics online.

Prediction uncertainty and optimal experimental design for learning dynamical systems

Dynamical systems are frequently used to model biological systems. When these models are fit to data, it is necessary to ascertain the uncertainty in the model fit. Here, we present prediction deviation, a metric of uncertainty that determines the extent to which observed data have constrained the model's predictions. This is accomplished by solving an optimization problem that searches for a pair of models that each provides a good fit for the observed data, yet has maximally different predictions. We develop a method for estimating a priori the impact that additional experiments would have on the prediction deviation, allowing the experimenter to design a set of experiments that would most reduce uncertainty. We use prediction deviation to assess uncertainty in a model of interferon-alpha inhibition of viral infection, and to select a sequence of experiments that reduces this uncertainty. Finally, we prove a theoretical result which shows that prediction deviation provides bounds on the trajectories of the underlying true model. These results show that prediction deviation is a meaningful metric of uncertainty that can be used for optimal experimental design.

Efficient Optimization of Stimuli for Model-Based Design of Experiments to Resolve Dynamical Uncertainty

This model-based design of experiments (MBDOE) method determines the input magnitudes of an experimental stimuli to apply and the associated measurements that should be taken to optimally constrain the uncertain dynamics of a biological system under study. The ideal global solution for this experiment design problem is generally computationally intractable because of parametric uncertainties in the mathematical model of the biological system. Others have addressed this issue by limiting the solution to a local estimate of the model parameters. Here we present an approach that is independent of the local parameter constraint. This approach is made computationally efficient and tractable by the use of: (1) sparse grid interpolation that approximates the biological system dynamics, (2) representative parameters that uniformly represent the data-consistent dynamical space, and (3) probability weights of the represented experimentally distinguishable dynamics. Our approach identifies data-consistent representative parameters using sparse grid interpolants, constructs the optimal input sequence from a greedy search, and defines the associated optimal measurements using a scenario tree. We explore the optimality of this MBDOE algorithm using a 3-dimensional Hes1 model and a 19-dimensional T-cell receptor model. The 19-dimensional T-cell model also demonstrates the MBDOE algorithm’s scalability to higher dimensions. In both cases, the dynamical uncertainty region that bounds the trajectories of the target system states were reduced by as much as 86% and 99% respectively after completing the designed experiments in silico. Our results suggest that for resolving dynamical uncertainty, the ability to design an input sequence paired with its associated measurements is particularly important when limited by the number of measurements.

Many mathematical models that have been developed for biological systems are limited because the complex systems are not well understood, the parameters are not known, and available data is limited and noisy. On the other hand, experiments to support model development are limited in terms of costs and time, feasible inputs and feasible measurements. MBDOE combines the mathematical models with experiment design to strategically design optimal experiments to obtain data that will contribute to the understanding of the systems. Our approach extends current capabilities of existing MBDOE techniques to make them more useful for scientists to resolve the trajectories of the system under study. It identifies the optimal conditions for stimuli and measurements that yield the most information about the system given the practical limitations. Exploration of the input space is not a trivial extension to MBDOE methods used for determining optimal measurements due to the nonlinear nature of many biological system models. The exploration of the system dynamics elicited by different inputs requires a computationally efficient and tractable approach. Our approach plans optimal experiments to reduce dynamical uncertainty in the output of selected target states of the biological system.

Rational selection of experimental readout and intervention sites for reducing uncertainties in computational model predictions

Background

Understanding the dynamics of biological processes can substantially be supported by computational models in the form of nonlinear ordinary differential equations (ODE). Typically, this model class contains many unknown parameters, which are estimated from inadequate and noisy data. Depending on the ODE structure, predictions based on unmeasured states and associated parameters are highly uncertain, even undetermined. For given data, profile likelihood analysis has been proven to be one of the most practically relevant approaches for analyzing the identifiability of an ODE structure, and thus model predictions. In case of highly uncertain or non-identifiable parameters, rational experimental design based on various approaches has shown to significantly reduce parameter uncertainties with minimal amount of effort.

Results

In this work we illustrate how to use profile likelihood samples for quantifying the individual contribution of parameter uncertainty to prediction uncertainty. For the uncertainty quantification we introduce the profile likelihood sensitivity (PLS) index. Additionally, for the case of several uncertain parameters, we introduce the PLS entropy to quantify individual contributions to the overall prediction uncertainty. We show how to use these two criteria as an experimental design objective for selecting new, informative readouts in combination with intervention site identification. The characteristics of the proposed multi-criterion objective are illustrated with an in silico example. We further illustrate how an existing practically non-identifiable model for the chlorophyll fluorescence induction in a photosynthetic organism, D. salina, can be rendered identifiable by additional experiments with new readouts.

Conclusions

Having data and profile likelihood samples at hand, the here proposed uncertainty quantification based on prediction samples from the profile likelihood provides a simple way for determining individual contributions of parameter uncertainties to uncertainties in model predictions. The uncertainty quantification of specific model predictions allows identifying regions, where model predictions have to be considered with care. Such uncertain regions can be used for a rational experimental design to render initially highly uncertain model predictions into certainty. Finally, our uncertainty quantification directly accounts for parameter interdependencies and parameter sensitivities of the specific prediction.

Electronic supplementary material

The online version of this article (doi:10.1186/s12859-014-0436-5) contains supplementary material, which is available to authorized users.

Model selection in systems biology depends on experimental design

Experimental design attempts to maximise the information available for modelling tasks. An optimal experiment allows the inferred models or parameters to be chosen with the highest expected degree of confidence. If the true system is faithfully reproduced by one of the models, the merit of this approach is clear - we simply wish to identify it and the true parameters with the most certainty. However, in the more realistic situation where all models are incorrect or incomplete, the interpretation of model selection outcomes and the role of experimental design needs to be examined more carefully. Using a novel experimental design and model selection framework for stochastic state-space models, we perform high-throughput in-silico analyses on families of gene regulatory cascade models, to show that the selected model can depend on the experiment performed. We observe that experimental design thus makes confidence a criterion for model choice, but that this does not necessarily correlate with a model's predictive power or correctness. Finally, in the special case of linear ordinary differential equation (ODE) models, we explore how wrong a model has to be before it influences the conclusions of a model selection analysis.

Different models of the same process represent distinct hypotheses about reality. These can be decided between within the framework of model selection, where the evidence for each is given by their ability to reproduce a set of experimental data. Even if one of the models is correct, the chances of identifying it can be hindered by the quality of the data, both in terms of its signal to measurement error ratio and the intrinsic discriminatory potential of the experiment undertaken. This potential can be predicted in various ways, and maximising it is one aim of experimental design. In this work we present a computationally efficient method of experimental design for model selection. We exploit the efficiency to consider the implications of the realistic case where all models are more or less incorrect, showing that experiments can be chosen that, considered individually, lead to unequivocal support for opposed hypotheses.

Experimental design, validation and computational modeling uncover DNA damage sensing by DNA-PK and ATM

Reliable and efficient detection of DNA damage constitutes a vital capability of human cells to maintain genome stability. Following DNA damage, the histone variant H2AX becomes rapidly phosphorylated by the DNA damage response kinases DNA-PKcs and ATM. H2AX phosphorylation plays a central role in signal amplification leading to chromatin remodeling and DNA repair initiation. The contribution of DNA-PKcs and ATM to H2AX phosphorylation is however puzzling. Although ATM is required, DNA-PKcs can substitute for it. Here we analyze the interplay between DNA-PKcs and ATM with a computational model derived by an iterative workflow: switching between experimental design, experiment and model analysis, we generated an extensive set of time-resolved data and identified a conclusive dynamic signaling model out of several alternatives. Our work shows that DNA-PKcs and ATM enforce a biphasic H2AX phosphorylation. DNA-PKcs can be associated to the initial, and ATM to the succeeding phosphorylation phase of H2AX resulting into a signal persistence detection function for reliable damage sensing. Further, our model predictions emphasize that DNA-PKcs inhibition significantly delays H2AX phosphorylation and associated DNA repair initiation.

Optimal experiment design for model selection in biochemical networks

Background

Mathematical modeling is often used to formalize hypotheses on how a biochemical network operates by discriminating between competing models. Bayesian model selection offers a way to determine the amount of evidence that data provides to support one model over the other while favoring simple models. In practice, the amount of experimental data is often insufficient to make a clear distinction between competing models. Often one would like to perform a new experiment which would discriminate between competing hypotheses.

Results

We developed a novel method to perform Optimal Experiment Design to predict which experiments would most effectively allow model selection. A Bayesian approach is applied to infer model parameter distributions. These distributions are sampled and used to simulate from multivariate predictive densities. The method is based on a k-Nearest Neighbor estimate of the Jensen Shannon divergence between the multivariate predictive densities of competing models.

Conclusions

We show that the method successfully uses predictive differences to enable model selection by applying it to several test cases. Because the design criterion is based on predictive distributions, which can be computed for a wide range of model quantities, the approach is very flexible. The method reveals specific combinations of experiments which improve discriminability even in cases where data is scarce. The proposed approach can be used in conjunction with existing Bayesian methodologies where (approximate) posteriors have been determined, making use of relations that exist within the inferred posteriors.

Convergence in parameters and predictions using computational experimental design

Typically, biological models fitted to experimental data suffer from significant parameter uncertainty, which can lead to inaccurate or uncertain predictions. One school of thought holds that accurate estimation of the true parameters of a biological system is inherently problematic. Recent work, however, suggests that optimal experimental design techniques can select sets of experiments whose members probe complementary aspects of a biochemical network that together can account for its full behaviour. Here, we implemented an experimental design approach for selecting sets of experiments that constrain parameter uncertainty. We demonstrated with a model of the epidermal growth factor–nerve growth factor pathway that, after synthetically performing a handful of optimal experiments, the uncertainty in all 48 parameters converged below 10 per cent. Furthermore, the fitted parameters converged to their true values with a small error consistent with the residual uncertainty. When untested experimental conditions were simulated with the fitted models, the predicted species concentrations converged to their true values with errors that were consistent with the residual uncertainty. This paper suggests that accurate parameter estimation is achievable with complementary experiments specifically designed for the task, and that the resulting parametrized models are capable of accurate predictions.

Reverse engineering and identification in systems biology: strategies, perspectives and challenges

The interplay of mathematical modelling with experiments is one of the central elements in systems biology. The aim of reverse engineering is to infer, analyse and understand, through this interplay, the functional and regulatory mechanisms of biological systems. Reverse engineering is not exclusive of systems biology and has been studied in different areas, such as inverse problem theory, machine learning, nonlinear physics, (bio)chemical kinetics, control theory and optimization, among others. However, it seems that many of these areas have been relatively closed to outsiders. In this contribution, we aim to compare and highlight the different perspectives and contributions from these fields, with emphasis on two key questions: (i) why are reverse engineering problems so hard to solve, and (ii) what methods are available for the particular problems arising from systems biology?

Robust optimal design of experiments for model discrimination using an interactive software tool

Mathematical modeling of biochemical processes significantly contributes to a better understanding of biological functionality and underlying dynamic mechanisms. To support time consuming and costly lab experiments, kinetic reaction equations can be formulated as a set of ordinary differential equations, which in turn allows to simulate and compare hypothetical models in silico. To identify new experimental designs that are able to discriminate between investigated models, the approach used in this work solves a semi-infinite constrained nonlinear optimization problem using derivative based numerical algorithms. The method takes into account parameter variabilities such that new experimental designs are robust against parameter changes while maintaining the optimal potential to discriminate between hypothetical models. In this contribution we present a newly developed software tool that offers a convenient graphical user interface for model discrimination. We demonstrate the beneficial operation of the discrimination approach and the usefulness of the software tool by analyzing a realistic benchmark experiment from literature. New robust optimal designs that allow to discriminate between the investigated model hypotheses of the benchmark experiment are successfully calculated and yield promising results. The involved robustification approach provides maximally discriminating experiments for the worst parameter configurations, which can be used to estimate the meaningfulness of upcoming experiments. A major benefit of the graphical user interface is the ability to interactively investigate the model behavior and the clear arrangement of numerous variables. In addition to a brief theoretical overview of the discrimination method and the functionality of the software tool, the importance of robustness of experimental designs against parameter variability is demonstrated on a biochemical benchmark problem. The software is licensed under the GNU General Public License and freely available at http://sourceforge.net/projects/mdtgui/.