Building Predictive Models of Genetic Circuits Using the Principle of Maximum Caliber

Probabilistic and maximum entropy modeling of chemical reaction systems: Characteristics and comparisons to mass action kinetic models

We demonstrate and characterize a first-principles approach to modeling the mass action dynamics of metabolism. Starting from a basic definition of entropy expressed as a multinomial probability density using Boltzmann probabilities with standard chemical potentials, we derive and compare the free energy dissipation and the entropy production rates. We express the relation between entropy production and the chemical master equation for modeling metabolism, which unifies chemical kinetics and chemical thermodynamics. Because prediction uncertainty with respect to parameter variability is frequently a concern with mass action models utilizing rate constants, we compare and contrast the maximum entropy model, which has its own set of rate parameters, to a population of standard mass action models in which the rate constants are randomly chosen. We show that a maximum entropy model is characterized by a high probability of free energy dissipation rate and likewise entropy production rate, relative to other models. We then characterize the variability of the maximum entropy model predictions with respect to uncertainties in parameters (standard free energies of formation) and with respect to ionic strengths typically found in a cell.

The calculus of codes - From entropy, complexity, and information to life

Exploring the core components that define living systems and their operational mechanisms within emerging biological entities is a complex endeavor. In the realm of biological systems literature, the terms matter, energy, information, complexity, and entropy are frequently referenced. However, possessing these concepts alone does not guarantee a comprehensive understanding or the ability to reconstruct the intricate nature of life. This study aims to illuminate the trajectory of these organic attributes, presenting a theoretical framework that delves into the integrated role of these concepts in biology. We assert that Code Biology serves as a pivotal steppingstone for unraveling the mechanisms underlying life. Biological codes (BCs) emerge not only from the interplay of matter and energy but also from Information. Contrary to deriving information from the former elements, we propose that information holds its place as a fundamental physical aspect. Consequently, we propose a continuum perspective called Calculus of Fundamentals involving three fundamentals: Matter, Energy, and Information, to depict the dynamics of BCs. To achieve this, we emphasize the necessity of studying Entropy and Complexity as integral organic descriptors. This perspective also facilitates the introduction of a mathematical theoretical framework that aids in comprehending continuous changes, the driving dynamics of biological fundamentals. We posit that Energy, Matter, and Information constitute the essential building blocks of living systems, and their interactions are governed by Entropy and Complexity analyses, redefined as biological descriptors. This interdisciplinary perspective of Code Biology sheds light on the intricate interplay between the controversial phenomenon of life and advances the idea of constructing a theory rooted in information as an organic fundamental.

Advances in automated and reactive flow cytometry for synthetic biotechnology

Automated flow cytometry (FC) has been initially considered for bioprocess monitoring and optimization. More recently, new physical and software interfaces have been made available, facilitating the access to this technology for labs and industries. It also comes with new capabilities, such as being able to act on the cultivation conditions based on population data. This approach, known as reactive FC, extended the range of applications of automated FC to bioprocess control and the stabilization of cocultures, but also to the broad field of synthetic and systems biology for the characterization of gene circuits. However, several issues must be addressed before automated and reactive FC can be considered standard and modular technologies.

MaxCal can infer models from coupled stochastic trajectories of gene expression and cell division

Gene expression is inherently noisy due to small numbers of proteins and nucleic acids inside a cell. Likewise, cell division is stochastic, particularly when tracking at the level of a single cell. The two can be coupled when gene expression affects the rate of cell division. Single-cell time-lapse experiments can measure both fluctuations by simultaneously recording protein levels inside a cell and its stochastic division. These information-rich noisy trajectory data sets can be harnessed to learn about the underlying molecular and cellular details that are often not known a priori. A critical question is: How can we infer a model given data where fluctuations at two levels-gene expression and cell division-are intricately convoluted? We show the principle of maximum caliber (MaxCal)-integrated within a Bayesian framework-can be used to infer several cellular and molecular details (division rates, protein production, and degradation rates) from these coupled stochastic trajectories (CSTs). We demonstrate this proof of concept using synthetic data generated from a known model. An additional challenge in data analysis is that trajectories are often not in protein numbers, but in noisy fluorescence that depends on protein number in a probabilistic manner. We again show that MaxCal can infer important molecular and cellular rates even when data are in fluorescence, another example of CST with three confounding factors-gene expression noise, cell division noise, and fluorescence distortion-all coupled. Our approach will provide guidance to build models in synthetic biology experiments as well as general biological systems where examples of CSTs are abundant.

On Conditional Tsallis Entropy

There is no generally accepted definition for conditional Tsallis entropy. The standard definition of (unconditional) Tsallis entropy depends on a parameter α that converges to the Shannon entropy as α approaches 1. In this paper, we describe three proposed definitions of conditional Tsallis entropy suggested in the literature—their properties are studied and their values, as a function of α, are compared. We also consider another natural proposal for conditional Tsallis entropy and compare it with the existing ones. Lastly, we present an online tool to compute the four conditional Tsallis entropies, given the probability distributions and the value of the parameter α.

Critical Comparison of MaxCal and Other Stochastic Modeling Approaches in Analysis of Gene Networks

Learning the underlying details of a gene network with feedback is critical in designing new synthetic circuits. Yet, quantitative characterization of these circuits remains limited. This is due to the fact that experiments can only measure partial information from which the details of the circuit must be inferred. One potentially useful avenue is to harness hidden information from single-cell stochastic gene expression time trajectories measured for long periods of time—recorded at frequent intervals—over multiple cells. This raises the feasibility vs. accuracy dilemma while deciding between different models of mining these stochastic trajectories. We demonstrate that inference based on the Maximum Caliber (MaxCal) principle is the method of choice by critically evaluating its computational efficiency and accuracy against two other typical modeling approaches: (i) a detailed model (DM) with explicit consideration of multiple molecules including protein-promoter interaction, and (ii) a coarse-grain model (CGM) using Hill type functions to model feedback. MaxCal provides a reasonably accurate model while being significantly more computationally efficient than DM and CGM. Furthermore, MaxCal requires minimal assumptions since it is a top-down approach and allows systematic model improvement by including constraints of higher order, in contrast to traditional bottom-up approaches that require more parameters or ad hoc assumptions. Thus, based on efficiency, accuracy, and ability to build minimal models, we propose MaxCal as a superior alternative to traditional approaches (DM, CGM) when inferring underlying details of gene circuits with feedback from limited data.

Use and Abuse of Entropy in Biology: A Case for Caliber

Here, I discuss entropy and its use as a tool in fields of biology such as bioenergetics, ecology, and evolutionary biology. Statistical entropy concepts including Shannon's diversity, configurational entropy, and informational entropy are discussed in connection to their use in describing the diversity, heterogeneity, and spatial patterning of biological systems. The use of entropy as a measure of biological complexity is also discussed, and I explore the extension of thermodynamic entropy principles to open, nonequilibrium systems operating in finite time. I conclude with suggestions for use of caliber, a metric similar to entropy but for time-dependent trajectories rather than static distributions, and propose the complementary notion of path information.

Inferring a network from dynamical signals at its nodes

We give an approximate solution to the difficult inverse problem of inferring the topology of an unknown network from given time-dependent signals at the nodes. For example, we measure signals from individual neurons in the brain, and infer how they are inter-connected. We use Maximum Caliber as an inference principle. The combinatorial challenge of high-dimensional data is handled using two different approximations to the pairwise couplings. We show two proofs of principle: in a nonlinear genetic toggle switch circuit, and in a toy neural network.

The Maximum Caliber Variational Principle for Nonequilibria

Ever since Clausius in 1865 and Boltzmann in 1877, the concepts of entropy and of its maximization have been the foundations for predicting how material equilibria derive from microscopic properties. But, despite much work, there has been no equally satisfactory general variational principle for nonequilibrium situations. However, in 1980, a new avenue was opened by E.T. Jaynes and by Shore and Johnson. We review here maximum caliber, which is a maximum-entropy-like principle that can infer distributions of flows over pathways, given dynamical constraints. This approach is providing new insights, particularly into few-particle complex systems, such as gene circuits, protein conformational reaction coordinates, network traffic, bird flocking, cell motility, and neuronal firing.

Quantitative Kinetic Models from Intravital Microscopy: A Case Study Using Hepatic Transport

The liver performs critical physiological functions, including metabolizing and removing substances, such as toxins and drugs, from the bloodstream. Hepatotoxicity itself is intimately linked to abnormal hepatic transport, and hepatotoxicity remains the primary reason drugs in development fail and approved drugs are withdrawn from the market. For this reason, we propose to analyze, across liver compartments, the transport kinetics of fluorescein-a fluorescent marker used as a proxy for drug molecules-using intravital microscopy data. To resolve the transport kinetics quantitatively from fluorescence data, we account for the effect that different liver compartments (with different chemical properties) have on fluorescein's emission rate. To do so, we develop ordinary differential equation transport models from the data where the kinetics is related to the observable fluorescence levels by "measurement parameters" that vary across different liver compartments. On account of the steep non-linearities in the kinetics and stochasticity inherent to the model, we infer kinetic and measurement parameters by generalizing the method of parameter cascades. For this application, the method of parameter cascades ensures fast and precise parameter estimates from noisy time traces.

Screening by changes in stereotypical behavior during cell motility

Stereotyped behaviors are series of postures that show very little variability between repeats. They have been used to classify the dynamics of individuals, groups and species without reference to the lower-level mechanisms that drive them. Stereotypes are easily identified in animals due to strong constraints on the number, shape, and relative positions of anatomical features, such as limbs, that may be used as landmarks for posture identification. In contrast, the identification of stereotypes in single cells poses a significant challenge as the cell lacks these landmark features, and finding constraints on cell shape is a non-trivial task. Here, we use the maximum caliber variational method to build a minimal model of cell behavior during migration. Without reference to biochemical details, we are able to make behavioral predictions over timescales of minutes using only changes in cell shape over timescales of seconds. We use drug treatment and genetics to demonstrate that maximum caliber descriptors can discriminate between healthy and aberrant migration, thereby showing potential applications for maximum caliber methods in automated disease screening, for example in the identification of behaviors associated with cancer metastasis.

Optimal parameter identification of synthetic gene networks using harmony search algorithm

Computational modeling of engineered gene circuits is an important while challenged task in systems biology. In order to describe and predict the response behaviors of genetic circuits using reliable model parameters, this paper applies an optimal experimental design(OED) method to obtain input signals. In order to obtain informative observations, this study focuses on maximizing Fisher information matrix(FIM)-based optimal criteria and to provide optimal inputs. Furthermore, this paper designs a two-stage optimization with the modified E-optimal criteria and applies harmony search(HS)-based OED algorithm to minimize estimation errors. The proposed optimal identification methodology involves estimation errors and the sample size to pursue a trade-off between estimation accuracy and measurement cost in modeling gene networks. The designed cost function takes two major factors into account, in which experimental costs are proportional to the number of time points. Experiments select two types of synthetic genetic networks to validate the effectiveness of the proposed HS-OED approach. Identification outcomes and analysis indicate the proposed HS-OED method outperforms two candidate OED approaches, with reduced computational effort.

Maximum Caliber Can Build and Infer Models of Oscillation in a Three-Gene Feedback Network

Single-cell protein expression time trajectories provide rich temporal data quantifying cellular variability and its role in dictating fitness. However, theoretical models to analyze and fully extract information from these measurements remain limited for three reasons: (i) gene expression profiles are noisy, rendering models of averages inapplicable, (ii) experiments typically measure only a few protein species while leaving other molecular actors-necessary to build traditional bottom-up models-unnoticed, and (iii) measured data are in fluorescence, not particle number. We recently addressed these challenges in an alternate top-down approach using the principle of Maximum Caliber (MaxCal) to model genetic switches with one and two protein species. In the present work we address scalability and broader applicability of MaxCal by extending to a three-gene (A, B, C) feedback network that exhibits oscillation, commonly known as the repressilator. We test MaxCal's inferential power by using synthetic data of noisy protein number time traces-serving as a proxy for experimental data-generated from a known underlying model. We notice that the minimal MaxCal model-accounting for production, degradation, and only one type of symmetric coupling between all three species-reasonably infers several underlying features of the circuit such as the effective production rate, degradation rate, frequency of oscillation, and protein number distribution. Next, we build models of higher complexity including different levels of coupling between A, B, and C and rigorously assess their relative performance. While the minimal model (with four parameters) performs remarkably well, we note that the most complex model (with six parameters) allowing all possible forms of crosstalk between A, B, and C slightly improves prediction of rates, but avoids ad hoc assumption of all the other models. It is also the model of choice based on Bayesian information criteria. We further analyzed time trajectories in arbitrary fluorescence (using synthetic trajectories) to mimic realistic data. We conclude that even with a three-protein system including both fluorescence noise and intrinsic gene expression fluctuations, MaxCal can faithfully infer underlying details of the network, opening future directions to model other network motifs with many species.

Perspective: Maximum caliber is a general variational principle for dynamical systems

We review here Maximum Caliber (Max Cal), a general variational principle for inferring distributions of paths in dynamical processes and networks. Max Cal is to dynamical trajectories what the principle of maximum entropy is to equilibrium states or stationary populations. In Max Cal, you maximize a path entropy over all possible pathways, subject to dynamical constraints, in order to predict relative path weights. Many well-known relationships of non-equilibrium statistical physics-such as the Green-Kubo fluctuation-dissipation relations, Onsager's reciprocal relations, and Prigogine's minimum entropy production-are limited to near-equilibrium processes. Max Cal is more general. While it can readily derive these results under those limits, Max Cal is also applicable far from equilibrium. We give examples of Max Cal as a method of inference about trajectory distributions from limited data, finding reaction coordinates in bio-molecular simulations, and modeling the complex dynamics of non-thermal systems such as gene regulatory networks or the collective firing of neurons. We also survey its basis in principle and some limitations.

Maximum Caliber Can Characterize Genetic Switches with Multiple Hidden Species

Gene networks with feedback often involve interactions between multiple species of biomolecules, much more than experiments can actually monitor. Coupled with this is the challenge that experiments often measure gene expression in noisy fluorescence instead of protein numbers. How do we infer biophysical information and characterize the underlying circuits from this limited and convoluted data? We address this by building stochastic models using the principle of Maximum Caliber (MaxCal). MaxCal uses the basic information on synthesis, degradation, and feedback-without invoking any other auxiliary species and ad hoc reactions-to generate stochastic trajectories similar to those typically measured in experiments. MaxCal in conjunction with Maximum Likelihood (ML) can infer parameters of the model using fluctuating trajectories of protein expression over time. We demonstrate the success of the MaxCal + ML methodology using synthetic data generated from known circuits of different genetic switches: (i) a single-gene autoactivating circuit involving five species (including mRNA), (ii) a mutually repressing two-gene circuit (toggle switch) with seven species (including mRNA) considering stochastic time traces of two proteins, and (iii) the same toggle switch circuit considering stochastic time traces of only one of the two proteins. To further challenge the MaxCal + ML inference scheme, we repeat our analysis for the second and third scenario with traces expressed in noisy fluorescence instead of protein number to closely mimic typical experiments. We show that, for all of these models with increasing complexity and obfuscation, the minimal model of MaxCal is still able to capture the fluctuations of the trajectory and infer basic underlying rate parameters when benchmarked against the known values used to generate the synthetic data. Importantly, the model also yields an effective feedback parameter that can be used to quantify interactions within these circuits. These applications show the promise of MaxCal's ability to characterize circuits with limited data, and its utility to better understand evolution and advance design strategies for specific functions.