An information-theoretic framework for deciphering pleiotropic and noisy biochemical signaling

Repeated Decision Stumping Distils Simple Rules from Single-Cell Data

Single-cell data afford unprecedented insights into molecular processes. But the complexity and size of these data sets have proved challenging and given rise to a large armory of statistical and machine learning approaches. The majority of approaches focuses on either describing features of these data, or making predictions and classifying unlabeled samples. In this study, we introduce repeated decision stumping (ReDX) as a method to distill simple models from single-cell data. We develop decision trees of depth one-hence "stumps"-to identify in an inductive manner, gene products involved in driving cell fate transitions, and in applications to published data we are able to discover the key players involved in these processes in an unbiased manner without prior knowledge. Our algorithm is deliberately targeting the simplest possible candidate hypotheses that can be extracted from complex high-dimensional data. There are three reasons for this: (1) the predictions become straightforwardly testable hypotheses; (2) the identified candidates form the basis for further mechanistic model development, for example, for engineering and synthetic biology interventions; and (3) this approach complements existing descriptive modeling approaches and frameworks. The approach is computationally efficient, has remarkable predictive power, including in simulation studies where the ground truth is known, and yields robust and statistically stable predictors; the same set of candidates is generated by applying the algorithm to different subsamples of experimental data.

Quantifying stimulus-response specificity to probe the functional state of macrophages

Immune sentinel macrophages initiate responses to pathogens via hundreds of immune response genes. Each immune threat demands a tailored response, suggesting that the capacity for stimulus-specific gene expression is a key functional hallmark of healthy macrophages. To quantify this property, termed "stimulus-response specificity" (SRS), we developed a single-cell experimental workflow and analytical approaches based on information theory and machine learning. We found that the response specificity of macrophages is driven by combinations of specific immune genes that show low cell-to-cell heterogeneity and are targets of separate signaling pathways. The "response specificity profile," a systematic comparison of multiple stimulus-response distributions, was distinctly altered by polarizing cytokines, and it enabled an assessment of the functional state of macrophages. Indeed, the response specificity profile of peritoneal macrophages from old and obese mice showed characteristic differences, suggesting that SRS may be a basis for measuring the functional state of innate immune cells. A record of this paper's transparent peer review process is included in the supplemental information.

Quantifying information of intracellular signaling: progress with machine learning

Cells convey information about their extracellular environment to their core functional machineries. Studying the capacity of intracellular signaling pathways to transmit information addresses fundamental questions about living systems. Here, we review how information-theoretic approaches have been used to quantify information transmission by signaling pathways that are functionally pleiotropic and subject to molecular stochasticity. We describe how recent advances in machine learning have been leveraged to address the challenges of complex temporal trajectory datasets and how these have contributed to our understanding of how cells employ temporal coding to appropriately adapt to environmental perturbations.

Phenotypic variability, not noise, accounts for most of the cell-to-cell heterogeneity in IFN-γ and oncostatin M signaling responses

Cellular signaling responses show substantial cell-to-cell heterogeneity, which is often ascribed to the inherent randomness of biochemical reactions, termed molecular noise, wherein high noise implies low signaling fidelity. Alternatively, heterogeneity could arise from differences in molecular content between cells, termed molecular phenotypic variability, which does not necessarily imply imprecise signaling. The contribution of these two processes to signaling heterogeneity is unclear. Here, we fused fibroblasts to produce binuclear syncytia to distinguish noise from phenotypic variability in the analysis of cytokine signaling. We reasoned that the responses of the two nuclei within one syncytium could approximate the signaling outcomes of two cells with the same molecular content, thereby disclosing noise contribution, whereas comparison of different syncytia should reveal contribution of phenotypic variability. We found that ~90% of the variance in the primary response (which was the abundance of phosphorylated, nuclear STAT) to stimulation with the cytokines interferon-γ and oncostatin M resulted from differences in the molecular content of individual cells. Thus, our data reveal that cytokine signaling in the system used here operates in a reproducible, high-fidelity manner.

Multiplexed Single-Cell Plasmonic Immunoassay of Intracellular Signaling Proteins Enables Non-Destructive Monitoring of Cell Fate

It is of significant importance in cancer biology to identify signaling pathways that play key roles in cell fate determination. Dissecting cellular signaling pathways requires the measurement of a large number of signaling proteins. However, tools for simultaneously monitoring multiple signaling pathway components in single living cells remain limited at present. Herein, we describe an approach, termed multiplexed single-cell plasmonic immunosandwich assay (mxscPISA), for simultaneous detection of multiple signaling proteins in individual living cells. This approach enabled simultaneous non-destructive monitoring of multiple (up to five, currently the highest multiplexing capacity in living cells) cytoplasmic and nucleus signaling proteins in individual cells with ultrahigh detection sensitivity. As a proof of principle, the epidermal growth factor receptor (EGFR) pathway, which plays a central role in cell fate determination, was investigated using this approach in this study. We found that there were differential attenuation rate of pro-survival and accumulation rate of pro-death signaling protein of the EGFR pathway in response to EGFR inactivation. These findings implicate that, after EGFR inactivation, a transient imbalance between survival and apoptotic signaling outputs contributed to the final cell fate of death. The mxscPISA approach can be a promising tool to reveal a signaling dynamic pattern at the single-cell level and to identify key components of signaling pathways that contribute to the final cell fate using only a limited number of cells.

Pathway dynamics can delineate the sources of transcriptional noise in gene expression

Single-cell expression profiling opens up new vistas on cellular processes. Extensive cell-to-cell variability at the transcriptomic and proteomic level has been one of the stand-out observations. Because most experimental analyses are destructive we only have access to snapshot data of cellular states. This loss of temporal information presents significant challenges for inferring dynamics, as well as causes of cell-to-cell variability. In particular, we typically cannot separate dynamic variability from within cells ('intrinsic noise') from variability across the population ('extrinsic noise'). Here, we make this non-identifiability mathematically precise, allowing us to identify new experimental set-ups that can assist in resolving this non-identifiability. We show that multiple generic reporters from the same biochemical pathways (e.g. mRNA and protein) can infer magnitudes of intrinsic and extrinsic transcriptional noise, identifying sources of heterogeneity. Stochastic simulations support our theory, and demonstrate that 'pathway-reporters' compare favourably to the well-known, but often difficult to implement, dual-reporter method.

The tyrosine phosphatase SHP2 increases robustness and information transfer within IL-6-induced JAK/STAT signalling

Background

Cell-to-cell heterogeneity is an inherent feature of multicellular organisms and is central in all physiological and pathophysiological processes including cellular signal transduction. The cytokine IL-6 is an essential mediator of pro- and anti-inflammatory processes. Dysregulated IL-6-induced intracellular JAK/STAT signalling is associated with severe inflammatory and proliferative diseases. Under physiological conditions JAK/STAT signalling is rigorously controlled and timely orchestrated by regulatory mechanisms such as expression of the feedback-inhibitor SOCS3 and activation of the protein-tyrosine phosphatase SHP2 (PTPN11). Interestingly, the function of negative regulators seems not to be restricted to controlling the strength and timely orchestration of IL-6-induced STAT3 activation. Exemplarily, SOCS3 increases robustness of late IL-6-induced STAT3 activation against heterogenous STAT3 expression and reduces the amount of information transferred through JAK/STAT signalling.

Methods

Here we use multiplexed single-cell analyses and information theoretic approaches to clarify whether also SHP2 contributes to robustness of STAT3 activation and whether SHP2 affects the amount of information transferred through IL-6-induced JAK/STAT signalling.

Results

SHP2 increases robustness of both basal, cytokine-independent STAT3 activation and early IL-6-induced STAT3 activation against differential STAT3 expression. However, SHP2 does not affect robustness of late IL-6-induced STAT3 activation. In contrast to SOCS3, SHP2 increases the amount of information transferred through IL-6-induced JAK/STAT signalling, probably by reducing cytokine-independent STAT3 activation and thereby increasing sensitivity of the cells. These effects are independent of SHP2-dependent MAPK activation.

Conclusion

In summary, the results of this study extend our knowledge of the functions of SHP2 in IL-6-induced JAK/STAT signalling. SHP2 is not only a repressor of basal and cytokine-induced STAT3 activity, but also ensures robustness and transmission of information.

Plain English summary Cells within a multicellular organism communicate with each other to exchange information about the environment. Communication between cells is facilitated by soluble molecules that transmit information from one cell to the other. Cytokines such as interleukin-6 are important soluble mediators that are secreted when an organism is faced with infections or inflammation. Secreted cytokines bind to receptors within the membrane of their target cells. This binding induces activation of an intracellular cascade of reactions called signal transduction, which leads to cellular responses. An important example of intracellular signal transduction is JAK/STAT signalling. In healthy organisms signalling is controlled and timed by regulatory mechanisms, whose activation results in a controlled shutdown of signalling pathways. Interestingly, not all cells within an organism are identical. They differ in the amount of proteins involved in signal transduction, such as STAT3. These differences shape cellular communication and responses to intracellular signalling. Here, we show that an important negative regulatory protein called SHP2 (or PTPN11) is not only responsible for shutting down signalling, but also for steering signalling in heterogeneous cell populations. SHP2 increases robustness of STAT3 activation against variable STAT3 amounts in individual cells. Additionally, it increases the amount of information transferred through JAK/STAT signalling by increasing the dynamic range of pathway activation in heterogeneous cell populations. This is an amazing new function of negative regulatory proteins that contributes to communication in heterogeneous multicellular organisms in health and disease.

Pleiotropy enables specific and accurate signaling in the presence of ligand cross talk

Living cells sense their environment through the binding of extracellular molecular ligands to cell surface receptors. Puzzlingly, vast numbers of signaling pathways exhibit a high degree of cross talk between different signals whereby different ligands act through the same receptor or shared components downstream. It remains unclear how a cell can accurately process information from the environment in such cross-wired pathways. We show that a feature which commonly accompanies cross talk-signaling pleiotropy (the ability of a receptor to produce multiple outputs)-offers a solution to the cross-talk problem. In a minimal model we show that a single pleiotropic receptor can simultaneously identify and accurately sense the concentrations of arbitrary unknown ligands present individually or in a mixture. We calculate the fundamental limits of the signaling specificity and accuracy of such signaling schemes. The model serves as an elementary "building block" toward understanding more complex cross-wired receptor-ligand signaling networks.

RVAgene: Generative modeling of gene expression time series data

Motivation

Methods to model dynamic changes in gene expression at a genome-wide level are not currently sufficient for large (temporally rich or single-cell) datasets. Variational autoencoders offer means to characterize large datasets and have been used effectively to characterize features of single-cell datasets. Here, we extend these methods for use with gene expression time series data.

Results

We present RVAgene: a recurrent variational autoencoder to model gene expression dynamics. RVAgene learns to accurately and efficiently reconstruct temporal gene profiles. It also learns a low dimensional representation of the data via a recurrent encoder network that can be used for biological feature discovery, and from which we can generate new gene expression data by sampling the latent space. We test RVAgene on simulated and real biological datasets, including embryonic stem cell differentiation and kidney injury response dynamics. In all cases, RVAgene accurately reconstructed complex gene expression temporal profiles. Via cross validation, we show that a low-error latent space representation can be learnt using only a fraction of the data. Through clustering and gene ontology term enrichment analysis on the latent space, we demonstrate the potential of RVAgene for unsupervised discovery. In particular, RVAgene identifies new programs of shared gene regulation of Lox family genes in response to kidney injury.

Availability and implementation

All datasets analyzed in this manuscript are publicly available and have been published previously. RVAgene is available in Python, at GitHub: https://github.com/maclean-lab/RVAgene; Zenodo archive: http://doi.org/10.5281/zenodo.4271097.

Supplementary information

Supplementary data are available at Bioinformatics online.

Experimental Study of the Free Space Optics Communication System Operating in the 8–12 µm Spectral Range

(1) Background: Free space optics communication (FSO) has improved wireless communication and data transfer thanks to high bandwidth, low power consumption, energy efficiency, a high transfer capacity, and a wide applicability field. The FSO systems also have their limitations, including weather conditions and obstacles in the way of transmission. (2) Methods: This research assesses the atmospheric conditions’ influence on the intensity of received radiation, both experimentally and theoretically. The construction of a laboratory test stand of the FSO system, which is operating in the third-atmosphere transmission window (8–12 µm), is proposed. Next, considering different atmospheric conditions, the experimental validation was conducted, both in a laboratory and real conditions. (3) Results: The measurements were carried out for two optical links working with wavelengths of 1.5 µm and 10 µm. It was found that optical radiation with a wavelength of about 10 µm is characterized by better transmission properties in the case of limited visibility (e.g., light rain and fogs) than in the case of near-infrared waves. The same conclusion was found in analytical investigations. (4) Conclusions: The results obtained show that optical radiation with a wavelength of about 10 µm in limited visibility is characterized by better transmission properties than near-infrared waves. This demonstrates the validity of designing FSO links operating in the range 8–12 µm band, e.g., based on quantum cascade lasers and HgCdTe photodiodes.

Concepts and Applications of Information Theory to Immuno-Oncology

Recent successes of immune-modulating therapies for cancer have stimulated research on information flow within the immune system and, in turn, clinical applications of concepts from information theory. Through information theory, one can describe and formalize, in a mathematically rigorous fashion, the function of interconnected components of the immune system in health and disease. Specifically, using concepts including entropy, mutual information, and channel capacity, one can quantify the storage, transmission, encoding, and flow of information within and between cellular components of the immune system on multiple temporal and spatial scales. To understand, at the quantitative level, immune signaling function and dysfunction in cancer, we present a methodology-oriented review of information-theoretic treatment of biochemical signal transduction and transmission coupled with mathematical modeling.

Quantifying information accumulation encoded in the dynamics of biochemical signaling

Cellular responses to environmental changes are encoded in the complex temporal patterns of signaling proteins. However, quantifying the accumulation of information over time to direct cellular decision-making remains an unsolved challenge. This is, in part, due to the combinatorial explosion of possible configurations that need to be evaluated for information in time-course measurements. Here, we develop a quantitative framework, based on inferred trajectory probabilities, to calculate the mutual information encoded in signaling dynamics while accounting for cell-cell variability. We use it to understand NFκB transcriptional dynamics in response to different immune threats, and reveal that some threats are distinguished faster than others. Our analyses also suggest specific temporal phases during which information distinguishing threats becomes available to immune response genes; one specific phase could be mapped to the functionality of the IκBα negative feedback circuit. The framework is generally applicable to single-cell time series measurements, and enables understanding how temporal regulatory codes transmit information over time.

Sensitivity minimization, biological homeostasis and information theory

All organisms must be able to adapt to changes in the environment. To this end, they have developed sophisticated regulatory mechanisms to ensure homeostasis. Control engineers, who must design similar regulatory systems, have developed a number of general principles that govern feedback regulation. These lead to constraints which impose trade-offs that arise when developing controllers to minimize the effect of external disturbances on systems. Here, we review some of these trade-offs, particularly Bode's integral formula. We also highlight its connection to information theory, by showing that the constraints in sensitivity minimization can be cast as limitations on the information transmission through a system, and these have their root in causality. Finally, we look at how these constraints arise in two biological systems: glycolytic oscillations and the energy cost of perfect adaptation in a bacterial chemotactic pathway.

Signal Fluctuations and the Information Transmission Rates in Binary Communication Channels

In the nervous system, information is conveyed by sequence of action potentials, called spikes-trains. As MacKay and McCulloch suggested, spike-trains can be represented as bits sequences coming from Information Sources (IS). Previously, we studied relations between spikes’ Information Transmission Rates (ITR) and their correlations, and frequencies. Now, I concentrate on the problem of how spikes fluctuations affect ITR. The IS are typically modeled as stationary stochastic processes, which I consider here as two-state Markov processes. As a spike-trains’ fluctuation measure, I assume the standard deviation σ, which measures the average fluctuation of spikes around the average spike frequency. I found that the character of ITR and signal fluctuations relation strongly depends on the parameter s being a sum of transitions probabilities from a no spike state to spike state. The estimate of the Information Transmission Rate was found by expressions depending on the values of signal fluctuations and parameter s. It turned out that for smaller s<1, the quotient ITRσ has a maximum and can tend to zero depending on transition probabilities, while for 1<s, the ITRσ is separated from 0. Additionally, it was also shown that ITR quotient by variance behaves in a completely different way. Similar behavior was observed when classical Shannon entropy terms in the Markov entropy formula are replaced by their approximation with polynomials. My results suggest that in a noisier environment (1<s), to get appropriate reliability and efficiency of transmission, IS with higher tendency of transition from the no spike to spike state should be applied. Such selection of appropriate parameters plays an important role in designing learning mechanisms to obtain networks with higher performance.

An information theoretic approach to insulin sensing by human kidney podocytes

Podocytes are key components of the glomerular filtration barrier (GFB). They are insulin-responsive but can become insulin-resistant, causing features of the leading global cause of kidney failure, diabetic nephropathy. Insulin acts via insulin receptors to control activities fundamental to GFB integrity, but the amount of information transferred is unknown. Here we measure this in human podocytes, using information theory-derived statistics that take into account cell-cell variability. High content imaging was used to measure insulin effects on Akt, FOXO and ERK. Mutual Information (MI) and Channel Capacity (CC) were calculated as measures of information transfer. We find that insulin acts via noisy communication channels with more information flow to Akt than to ERK. Information flow estimates were increased by consideration of joint sensing (ERK and Akt) and response trajectory (live cell imaging of FOXO1-clover translocation). Nevertheless, MI values were always <1Bit as most information was lost through signaling. Constitutive PI3K activity is a predominant feature of the system that restricts the proportion of CC engaged by insulin. Negative feedback from Akt supressed this activity and thereby improved insulin sensing, whereas sensing was robust to manipulation of feedforward signaling by inhibiting PI3K, PTEN or PTP1B. The decisions made by individual podocytes dictate GFB integrity, so we suggest that understanding the information on which the decisions are based will improve understanding of diabetic kidney disease and its treatment.

Optimal Design of Single-Cell Experiments within Temporally Fluctuating Environments

Modern biological experiments are becoming increasingly complex, and designing these experiments to yield the greatest possible quantitative insight is an open challenge. Increasingly, computational models of complex stochastic biological systems are being used to understand and predict biological behaviors or to infer biological parameters. Such quantitative analyses can also help to improve experiment designs for particular goals, such as to learn more about specific model mechanisms or to reduce prediction errors in certain situations. A classic approach to experiment design is to use the Fisher information matrix (FIM), which quantifies the expected information a particular experiment will reveal about model parameters. The Finite State Projection based FIM (FSP-FIM) was recently developed to compute the FIM for discrete stochastic gene regulatory systems, whose complex response distributions do not satisfy standard assumptions of Gaussian variations. In this work, we develop the FSP-FIM analysis for a stochastic model of stress response genes in S. cerevisae under time-varying MAPK induction. We verify this FSP-FIM analysis and use it to optimize the number of cells that should be quantified at particular times to learn as much as possible about the model parameters. We then extend the FSP-FIM approach to explore how different measurement times or genetic modifications help to minimize uncertainty in the sensing of extracellular environments, and we experimentally validate the FSP-FIM to rank single-cell experiments for their abilities to minimize estimation uncertainty of NaCl concentrations during yeast osmotic shock. This work demonstrates the potential of quantitative models to not only make sense of modern biological data sets, but to close the loop between quantitative modeling and experimental data collection.

Information-theoretic analysis of multivariate single-cell signaling responses

Mathematical methods of information theory appear to provide a useful language to describe how stimuli are encoded in activities of signaling effectors. Exploring the information-theoretic perspective, however, remains conceptually, experimentally and computationally challenging. Specifically, existing computational tools enable efficient analysis of relatively simple systems, usually with one input and output only. Moreover, their robust and readily applicable implementations are missing. Here, we propose a novel algorithm, SLEMI—statistical learning based estimation of mutual information, to analyze signaling systems with high-dimensional outputs and a large number of input values. Our approach is efficient in terms of computational time as well as sample size needed for accurate estimation. Analysis of the NF-κB single—cell signaling responses to TNF-α reveals that NF-κB signaling dynamics improves discrimination of high concentrations of TNF-α with a relatively modest impact on discrimination of low concentrations. Provided R-package allows the approach to be used by computational biologists with only elementary knowledge of information theory.

In light of single-cell, live-imaging experiments understanding of how cells transmit information about identity and quantity of stimuli is incomplete. When exposed to the same stimulus individual cells exhibit substantial cell-to-cell heterogeneity. Besides, stimuli have been shown to regulate temporal profiles of signaling effectors. Therefore, it is, for instance, not entirely clear whether single-cell responses are binary or contain more information about the quantity of stimuli. The above questions resulted in a considerable interest to study cellular signaling within the framework of information theory. Unfortunately, the utilization of the information-theoretic perspective is handicapped in part by the lack of suitable methods that account for multivariate signaling data. Here, we propose a novel algorithm that breaks a considerable computational barrier by allowing the effective information-theoretic analysis of highly-dimensional single-cell measurements. Our approach is computationally efficient, robust and straightforward to use. Moreover, we provide a simple R-package implementation.

The Limited Information Capacity of Cross-Reactive Sensors Drives the Evolutionary Expansion of Signaling

Signaling systems expand by duplications of various components, be it receptors or downstream effectors. However, whether and how duplicated components contribute to higher signaling capacity is unclear, especially because in most cases, their specificities overlap. Using information theory, we found that augmentation of capacity by an increase in the copy number is strongly limited by logarithmic diminishing returns. Moreover, counter to conventional biochemical wisdom, refinements of the response mechanism, e.g., by cooperativity or allostery, do not increase the overall signaling capacity. However, signaling capacity nearly doubles when a promiscuous, non-cognate ligand becomes explicitly recognized via duplication and partial divergence of signaling components. Our findings suggest that expansion of signaling components via duplication and enlistment of promiscuously acting cues is virtually the only accessible evolutionary strategy to achieve overall high-signaling capacity despite overlapping specificities and molecular noise. This mode of expansion also explains the highly cross-wired architecture of signaling pathways.