Neural network aided approximation and parameter inference of non-Markovian models of gene expression

A generative benchmark for evaluating the performance of fluorescent cell image segmentation

Fluorescent cell imaging technology is fundamental in life science research, offering a rich source of image data crucial for understanding cell spatial positioning, differentiation, and decision-making mechanisms. As the volume of this data expands, precise image analysis becomes increasingly critical. Cell segmentation, a key analysis step, significantly influences quantitative analysis outcomes. However, selecting the most effective segmentation method is challenging, hindered by existing evaluation methods' inaccuracies, lack of graded evaluation, and narrow assessment scope. Addressing this, we developed a novel framework with two modules: StyleGAN2-based contour generation and Pix2PixHD-based image rendering, producing diverse, graded-density cell images. Using this dataset, we evaluated three leading cell segmentation methods: DeepCell, CellProfiler, and CellPose. Our comprehensive comparison revealed CellProfiler's superior accuracy in segmenting cytoplasm and nuclei. Our framework diversifies cell image data generation and systematically addresses evaluation challenges in cell segmentation technologies, establishing a solid foundation for advancing research and applications in cell image analysis.

Efficient and scalable prediction of stochastic reaction-diffusion processes using graph neural networks

The dynamics of locally interacting particles that are distributed in space give rise to a multitude of complex behaviours. However the simulation of reaction-diffusion processes which model such systems is highly computationally expensive, the cost increasing rapidly with the size of space. Here, we devise a graph neural network based approach that uses cheap Monte Carlo simulations of reaction-diffusion processes in a small space to cast predictions of the dynamics of the same processes in a much larger and complex space, including spaces modelled by networks with heterogeneous topology. By applying the method to two biological examples, we show that it leads to accurate results in a small fraction of the computation time of standard stochastic simulation methods. The scalability and accuracy of the method suggest it is a promising approach for studying reaction-diffusion processes in complex spatial domains such as those modelling biochemical reactions, population evolution and epidemic spreading.

Advanced methods for gene network identification and noise decomposition from single-cell data

Central to analyzing noisy gene expression systems is solving the Chemical Master Equation (CME), which characterizes the probability evolution of the reacting species' copy numbers. Solving CMEs for high-dimensional systems suffers from the curse of dimensionality. Here, we propose a computational method for improved scalability through a divide-and-conquer strategy that optimally decomposes the whole system into a leader system and several conditionally independent follower subsystems. The CME is solved by combining Monte Carlo estimation for the leader system with stochastic filtering procedures for the follower subsystems. We demonstrate this method with high-dimensional numerical examples and apply it to identify a yeast transcription system at the single-cell resolution, leveraging mRNA time-course experimental data. The identification results enable an accurate examination of the heterogeneity in rate parameters among isogenic cells. To validate this result, we develop a noise decomposition technique exploiting time-course data but requiring no supplementary components, e.g., dual-reporters.

Inferring Stochastic Rates from Heterogeneous Snapshots of Particle Positions

Many imaging techniques for biological systems-like fixation of cells coupled with fluorescence microscopy-provide sharp spatial resolution in reporting locations of individuals at a single moment in time but also destroy the dynamics they intend to capture. These snapshot observations contain no information about individual trajectories, but still encode information about movement and demographic dynamics, especially when combined with a well-motivated biophysical model. The relationship between spatially evolving populations and single-moment representations of their collective locations is well-established with partial differential equations (PDEs) and their inverse problems. However, experimental data is commonly a set of locations whose number is insufficient to approximate a continuous-in-space PDE solution. Here, motivated by popular subcellular imaging data of gene expression, we embrace the stochastic nature of the data and investigate the mathematical foundations of parametrically inferring demographic rates from snapshots of particles undergoing birth, diffusion, and death in a nuclear or cellular domain. Toward inference, we rigorously derive a connection between individual particle paths and their presentation as a Poisson spatial process. Using this framework, we investigate the properties of the resulting inverse problem and study factors that affect quality of inference. One pervasive feature of this experimental regime is the presence of cell-to-cell heterogeneity. Rather than being a hindrance, we show that cell-to-cell geometric heterogeneity can increase the quality of inference on dynamics for certain parameter regimes. Altogether, the results serve as a basis for more detailed investigations of subcellular spatial patterns of RNA molecules and other stochastically evolving populations that can only be observed for single instants in their time evolution.

Solving stochastic gene-expression models using queueing theory: A tutorial review

Stochastic models of gene expression are typically formulated using the chemical master equation, which can be solved exactly or approximately using a repertoire of analytical methods. Here, we provide a tutorial review of an alternative approach based on queueing theory that has rarely been used in the literature of gene expression. We discuss the interpretation of six types of infinite-server queues from the angle of stochastic single-cell biology and provide analytical expressions for the stationary and nonstationary distributions and/or moments of mRNA/protein numbers and bounds on the Fano factor. This approach may enable the solution of complex models that have hitherto evaded analytical solution.

Partial Cross Mapping Based on Sparse Variable Selection for Direct Fault Root Cause Diagnosis for Industrial Processes

Root cause diagnosis of process industry is of significance to ensure safe production and improve production efficiency. Conventional contribution plot methods have challenges in root cause diagnosis due to the smearing effect. Other traditional root cause diagnosis methods, such as Granger causality (GC) and transfer entropy, have unsatisfactory performance in root cause diagnosis for complex industrial processes due to the existence of indirect causality. In this work, a regularization and partial cross mapping (PCM)-based root cause diagnosis framework is proposed for efficient direct causality inference and fault propagation path tracing. First, generalized Lasso-based variable selection is performed. The Hotelling T2 statistic is formulated and the Lasso-based fault reconstruction is applied to select candidate root cause variables. Second, the root cause is diagnosed through the PCM and the propagation path is drawn out according to the diagnosis result. The proposed framework is studied in four cases to verify its rationality and effectiveness, including a numerical example, the Tennessee Eastman benchmark process, the wastewater treatment process (WWTP), and the decarburization process of high-speed wire rod spring steel.

Distilling dynamical knowledge from stochastic reaction networks

Stochastic reaction networks are widely used in the modeling of stochastic systems across diverse domains such as biology, chemistry, physics, and ecology. However, the comprehension of the dynamic behaviors inherent in stochastic reaction networks is a formidable undertaking, primarily due to the exponential growth in the number of possible states or trajectories as the state space dimension increases. In this study, we introduce a knowledge distillation method based on reinforcement learning principles, aimed at compressing the dynamical knowledge encoded in stochastic reaction networks into a singular neural network construct. The trained neural network possesses the capability to accurately predict the state conditional joint probability distribution that corresponds to the given query contexts, when prompted with rate parameters, initial conditions, and time values. This obviates the need to track the dynamical process, enabling the direct estimation of normalized state and trajectory probabilities, without necessitating the integration over the complete state space. By applying our method to representative examples, we have observed a high degree of accuracy in both multimodal and high-dimensional systems. Additionally, the trained neural network can serve as a foundational model for developing efficient algorithms for parameter inference and trajectory ensemble generation. These results collectively underscore the efficacy of our approach as a universal means of distilling knowledge from stochastic reaction networks. Importantly, our methodology also spotlights the potential utility in harnessing a singular, pretrained, large-scale model to encapsulate the solution space underpinning a wide spectrum of stochastic dynamical systems.

Density physics-informed neural networks reveal sources of cell heterogeneity in signal transduction

The transduction time between signal initiation and final response provides valuable information on the underlying signaling pathway, including its speed and precision. Furthermore, multi-modality in a transduction-time distribution indicates that the response is regulated by multiple pathways with different transduction speeds. Here, we developed a method called density physics-informed neural networks (Density-PINNs) to infer the transduction-time distribution from measurable final stress response time traces. We applied Density-PINNs to single-cell gene expression data from sixteen promoters regulated by unknown pathways in response to antibiotic stresses. We found that promoters with slower signaling initiation and transduction exhibit larger cell-to-cell heterogeneity in response intensity. However, this heterogeneity was greatly reduced when the response was regulated by slow and fast pathways together. This suggests a strategy for identifying effective signaling pathways for consistent cellular responses to disease treatments. Density-PINNs can also be applied to understand other time delay systems, including infectious diseases.

Industrial Process Monitoring Based on Dynamic Overcomplete Broad Learning Network

Most industrial processes feature high nonlinearity, non-Gaussianity, and time correlation. Models based on overcomplete broad learning system (OBLS) have been successfully applied in the fault monitoring realm, which may relatively deal with the nonlinear and non-Gaussian characteristics. However, these models barely take time correlation into full consideration, hindering the further improvement of the monitoring accuracy of the network. Therefore, an effective dynamic overcomplete broad learning system (DOBLS) based on matrix extension is proposed, which extends the raw data in the batch process with the idea of "time lag" in this article. Subsequently, the OBLS monitoring network is employed to continue the analysis of the extended dynamic input data. Finally, a monitoring model is established to tackle the coexistence of nonlinearity, non-Gaussianity, and time correlation in process data. To illustrate the superiority and feasibility, the proposed model is conducted on the penicillin fermentation simulation platform, the experimental result of which illustrates that the model can extract the feature of process data more comprehensively and be self-updated more efficiently. With shorter training time and higher monitoring accuracy, the proposed model can witness an improvement of average monitoring accuracy by 3.69% and 1.26% in 26 process fault types compared to the state-of-the-art fault monitoring methods BLS and OBLS, respectively.

Exact solution of a three-stage model of stochastic gene expression including cell-cycle dynamics

The classical three-stage model of stochastic gene expression predicts the statistics of single cell mRNA and protein number fluctuations as a function of the rates of promoter switching, transcription, translation, degradation and dilution. While this model is easily simulated, its analytical solution remains an unsolved problem. Here we modify this model to explicitly include cell-cycle dynamics and then derive an exact solution for the time-dependent joint distribution of mRNA and protein numbers. We show large differences between this model and the classical model which captures cell-cycle effects implicitly via effective first-order dilution reactions. In particular we find that the Fano factor of protein numbers calculated from a population snapshot measurement are underestimated by the classical model whereas the correlation between mRNA and protein can be either over- or underestimated, depending on the timescales of mRNA degradation and promoter switching relative to the mean cell-cycle duration time.

Inferring stochastic rates from heterogeneous snapshots of particle positions

Many imaging techniques for biological systems - like fixation of cells coupled with fluorescence microscopy - provide sharp spatial resolution in reporting locations of individuals at a single moment in time but also destroy the dynamics they intend to capture. These snapshot observations contain no information about individual trajectories, but still encode information about movement and demographic dynamics, especially when combined with a well-motivated biophysical model. The relationship between spatially evolving populations and single-moment representations of their collective locations is well-established with partial differential equations (PDEs) and their inverse problems. However, experimental data is commonly a set of locations whose number is insufficient to approximate a continuous-in-space PDE solution. Here, motivated by popular subcellular imaging data of gene expression, we embrace the stochastic nature of the data and investigate the mathematical foundations of parametrically inferring demographic rates from snapshots of particles undergoing birth, diffusion, and death in a nuclear or cellular domain. Toward inference, we rigorously derive a connection between individual particle paths and their presentation as a Poisson spatial process. Using this framework, we investigate the properties of the resulting inverse problem and study factors that affect quality of inference. One pervasive feature of this experimental regime is the presence of cell-to-cell heterogeneity. Rather than being a hindrance, we show that cell-to-cell geometric heterogeneity can increase the quality of inference on dynamics for certain parameter regimes. Altogether, the results serve as a basis for more detailed investigations of subcellular spatial patterns of RNA molecules and other stochastically evolving populations that can only be observed for single instants in their time evolution.

Inferring delays in partially observed gene regulation processes

MOTIVATION

Cell function is regulated by gene regulatory networks (GRNs) defined by protein-mediated interaction between constituent genes. Despite advances in experimental techniques, we can still measure only a fraction of the processes that govern GRN dynamics. To infer the properties of GRNs using partial observation, unobserved sequential processes can be replaced with distributed time delays, yielding non-Markovian models. Inference methods based on the resulting model suffer from the curse of dimensionality.

RESULTS

We develop a simulation-based Bayesian MCMC method employing an approximate likelihood for the efficient and accurate inference of GRN parameters when only some of their products are observed. We illustrate our approach using a two-step activation model: an activation signal leads to the accumulation of an unobserved regulatory protein, which triggers the expression of observed fluorescent proteins. With prior information about observed fluorescent protein synthesis, our method successfully infers the dynamics of the unobserved regulatory protein. We can estimate the delay and kinetic parameters characterizing target regulation including transcription, translation, and target searching of an unobserved protein from experimental measurements of the products of its target gene. Our method is scalable and can be used to analyze non-Markovian models with hidden components.

AVAILABILITY AND IMPLEMENTATION

Our code is implemented in R and is freely available with a simple example data at https://github.com/Mathbiomed/SimMCMC.

Studying stochastic systems biology of the cell with single-cell genomics data

Recent experimental developments in genome-wide RNA quantification hold considerable promise for systems biology. However, rigorously probing the biology of living cells requires a unified mathematical framework that accounts for single-molecule biological stochasticity in the context of technical variation associated with genomics assays. We review models for a variety of RNA transcription processes, as well as the encapsulation and library construction steps of microfluidics-based single-cell RNA sequencing, and present a framework to integrate these phenomena by the manipulation of generating functions. Finally, we use simulated scenarios and biological data to illustrate the implications and applications of the approach.

Stochastic modeling of the mRNA life process: A generalized master equation

The mRNA life cycle is a complex biochemical process, involving transcription initiation, elongation, termination, splicing, and degradation. Each of these molecular events is multistep and can create a memory. The effect of this molecular memory on gene expression is not clear, although there are many related yet scattered experimental reports. To address this important issue, we develop a general theoretical framework formulated as a master equation in the sense of queue theory, which can reduce to multiple previously studied gene models in limiting cases. This framework allows us to interpret experimental observations, extract kinetic parameters from experimental data, and identify how the mRNA kinetics vary under regulatory influences. Notably, it allows us to evaluate the influences of elongation processes on mature RNA distribution; e.g., we find that the non-exponential elongation time can induce the bimodal mRNA expression and there is an optimal elongation noise intensity such that the mature RNA noise achieves the lowest level. In a word, our framework can not only provide insight into complex mRNA life processes but also bridge a dialogue between theoretical studies and experimental data.

Assessing Markovian and Delay Models for Single-Nucleus RNA Sequencing

The serial nature of reactions involved in the RNA life-cycle motivates the incorporation of delays in models of transcriptional dynamics. The models couple a transcriptional process to a fairly general set of delayed monomolecular reactions with no feedback. We provide numerical strategies for calculating the RNA copy number distributions induced by these models, and solve several systems with splicing, degradation, and catalysis. An analysis of single-cell and single-nucleus RNA sequencing data using these models reveals that the kinetics of nuclear export do not appear to require invocation of a non-Markovian waiting time.

Uncovering the effect of RNA polymerase steric interactions on gene expression noise: Analytical distributions of nascent and mature RNA numbers

The telegraph model is the standard model of stochastic gene expression, which can be solved exactly to obtain the distribution of mature RNA numbers per cell. A modification of this model also leads to an analytical distribution of nascent RNA numbers. These solutions are routinely used for the analysis of single-cell data, including the inference of transcriptional parameters. However, these models neglect important mechanistic features of transcription elongation, such as the stochastic movement of RNA polymerases and their steric (excluded-volume) interactions. Here we construct a model of gene expression describing promoter switching between inactive and active states, binding of RNA polymerases in the active state, their stochastic movement including steric interactions along the gene, and their unbinding leading to a mature transcript that subsequently decays. We derive the steady-state distributions of the nascent and mature RNA numbers in two important limiting cases: constitutive expression and slow promoter switching. We show that RNA fluctuations are suppressed by steric interactions between RNA polymerases, and that this suppression can in some instances even lead to sub-Poissonian fluctuations; these effects are most pronounced for nascent RNA and less prominent for mature RNA, since the latter is not a direct sensor of transcription. We find a relationship between the parameters of our microscopic mechanistic model and those of the standard models that ensures excellent consistency in their prediction of the first and second RNA number moments over vast regions of parameter space, encompassing slow, intermediate, and rapid promoter switching, provided the RNA number distributions are Poissonian or super-Poissonian. Furthermore, we identify the limitations of inference from mature RNA data, specifically showing that it cannot differentiate between highly distinct RNA polymerase traffic patterns on a gene.

Monte Carlo samplers for efficient network inference

Accessing information on an underlying network driving a biological process often involves interrupting the process and collecting snapshot data. When snapshot data are stochastic, the data's structure necessitates a probabilistic description to infer underlying reaction networks. As an example, we may imagine wanting to learn gene state networks from the type of data collected in single molecule RNA fluorescence in situ hybridization (RNA-FISH). In the networks we consider, nodes represent network states, and edges represent biochemical reaction rates linking states. Simultaneously estimating the number of nodes and constituent parameters from snapshot data remains a challenging task in part on account of data uncertainty and timescale separations between kinetic parameters mediating the network. While parametric Bayesian methods learn parameters given a network structure (with known node numbers) with rigorously propagated measurement uncertainty, learning the number of nodes and parameters with potentially large timescale separations remain open questions. Here, we propose a Bayesian nonparametric framework and describe a hybrid Bayesian Markov Chain Monte Carlo (MCMC) sampler directly addressing these challenges. In particular, in our hybrid method, Hamiltonian Monte Carlo (HMC) leverages local posterior geometries in inference to explore the parameter space; Adaptive Metropolis Hastings (AMH) learns correlations between plausible parameter sets to efficiently propose probable models; and Parallel Tempering takes into account multiple models simultaneously with tempered information content to augment sampling efficiency. We apply our method to synthetic data mimicking single molecule RNA-FISH, a popular snapshot method in probing transcriptional networks to illustrate the identified challenges inherent to learning dynamical models from these snapshots and how our method addresses them.

Studying stochastic systems biology of the cell with single-cell genomics data

Recent experimental developments in genome-wide RNA quantification hold considerable promise for systems biology. However, rigorously probing the biology of living cells requires a unified mathematical framework that accounts for single-molecule biological stochasticity in the context of technical variation associated with genomics assays. We review models for a variety of RNA transcription processes, as well as the encapsulation and library construction steps of microfluidics-based single-cell RNA sequencing, and present a framework to integrate these phenomena by the manipulation of generating functions. Finally, we use simulated scenarios and biological data to illustrate the implications and applications of the approach.

Biophysical modeling with variational autoencoders for bimodal, single-cell RNA sequencing data

We motivate and present biVI, which combines the variational autoencoder framework of scVI with biophysically motivated, bivariate models for nascent and mature RNA distributions. While previous approaches to integrate bimodal data via the variational autoencoder framework ignore the causal relationship between measurements, biVI models the biophysical processes that give rise to observations. We demonstrate through simulated benchmarking that biVI captures cell type structure in a low-dimensional space and accurately recapitulates parameter values and copy number distributions. On biological data, biVI provides a scalable route for identifying the biophysical mechanisms underlying gene expression. This analytical approach outlines a generalizable strategy for treating multimodal datasets generated by high-throughput, single-cell genomic assays.

Gene expression model inference from snapshot RNA data using Bayesian non-parametrics

Gene expression models, which are key towards understanding cellular regulatory response, underlie observations of single-cell transcriptional dynamics. Although RNA expression data encode information on gene expression models, existing computational frameworks do not perform simultaneous Bayesian inference of gene expression models and parameters from such data. Rather, gene expression models-composed of gene states, their connectivities and associated parameters-are currently deduced by pre-specifying gene state numbers and connectivity before learning associated rate parameters. Here we propose a method to learn full distributions over gene states, state connectivities and associated rate parameters, simultaneously and self-consistently from single-molecule RNA counts. We propagate noise from fluctuating RNA counts over models by treating models themselves as random variables. We achieve this within a Bayesian non-parametric paradigm. We demonstrate our method on the Escherichia coli lacZ pathway and the Saccharomyces cerevisiae STL1 pathway, and verify its robustness on synthetic data.

Dynamic Batch Process Monitoring Based on Time-Slice Latent Variable Correlation Analysis

Batch processes are generally characterized by complex dynamics and remarkable data collinearity, thereby rendering the monitoring of such processes necessary but challenging. This paper proposes a data-driven time-slice latent variable correlation analysis-based model predictive fault detection framework to ensure accurate fault detection in dynamic batch processes. The three-way batch process data are first unfolded into the two-way time slice. For each single time slice, process data are mapped to both major latent variables and residual subspaces to deal with the variable-wise data collinearity and extract dominant data information. A measurement status is then determined with a canonical correlation analysis of the major latent variables and correlated variables, using both the time and batch perspectives. Prediction-based residuals are generated, which provide the basis for identifying the property of faults detected, namely, static or dynamic. Based on experiments using a simulated penicillin production and an industrial inject molding process, the proposed monitoring scheme has been proven feasible and effective.

Approximating solutions of the Chemical Master equation using neural networks

The Chemical Master Equation (CME) provides an accurate description of stochastic biochemical reaction networks in well-mixed conditions, but it cannot be solved analytically for most systems of practical interest. Although Monte Carlo methods provide a principled means to probe system dynamics, the large number of simulations typically required can render the estimation of molecule number distributions and other quantities infeasible. In this article, we aim to leverage the representational power of neural networks to approximate the solutions of the CME and propose a framework for the Neural Estimation of Stochastic Simulations for Inference and Exploration (Nessie). Our approach is based on training neural networks to learn the distributions predicted by the CME from relatively few stochastic simulations. We show on biologically relevant examples that simple neural networks with one hidden layer can capture highly complex distributions across parameter space, thereby accelerating computationally intensive tasks such as parameter exploration and inference.

DelaySSAToolkit.jl: stochastic simulation of reaction systems with time delays in Julia

SUMMARY

DelaySSAToolkit.jl is a Julia package for modelling reaction systems with non-Markovian dynamics, specifically those with time delays. These delays implicitly capture multiple intermediate reaction steps and hence serve as an effective model reduction technique for complex systems in biology, chemistry, ecology and genetics. The package implements a variety of exact formulations of the delay stochastic simulation algorithm.

AVAILABILITY AND IMPLEMENTATION

The source code and documentation of DelaySSAToolkit.jl are available at https://github.com/palmtree2013/DelaySSAToolkit.jl.

Inference and uncertainty quantification of stochastic gene expression via synthetic models

Estimating uncertainty in model predictions is a central task in quantitative biology. Biological models at the single-cell level are intrinsically stochastic and nonlinear, creating formidable challenges for their statistical estimation which inevitably has to rely on approximations that trade accuracy for tractability. Despite intensive interest, a sweet spot in this trade-off has not been found yet. We propose a flexible procedure for uncertainty quantification in a wide class of reaction networks describing stochastic gene expression including those with feedback. The method is based on creating a tractable coarse-graining of the model that is learned from simulations, a synthetic model, to approximate the likelihood function. We demonstrate that synthetic models can substantially outperform state-of-the-art approaches on a number of non-trivial systems and datasets, yielding an accurate and computationally viable solution to uncertainty quantification in stochastic models of gene expression.

Reliable and efficient parameter estimation using approximate continuum limit descriptions of stochastic models

Stochastic individual-based mathematical models are attractive for modelling biological phenomena because they naturally capture the stochasticity and variability that is often evident in biological data. Such models also allow us to track the motion of individuals within the population of interest. Unfortunately, capturing this microscopic detail means that simulation and parameter inference can become computationally expensive. One approach for overcoming this computational limitation is to coarse-grain the stochastic model to provide an approximate continuum model that can be solved using far less computational effort. However, coarse-grained continuum models can be biased or inaccurate, particularly for certain parameter regimes. In this work, we combine stochastic and continuum mathematical models in the context of lattice-based models of two-dimensional cell biology experiments by demonstrating how to simulate two commonly used experiments: cell proliferation assays and barrier assays. Our approach involves building a simple statistical model of the discrepancy between the expensive stochastic model and the associated computationally inexpensive coarse-grained continuum model. We form this statistical model based on a limited number of expensive stochastic model evaluations at design points sampled from a user-chosen distribution, corresponding to a computer experiment design problem. With straightforward design point selection schemes, we show that using the statistical model of the discrepancy in tandem with the computationally inexpensive continuum model allows us to carry out prediction and inference while correcting for biases and inaccuracies due to the continuum approximation. We demonstrate this approach by simulating a proliferation assay, where the continuum limit model is the well-known logistic ordinary differential equation, as well as a barrier assay where the continuum limit model is closely related to the well-known Fisher-KPP partial differential equation. We construct an approximate likelihood function for parameter inference, both with and without discrepancy correction terms. Using maximum likelihood estimation, we provide point estimates of the unknown parameters, and use the profile likelihood to characterise the uncertainty in these estimates and form approximate confidence intervals. For the range of inference problems considered, working with the continuum limit model alone leads to biased parameter estimation and confidence intervals with poor coverage. In contrast, incorporating correction terms arising from the statistical model of the model discrepancy allows us to recover the parameters accurately with minimal computational overhead. The main tradeoff is that the associated confidence intervals are typically broader, reflecting the additional uncertainty introduced by the approximation process. All algorithms required to replicate the results in this work are written in the open source Julia language and are available at GitHub.

Learning of Iterative Learning Control for Flexible Manufacturing of Batch Processes

Flexible manufacturing as an essential component of smart manufacturing implements the customized production mode, thereby requesting fast controller adaptation for producing different goods but still with high precision. This problem becomes even more acute for batch processes. Here we present a solution called learning of iterative learning control (ILC) based on neural networks. It is able to recommend control parameters for ILC controllers accordingly, so as to yield fast tracking error convergence and smaller steady-state error for disparate set-point profiles, which is deemed an abstraction of different production needs. The method substantially outperforms a benchmark ILC on a variety of systems and cases, thereby showing its potential for deployment in the industrial Internet of Things.

Systematic inference identifies a major source of heterogeneity in cell signaling dynamics: The rate-limiting step number

Identifying the sources of cell-to-cell variability in signaling dynamics is essential to understand drug response variability and develop effective therapeutics. However, it is challenging because not all signaling intermediate reactions can be experimentally measured simultaneously. This can be overcome by replacing them with a single random time delay, but the resulting process is non-Markovian, making it difficult to infer cell-to-cell heterogeneity in reaction rates and time delays. To address this, we developed an efficient and scalable moment-based Bayesian inference method (MBI) with a user-friendly computational package that infers cell-to-cell heterogeneity in the non-Markovian signaling process. We applied MBI to single-cell expression profiles from promoters responding to antibiotics and discovered a major source of cell-to-cell variability in antibiotic stress response: the number of rate-limiting steps in signaling cascades. This knowledge can help identify effective therapies that destroy all pathogenic or cancer cells, and the approach can be applied to precision medicine.

Application of multivariate statistical analysis and network pharmacology to explore the mechanism of Danggui Liuhuang Tang in treating perimenopausal syndrome

ETHNOPHARMACOLOGICAL RELEVANCE

Danggui Liuhuang Tang (DGLHT), first recorded in "Lan-Shi-Mi-Cang" (written in 1276 AD), is a famous classical formula. In 2018, it was listed in the Catalogue of Ancient Classic and Famous Prescriptions (First Batch) formulated by the National Administration of Traditional Chinese Medicine and the National Medical Products Administration. Perimenopausal syndrome (PMS) refers to a series of syndromes with autonomic nervous system dysfunction and neuropsychological symptoms. The treatment of PMS demands non-hormonal drugs. Natural products are considered to be effective substitutes for the treatment of PMS. It is reported that DGLHT has not only good therapeutic effects but also higher safety and fewer side effects in the treatment of PMS. However, the mechanism of DGLHT in treating PMS is not clear.

AIM OF THE STUDY

To explore the chemical basis and the mechanism of DGLHT in treating PMS.

MATERIALS AND METHODS

Multivariate statistical analysis was used to analyze the difference of components in supernatant before and after compatibility of DGLHT based on LC-MS data. The qualitative analysis was performed on the precipitate formed in the decocting process using LC-MS while the quantitative analysis on the potential markers using LC-UV. Then, the potential markers were analyzed by network pharmacology. The regulatory effect of DGLHT on FSH, P and E₂ were carried out in PMS rats.

RESULTS

Five potential markers, epiberberine, coptisine, palmatine, berberine and baicalin, were screened from the analysis of compounds in the supernatant. Four complexes, composed of potential marker monomers, were identified in the sediment, including two that have not been reported. The key targets of potential markers include TNF, NOS3, EGFR, ESR1, PTGS2, AR, CDC42 and RPS6KB1. The top signaling pathways include the cGMP-PKG signaling pathway, PI3K-Akt signaling pathway and estrogen signaling pathway. DGLHT could call back the hormone levels of P and E₂ in PMS rats.

CONCLUSION

DGLHT active ingredients, epiberberine, coptisine, palmatine, berberine and baicalin contribute a lot to the therapeutic effect. And DGLHT takes effect by regulating hormones secreted by the ovary.

Using average transcription level to understand the regulation of stochastic gene activation

Gene activation is a random process, modelled as a framework of multiple rate-limiting steps listed sequentially, in parallel or in combination. Together with suitably assumed processes of gene inactivation, transcript birth and death, the step numbers and parameters in activation frameworks can be estimated by fitting single-cell transcription data. However, current algorithms require computing master equations that are tightly correlated with prior hypothetical frameworks of gene activation. We found that prior estimation of the framework can be facilitated by the traditional dynamical data of mRNA average level M(t), presenting discriminated dynamical features. Rigorous theory regarding M(t) profiles allows to confidently rule out the frameworks that fail to capture M(t) features and to test potential competent frameworks by fitting M(t) data. We implemented this procedure for a large number of mouse fibroblast genes under tumour necrosis factor induction and determined exactly the 'cross-talking n-state' framework; the cross-talk between the signalling and basal pathways is crucial to trigger the first peak of M(t), while the following damped gentle M(t) oscillation is regulated by the multi-step basal pathway. This framework can be used to fit sophisticated single-cell data and may facilitate a more accurate understanding of stochastic activation of mouse fibroblast genes.

A Novel Approach for Calculating Exact Forms of mRNA Distribution in Single-Cell Measurements

Gene transcription is a stochastic process manifested by fluctuations in mRNA copy numbers in individual isogenic cells. Together with mathematical models of stochastic transcription, the massive mRNA distribution data that can be used to quantify fluctuations in mRNA levels can be fitted by Pm(t), which is the probability of producing m mRNA molecules at time t in a single cell. Tremendous efforts have been made to derive analytical forms of Pm(t), which rely on solving infinite arrays of the master equations of models. However, current approaches focus on the steady-state (t→∞) or require several parameters to be zero or infinity. Here, we present an approach for calculating Pm(t) with time, where all parameters are positive and finite. Our approach was successfully implemented for the classical two-state model and the widely used three-state model and may be further developed for different models with constant kinetic rates of transcription. Furthermore, the direct computations of Pm(t) for the two-state model and three-state model showed that the different regulations of gene activation can generate discriminated dynamical bimodal features of mRNA distribution under the same kinetic rates and similar steady-state mRNA distribution.

Distinguishing between models of mammalian gene expression: telegraph-like models versus mechanistic models

Two-state models (telegraph-like models) have a successful history of predicting distributions of cellular and nascent mRNA numbers that can well fit experimental data. These models exclude key rate limiting steps, and hence it is unclear why they are able to accurately predict the number distributions. To answer this question, here we compare these models to a novel stochastic mechanistic model of transcription in mammalian cells that presents a unified description of transcriptional factor, polymerase and mature mRNA dynamics. We show that there is a large region of parameter space where the first, second and third moments of the distributions of the waiting times between two consecutively produced transcripts (nascent or mature) of two-state and mechanistic models exactly match. In this region: (i) one can uniquely express the two-state model parameters in terms of those of the mechanistic model, (ii) the models are practically indistinguishable by comparison of their transcript numbers distributions, and (iii) they are distinguishable from the shape of their waiting time distributions. Our results clarify the relationship between different gene expression models and identify a means to select between them from experimental data.

Intelligent system for human activity recognition in IoT environment

In recent years, the adoption of machine learning has grown steadily in different fields affecting the day-to-day decisions of individuals. This paper presents an intelligent system for recognizing human's daily activities in a complex IoT environment. An enhanced model of capsule neural network called 1D-HARCapsNe is proposed. This proposed model consists of convolution layer, primary capsule layer, activity capsules flat layer and output layer. It is validated using WISDM dataset collected via smart devices and normalized using the random-SMOTE algorithm to handle the imbalanced behavior of the dataset. The experimental results indicate the potential and strengths of the proposed 1D-HARCapsNet that achieved enhanced performance with an accuracy of 98.67%, precision of 98.66%, recall of 98.67%, and F1-measure of 0.987 which shows major performance enhancement compared to the Conventional CapsNet (accuracy 90.11%, precision 91.88%, recall 89.94%, and F1-measure 0.93).

Hierarchical Bayesian models of transcriptional and translational regulation processes with delays

MOTIVATION

Simultaneous recordings of gene network dynamics across large populations have revealed that cell characteristics vary considerably even in clonal lines. Inferring the variability of parameters that determine gene dynamics is key to understanding cellular behavior. However, this is complicated by the fact that the outcomes and effects of many reactions are not observable directly. Unobserved reactions can be replaced with time delays to reduce model dimensionality and simplify inference. However, the resulting models are non-Markovian, and require the development of new inference techniques.

RESULTS

We propose a non-Markovian, hierarchical Bayesian inference framework for quantifying the variability of cellular processes within and across cells in a population. We illustrate our approach using a delayed birth-death process. In general, a distributed delay model, rather than a popular fixed delay model, is needed for inference, even if only mean reaction delays are of interest. Using in silico and experimental data we show that the proposed hierarchical framework is robust and leads to improved estimates compared to its non-hierarchical counterpart. We apply our method to data obtained using time-lapse microscopy and infer the parameters that describe the dynamics of protein production at the single cell and population level. The mean delays in protein production are larger than previously reported, have a coefficient of variation of around 0.2 across the population, and are not strongly correlated with protein production or growth rates.

AVAILABILITY AND IMPLEMENTATION

Accompanying code in Python is available at https://github.com/mvcortez/Bayesian-Inference.

CONTACT

kresimir.josic@gmail.com or jaekkim@kaist.ac.kr or cbskust@korea.ac.kr.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Exact results for queuing models of stochastic transcription with memory and crosstalk

Gene transcription is a complex multistep biochemical process, which can create memory between individual reaction events. On the other hand, many inducible genes, when activated by external cues, are often coregulated by several competitive pathways with crosstalk. This raises an unexplored question: how do molecular memory and crosstalk together affect gene expressions? To address this question, we introduce a queuing model of stochastic transcription, where two crossing signaling pathways are used to direct gene activation in response to external signals and memory functions to model multistep reaction processes involved in transcription. We first establish, based on the total probability principle, the chemical master equation for this queuing model, and then we derive, based on the binomial moment approach, exact expressions for statistical quantities (including distributions) of mRNA, which provide insights into the roles of crosstalk and memory in controlling the mRNA level and noise. We find that molecular memory of gene activation decreases the mRNA level but increases the mRNA noise, and double activation pathways always reduce the mRNA noise in contrast to a single pathway. In addition, we find that molecular memory can make the mRNA bimodality disappear.