Generalizing HMMs to Continuous Time for Fast Kinetics: Hidden Markov Jump Processes

Information-theoretical limit on the estimates of dissipation by molecular machines using single-molecule fluorescence resonance energy transfer experiments

Single-molecule fluorescence resonance energy transfer (FRET) experiments are commonly used to study the dynamics of molecular machines. While in vivo molecular processes often break time-reversal symmetry, the temporal directionality of cyclically operating molecular machines is often not evident from single-molecule FRET trajectories, especially in the most common two-color FRET studies. Solving a more quantitative problem of estimating the energy dissipation/entropy production by a molecular machine from single-molecule data is even more challenging. Here, we present a critical assessment of several practical methods of doing so, including Markov-model-based methods and a model-free approach based on an information-theoretical measure of entropy production that quantifies how (statistically) dissimilar observed photon sequences are from their time reverses. The Markov model approach is computationally feasible and may outperform model free approaches, but its performance strongly depends on how well the assumed model approximates the true microscopic dynamics. Markov models are also not guaranteed to give a lower bound on dissipation. Meanwhile, model-free, information-theoretical methods systematically underestimate entropy production at low photoemission rates, and long memory effects in the photon sequences make these methods demanding computationally. There is no clear winner among the approaches studied here, and all methods deserve to belong to a comprehensive data analysis toolkit.

Avoiding matrix exponentials for large transition rate matrices

Exact methods for the exponentiation of matrices of dimension N can be computationally expensive in terms of execution time (N3) and memory requirements (N2), not to mention numerical precision issues. A matrix often exponentiated in the natural sciences is the rate matrix. Here, we explore five methods to exponentiate rate matrices, some of which apply more broadly to other matrix types. Three of the methods leverage a mathematical analogy between computing matrix elements of a matrix exponential process and computing transition probabilities of a dynamical process (technically a Markov jump process, MJP, typically simulated using Gillespie). In doing so, we identify a novel MJP-based method relying on restricting the number of "trajectory" jumps that incurs improved computational scaling. We then discuss this method's downstream implications on mixing properties of Monte Carlo posterior samplers. We also benchmark two other methods of matrix exponentiation valid for any matrix (beyond rate matrices and, more generally, positive definite matrices) related to solving differential equations: Runge-Kutta integrators and Krylov subspace methods. Under conditions where both the largest matrix element and the number of non-vanishing elements scale linearly with N-reasonable conditions for rate matrices often exponentiated-computational time scaling with the most competitive methods (Krylov and one of the MJP-based methods) reduces to N2 with total memory requirements of N.

Increasing the accuracy of single-molecule data analysis using tMAVEN

Time-dependent single-molecule experiments contain rich kinetic information about the functional dynamics of biomolecules. A key step in extracting this information is the application of kinetic models, such as hidden Markov models (HMMs), which characterize the molecular mechanism governing the experimental system. Unfortunately, researchers rarely know the physicochemical details of this molecular mechanism a priori, which raises questions about how to select the most appropriate kinetic model for a given single-molecule data set and what consequences arise if the wrong model is chosen. To address these questions, we have developed and used time-series modeling, analysis, and visualization environment (tMAVEN), a comprehensive, open-source, and extensible software platform. tMAVEN can perform each step of the single-molecule analysis pipeline, from preprocessing to kinetic modeling to plotting, and has been designed to enable the analysis of a single-molecule data set with multiple types of kinetic models. Using tMAVEN, we have systematically investigated mismatches between kinetic models and molecular mechanisms by analyzing simulated examples of prototypical single-molecule data sets exhibiting common experimental complications, such as molecular heterogeneity, with a series of different types of HMMs. Our results show that no single kinetic modeling strategy is mathematically appropriate for all experimental contexts. Indeed, HMMs only correctly capture the underlying molecular mechanism in the simplest of cases. As such, researchers must modify HMMs using physicochemical principles to avoid the risk of missing the significant biological and biophysical insights into molecular heterogeneity that their experiments provide. By enabling the facile, side-by-side application of multiple types of kinetic models to individual single-molecule data sets, tMAVEN allows researchers to carefully tailor their modeling approach to match the complexity of the underlying biomolecular dynamics and increase the accuracy of their single-molecule data analyses.

Increasing the accuracy of single-molecule data analysis using tMAVEN

Time-dependent single-molecule experiments contain rich kinetic information about the functional dynamics of biomolecules. A key step in extracting this information is the application of kinetic models, such as hidden Markov models (HMMs), which characterize the molecular mechanism governing the experimental system. Unfortunately, researchers rarely know the physico-chemical details of this molecular mechanism a priori, which raises questions about how to select the most appropriate kinetic model for a given single-molecule dataset and what consequences arise if the wrong model is chosen. To address these questions, we have developed and used time-series Modeling, Analysis, and Visualization ENvironment (tMAVEN), a comprehensive, open-source, and extensible software platform. tMAVEN can perform each step of the single-molecule analysis pipeline, from pre-processing to kinetic modeling to plotting, and has been designed to enable the analysis of a single-molecule dataset with multiple types of kinetic models. Using tMAVEN, we have systematically investigated mismatches between kinetic models and molecular mechanisms by analyzing simulated examples of prototypical single-molecule datasets exhibiting common experimental complications, such as molecular heterogeneity, with a series of different types of HMMs. Our results show that no single kinetic modeling strategy is mathematically appropriate for all experimental contexts. Indeed, HMMs only correctly capture the underlying molecular mechanism in the simplest of cases. As such, researchers must modify HMMs using physico-chemical principles to avoid the risk of missing the significant biological and biophysical insights into molecular heterogeneity that their experiments provide. By enabling the facile, side-by-side application of multiple types of kinetic models to individual single-molecule datasets, tMAVEN allows researchers to carefully tailor their modeling approach to match the complexity of the underlying biomolecular dynamics and increase the accuracy of their single-molecule data analyses.

Fluorescence Microscopy: a statistics-optics perspective

Fundamental properties of light unavoidably impose features on images collected using fluorescence microscopes. Modeling these features is ever more important in quantitatively interpreting microscopy images collected at scales on par or smaller than light's wavelength. Here we review the optics responsible for generating fluorescent images, fluorophore properties, microscopy modalities leveraging properties of both light and fluorophores, in addition to the necessarily probabilistic modeling tools imposed by the stochastic nature of light and measurement.

Single-Molecule Reaction-Diffusion

We propose to capture reaction-diffusion on a molecule-by-molecule basis from the fastest acquirable timescale, namely individual photon arrivals. We illustrate our method on intrinsically disordered human proteins, the linker histone H1.0 as well as its chaperone prothymosin α , as these diffuse through an illuminated confocal spot and interact forming larger ternary complexes on millisecond timescales. Most importantly, single-molecule reaction-diffusion, smRD, reveals single molecule properties without trapping or otherwise confining molecules to surfaces. We achieve smRD within a Bayesian paradigm and term our method Bayes-smRD. Bayes-smRD is further free of the average, bulk, results inherent to the analysis of long photon arrival traces by fluorescence correlation spectroscopy. In learning from thousands of photon arrivals continuous spatial positions and discrete conformational and photophysical state changes, Bayes-smRD estimates kinetic parameters on a molecule-by-molecule basis with two to three orders of magnitude less data than tools such as fluorescence correlation spectroscopy thereby also dramatically reducing sample photodamage.

Fluorescence lifetime: Beating the IRF and interpulse window

Fluorescence lifetime imaging captures the spatial distribution of chemical species across cellular environments employing pulsed illumination confocal setups. However, quantitative interpretation of lifetime data continues to face critical challenges. For instance, fluorescent species with known in vitro excited-state lifetimes may split into multiple species with unique lifetimes when introduced into complex living environments. What is more, mixtures of species, which may be both endogenous and introduced into the sample, may exhibit 1) very similar lifetimes as well as 2) wide ranges of lifetimes including lifetimes shorter than the instrumental response function or whose duration may be long enough to be comparable to the interpulse window. By contrast, existing methods of analysis are optimized for well-separated and intermediate lifetimes. Here, we broaden the applicability of fluorescence lifetime analysis by simultaneously treating unknown mixtures of arbitrary lifetimes-outside the intermediate, Goldilocks, zone-for data drawn from a single confocal spot leveraging the tools of Bayesian nonparametrics (BNP). We benchmark our algorithm, termed BNP lifetime analysis, using a range of synthetic and experimental data. Moreover, we show that the BNP lifetime analysis method can distinguish and deduce lifetimes using photon counts as small as 500.

Gene expression model inference from snapshot RNA data using Bayesian non-parametrics

Gene expression models, which are key towards understanding cellular regulatory response, underlie observations of single-cell transcriptional dynamics. Although RNA expression data encode information on gene expression models, existing computational frameworks do not perform simultaneous Bayesian inference of gene expression models and parameters from such data. Rather, gene expression models-composed of gene states, their connectivities and associated parameters-are currently deduced by pre-specifying gene state numbers and connectivity before learning associated rate parameters. Here we propose a method to learn full distributions over gene states, state connectivities and associated rate parameters, simultaneously and self-consistently from single-molecule RNA counts. We propagate noise from fluctuating RNA counts over models by treating models themselves as random variables. We achieve this within a Bayesian non-parametric paradigm. We demonstrate our method on the Escherichia coli lacZ pathway and the Saccharomyces cerevisiae STL1 pathway, and verify its robustness on synthetic data.

Single-photon smFRET. III. Application to pulsed illumination

Förster resonance energy transfer (FRET) using pulsed illumination has been pivotal in leveraging lifetime information in FRET analysis. However, there remain major challenges in quantitative single-photon, single-molecule FRET (smFRET) data analysis under pulsed illumination including 1) simultaneously deducing kinetics and number of system states; 2) providing uncertainties over estimates, particularly uncertainty over the number of system states; and 3) taking into account detector noise sources such as cross talk and the instrument response function contributing to uncertainty; in addition to 4) other experimental noise sources such as background. Here, we implement the Bayesian nonparametric framework described in the first companion article that addresses all aforementioned issues in smFRET data analysis specialized for the case of pulsed illumination. Furthermore, we apply our method to both synthetic as well as experimental data acquired using Holliday junctions.

Single-photon smFRET. I: Theory and conceptual basis

We present a unified conceptual framework and the associated software package for single-molecule Förster resonance energy transfer (smFRET) analysis from single-photon arrivals leveraging Bayesian nonparametrics, BNP-FRET. This unified framework addresses the following key physical complexities of a single-photon smFRET experiment, including: 1) fluorophore photophysics; 2) continuous time kinetics of the labeled system with large timescale separations between photophysical phenomena such as excited photophysical state lifetimes and events such as transition between system states; 3) unavoidable detector artefacts; 4) background emissions; 5) unknown number of system states; and 6) both continuous and pulsed illumination. These physical features necessarily demand a novel framework that extends beyond existing tools. In particular, the theory naturally brings us to a hidden Markov model with a second-order structure and Bayesian nonparametrics on account of items 1, 2, and 5 on the list. In the second and third companion articles, we discuss the direct effects of these key complexities on the inference of parameters for continuous and pulsed illumination, respectively.

Single-photon smFRET: II. Application to continuous illumination

Here we adapt the Bayesian nonparametrics (BNP) framework presented in the first companion article to analyze kinetics from single-photon, single-molecule Förster resonance energy transfer (smFRET) traces generated under continuous illumination. Using our sampler, BNP-FRET, we learn the escape rates and the number of system states given a photon trace. We benchmark our method by analyzing a range of synthetic and experimental data. Particularly, we apply our method to simultaneously learn the number of system states and the corresponding kinetics for intrinsically disordered proteins using two-color FRET under varying chemical conditions. Moreover, using synthetic data, we show that our method can deduce the number of system states even when kinetics occur at timescales of interphoton intervals.

A blind benchmark of analysis tools to infer kinetic rate constants from single-molecule FRET trajectories

Single-molecule FRET (smFRET) is a versatile technique to study the dynamics and function of biomolecules since it makes nanoscale movements detectable as fluorescence signals. The powerful ability to infer quantitative kinetic information from smFRET data is, however, complicated by experimental limitations. Diverse analysis tools have been developed to overcome these hurdles but a systematic comparison is lacking. Here, we report the results of a blind benchmark study assessing eleven analysis tools used to infer kinetic rate constants from smFRET trajectories. We test them against simulated and experimental data containing the most prominent difficulties encountered in analyzing smFRET experiments: different noise levels, varied model complexity, non-equilibrium dynamics, and kinetic heterogeneity. Our results highlight the current strengths and limitations in inferring kinetic information from smFRET trajectories. In addition, we formulate concrete recommendations and identify key targets for future developments, aimed to advance our understanding of biomolecular dynamics through quantitative experiment-derived models.

A blind benchmark of analysis tools to infer kinetic rate constants from single-molecule FRET trajectories

Single-molecule FRET (smFRET) is a versatile technique to study the dynamics and function of biomolecules since it makes nanoscale movements detectable as fluorescence signals. The powerful ability to infer quantitative kinetic information from smFRET data is, however, complicated by experimental limitations. Diverse analysis tools have been developed to overcome these hurdles but a systematic comparison is lacking. Here, we report the results of a blind benchmark study assessing eleven analysis tools used to infer kinetic rate constants from smFRET trajectories. We test them against simulated and experimental data containing the most prominent difficulties encountered in analyzing smFRET experiments: different noise levels, varied model complexity, non-equilibrium dynamics, and kinetic heterogeneity. Our results highlight the current strengths and limitations in inferring kinetic information from smFRET trajectories. In addition, we formulate concrete recommendations and identify key targets for future developments, aimed to advance our understanding of biomolecular dynamics through quantitative experiment-derived models.

Modeling Non-additive Effects in Neighboring Chemically Identical Fluorophores

Quantitative fluorescence analysis is often used to derive chemical properties, including stoichiometries, of biomolecular complexes. One fundamental underlying assumption in the analysis of fluorescence data─whether it be the determination of protein complex stoichiometry by super-resolution, or step-counting by photobleaching, or the determination of RNA counts in diffraction-limited spots in RNA fluorescence in situ hybridization (RNA-FISH) experiments─is that fluorophores behave identically and do not interact. However, recent experiments on fluorophore-labeled DNA origami structures such as fluorocubes have shed light on the nature of the interactions between identical fluorophores as these are brought closer together, thereby raising questions on the validity of the modeling assumption that fluorophores do not interact. Here, we analyze photon arrival data under pulsed illumination from fluorocubes where distances between dyes range from 2 to 10 nm. We discuss the implications of non-additivity of brightness on quantitative fluorescence analysis.

On distributions of barrier crossing times as observed in single-molecule studies of biomolecules

Single-molecule experiments that monitor time evolution of molecular observables in real time have expanded beyond measuring transition rates toward measuring distributions of times of various molecular events. Of particular interest is the first-passage time for making a transition from one molecular configuration ( a ) to another ( b ) and conditional first-passage times such as the transition path time, which is the first-passage time from a to b conditional upon not leaving the transition region intervening between a and b . Another experimentally accessible (but not yet studied experimentally) observable is the conditional exit time, i.e., the time to leave the transition region through a specified boundary. The distributions of such times contain a wealth of mechanistic information about the transitions in question. Here, we use the first and the second (and, if desired, higher) moments of these distributions to characterize their relative width for the model in which the experimental observable undergoes Brownian motion in a potential of mean force. We show that although the distributions of transition path times are always narrower than exponential (in that the ratio of the standard deviation to the distribution's mean is always less than 1), distributions of first-passage times and of conditional exit times can be either narrow or broad, in some cases displaying long power-law tails. The conditional exit time studied here provides a generalization of the transition path time that also allows one to characterize the temporal scales of failed barrier crossing attempts.

Extraction of rapid kinetics from smFRET measurements using integrative detectors

Hidden Markov models (HMMs) are used to learn single-molecule kinetics across a range of experimental techniques. By their construction, HMMs assume that single-molecule events occur on slower timescales than those of data acquisition. To move beyond that HMM limitation and allow for single-molecule events to occur on any timescale, we must treat single-molecule events in continuous time as they occur in nature. We propose a method to learn kinetic rates from single-molecule Förster resonance energy transfer (smFRET) data collected by integrative detectors, even if those rates exceed data acquisition rates. To achieve that, we exploit our recently proposed "hidden Markov jump process" (HMJP), with which we learn transition kinetics from parallel measurements in donor and acceptor channels. HMJPs generalize the HMM paradigm in two critical ways: (1) they deal with physical smFRET systems as they switch between conformational states in continuous time, and (2) they estimate transition rates between conformational states directly without having recourse to transition probabilities or assuming slow dynamics. Our continuous-time treatment learns the transition kinetics and photon emission rates for dynamic regimes that are inaccessible to HMMs, which treat system kinetics in discrete time. We validate our framework's robustness on simulated data and demonstrate its performance on experimental data from FRET-labeled Holliday junctions.

Residence time analysis of RNA polymerase transcription dynamics: A Bayesian sticky HMM approach

The time spent by a single RNA polymerase (RNAP) at specific locations along the DNA, termed "residence time," reports on the initiation, elongation, and termination stages of transcription. At the single-molecule level, this information can be obtained from dual ultrastable optical trapping experiments, revealing a transcriptional elongation of RNAP interspersed with residence times of variable duration. Successfully discriminating between long and short residence times was used by previous approaches to learn about RNAP's transcription elongation dynamics. Here, we propose an approach based on the Bayesian sticky hidden Markov model that treats all residence times for an Escherichia coli RNAP on an equal footing without a priori discriminating between long and short residence times. Furthermore, our method has two additional advantages: we provide full distributions around key point statistics and directly treat the sequence dependence of RNAP's elongation rate. By applying our approach to experimental data, we find assigned relative probabilities on long versus short residence times, force-dependent average residence time transcription elongation dynamics, ∼10% drop in the average backtracking durations in the presence of GreB, and ∼20% drop in the average residence time as a function of applied force in the presence of RNaseA.