Gauge fixing for sequence-function relationships

Quantitative models of sequence-function relationships are ubiquitous in computational biology, e.g., for modeling the DNA binding of transcription factors or the fitness landscapes of proteins. Interpreting these models, however, is complicated by the fact that the values of model parameters can often be changed without affecting model predictions. Before the values of model parameters can be meaningfully interpreted, one must remove these degrees of freedom (called "gauge freedoms" in physics) by imposing additional constraints (a process called "fixing the gauge"). However, strategies for fixing the gauge of sequence-function relationships have received little attention. Here we derive an analytically tractable family of gauges for a large class of sequence-function relationships. These gauges are derived in the context of models with all-order interactions, but an important subset of these gauges can be applied to diverse types of models, including additive models, pairwise-interaction models, and models with higher-order interactions. Many commonly used gauges are special cases of gauges within this family. We demonstrate the utility of this family of gauges by showing how different choices of gauge can be used both to explore complex activity landscapes and to reveal simplified models that are approximately correct within localized regions of sequence space. The results provide practical gauge-fixing strategies and demonstrate the utility of gauge-fixing for model exploration and interpretation.

Symmetry, gauge freedoms, and the interpretability of sequence-function relationships

Quantitative models that describe how biological sequences encode functional activities are ubiquitous in modern biology. One important aspect of these models is that they commonly exhibit gauge freedoms, i.e., directions in parameter space that do not affect model predictions. In physics, gauge freedoms arise when physical theories are formulated in ways that respect fundamental symmetries. However, the connections that gauge freedoms in models of sequence-function relationships have to the symmetries of sequence space have yet to be systematically studied. Here we study the gauge freedoms of models that respect a specific symmetry of sequence space: the group of position-specific character permutations. We find that gauge freedoms arise when model parameters transform under redundant irreducible matrix representations of this group. Based on this finding, we describe an "embedding distillation" procedure that enables analytic calculation of the number of independent gauge freedoms, as well as efficient computation of a sparse basis for the space of gauge freedoms. We also study how parameter transformation behavior affects parameter interpretability. We find that in many (and possibly all) nontrivial models, the ability to interpret individual model parameters as quantifying intrinsic allelic effects requires that gauge freedoms be present. This finding establishes an incompatibility between two distinct notions of parameter interpretability. Our work thus advances the understanding of symmetries, gauge freedoms, and parameter interpretability in sequence-function relationships.

Applications of information geometry to spiking neural network activity

The space of possible behaviors that complex biological systems may exhibit is unimaginably vast, and these systems often appear to be stochastic, whether due to variable noisy environmental inputs or intrinsically generated chaos. The brain is a prominent example of a biological system with complex behaviors. The number of possible patterns of spikes emitted by a local brain circuit is combinatorially large, although the brain may not make use of all of them. Understanding which of these possible patterns are actually used by the brain, and how those sets of patterns change as properties of neural circuitry change is a major goal in neuroscience. Recently, tools from information geometry have been used to study embeddings of probabilistic models onto a hierarchy of model manifolds that encode how model outputs change as a function of their parameters, giving a quantitative notion of "distances" between outputs. We apply this method to a network model of excitatory and inhibitory neural populations to understand how the competition between membrane and synaptic response timescales shapes the network's information geometry. The hyperbolic embedding allows us to identify the statistical parameters to which the model behavior is most sensitive, and demonstrate how the ranking of these coordinates changes with the balance of excitation and inhibition in the network.

Sloppy model analysis identifies bifurcation parameters without normal form analysis

Bifurcation phenomena are common in multidimensional multiparameter dynamical systems. Normal form theory suggests that bifurcations are driven by relatively few combinations of parameters. Models of complex systems, however, rarely appear in normal form, and bifurcations are controlled by nonlinear combinations of the bare parameters of differential equations. Discovering reparameterizations to transform complex equations into a normal form is often very difficult, and the reparameterization may not even exist in a closed form. Here we show that information geometry and sloppy model analysis using the Fisher information matrix can be used to identify the combination of parameters that control bifurcations. By considering observations on increasingly long timescales, we find those parameters that rapidly characterize the system's topological inhomogeneities, whether the system is in normal form or not. We anticipate that this novel analytical method, which we call time-widening information geometry (TWIG), will be useful in applied network analysis.

Canalisation and plasticity on the developmental manifold of Caenorhabditis elegans

How do the same mechanisms that faithfully regenerate complex developmental programmes in spite of environmental and genetic perturbations also allow responsiveness to environmental signals, adaptation and genetic evolution? Using the nematode Caenorhabditis elegans as a model, we explore the phenotypic space of growth and development in various genetic and environmental contexts. Our data are growth curves and developmental parameters obtained by automated microscopy. Using these, we show that among the traits that make up the developmental space, correlations within a particular context are predictive of correlations among different contexts. Furthermore, we find that the developmental variability of this animal can be captured on a relatively low-dimensional phenotypic manifold and that on this manifold, genetic and environmental contributions to plasticity can be deconvolved independently. Our perspective offers a new way of understanding the relationship between robustness and flexibility in complex systems, suggesting that projection and concentration of dimension can naturally align these forces as complementary rather than competing.

Classical Fisher information for differentiable dynamical systems

Fisher information is a lower bound on the uncertainty in the statistical estimation of classical and quantum mechanical parameters. While some deterministic dynamical systems are not subject to random fluctuations, they do still have a form of uncertainty. Infinitesimal perturbations to the initial conditions can grow exponentially in time, a signature of deterministic chaos. As a measure of this uncertainty, we introduce another classical information, specifically for the deterministic dynamics of isolated, closed, or open classical systems not subject to noise. This classical measure of information is defined with Lyapunov vectors in tangent space, making it less akin to the classical Fisher information and more akin to the quantum Fisher information defined with wavevectors in Hilbert space. Our analysis of the local state space structure and linear stability leads to upper and lower bounds on this information, giving it an interpretation as the net stretching action of the flow. Numerical calculations of this information for illustrative mechanical examples show that it depends directly on the phase space curvature and speed of the flow.

Is Target-Based Drug Discovery Efficient? Discovery and "Off-Target" Mechanisms of All Drugs

Target-based drug discovery is the dominant paradigm of drug discovery; however, a comprehensive evaluation of its real-world efficiency is lacking. Here, a manual systematic review of about 32000 articles and patents dating back to 150 years ago demonstrates its apparent inefficiency. Analyzing the origins of all approved drugs reveals that, despite several decades of dominance, only 9.4% of small-molecule drugs have been discovered through "target-based" assays. Moreover, the therapeutic effects of even this minimal share cannot be solely attributed and reduced to their purported targets, as they depend on numerous off-target mechanisms unconsciously incorporated by phenotypic observations. The data suggest that reductionist target-based drug discovery may be a cause of the productivity crisis in drug discovery. An evidence-based approach to enhance efficiency seems to be prioritizing, in selecting and optimizing molecules, higher-level phenotypic observations that are closer to the sought-after therapeutic effects using tools like artificial intelligence and machine learning.

Unveiling order from chaos by approximate 2-localization of random matrices

Quantum many-body systems are typically endowed with a tensor product structure. A structure they inherited from probability theory, where the probability of two independent events is the product of the probabilities. The tensor product structure of a Hamiltonian thus gives a natural decomposition of the system into independent smaller subsystems. It is interesting to understand whether a given Hamiltonian is compatible with some particular tensor product structure. In particular, we ask, is there a basis in which an arbitrary Hamiltonian has a 2-local form, i.e., it contains only pairwise interactions? Here we show, using analytical and numerical calculations, that a generic Hamiltonian (e.g., a large random matrix) can be approximately written as a linear combination of two-body interaction terms with high precision; that is, the Hamiltonian is 2-local in a carefully chosen basis. Moreover, we show that these Hamiltonians are not fine-tuned, meaning that the spectrum is robust against perturbations of the coupling constants. Finally, by analyzing the adjacency structure of the couplings [Formula: see text], we suggest a possible mechanism for the emergence of geometric locality from quantum chaos.

Predictive power of non-identifiable models

Resolving practical non-identifiability of computational models typically requires either additional data or non-algorithmic model reduction, which frequently results in models containing parameters lacking direct interpretation. Here, instead of reducing models, we explore an alternative, Bayesian approach, and quantify the predictive power of non-identifiable models. We considered an example biochemical signalling cascade model as well as its mechanical analogue. For these models, we demonstrated that by measuring a single variable in response to a properly chosen stimulation protocol, the dimensionality of the parameter space is reduced, which allows for predicting the measured variable's trajectory in response to different stimulation protocols even if all model parameters remain unidentified. Moreover, one can predict how such a trajectory will transform in the case of a multiplicative change of an arbitrary model parameter. Successive measurements of remaining variables further reduce the dimensionality of the parameter space and enable new predictions. We analysed potential pitfalls of the proposed approach that can arise when the investigated model is oversimplified, incorrect, or when the training protocol is inadequate. The main advantage of the suggested iterative approach is that the predictive power of the model can be assessed and practically utilised at each step.

Reliable and robust control of nucleus centering is contingent on nonequilibrium force patterns

Cell centers their division apparatus to ensure symmetric cell division, a challenging task when the governing dynamics is stochastic. Using fission yeast, we show that the patterning of nonequilibrium polymerization forces of microtubule (MT) bundles controls the precise localization of spindle pole body (SPB), and hence the division septum, at the onset of mitosis. We define two cellular objectives, reliability, the mean SPB position relative to the geometric center, and robustness, the variance of the SPB position, which are sensitive to genetic perturbations that change cell length, MT bundle number/orientation, and MT dynamics. We show that simultaneous control of reliability and robustness is required to minimize septum positioning error achieved by the wild type (WT). A stochastic model for the MT-based nucleus centering, with parameters measured directly or estimated using Bayesian inference, recapitulates the maximum fidelity of WT. Using this, we perform a sensitivity analysis of the parameters that control nuclear centering.

Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data

Inference from limited data requires a notion of measure on parameter space, which is most explicit in the Bayesian framework as a prior distribution. Jeffreys prior is the best-known uninformative choice, the invariant volume element from information geometry, but we demonstrate here that this leads to enormous bias in typical high-dimensional models. This is because models found in science typically have an effective dimensionality of accessible behaviors much smaller than the number of microscopic parameters. Any measure which treats all of these parameters equally is far from uniform when projected onto the sub-space of relevant parameters, due to variations in the local co-volume of irrelevant directions. We present results on a principled choice of measure which avoids this issue and leads to unbiased posteriors by focusing on relevant parameters. This optimal prior depends on the quantity of data to be gathered, and approaches Jeffreys prior in the asymptotic limit. However, for typical models, this limit cannot be justified without an impossibly large increase in the quantity of data, exponential in the number of microscopic parameters.

On the Role of Theory and Modeling in Neuroscience

In recent years, the field of neuroscience has gone through rapid experimental advances and a significant increase in the use of quantitative and computational methods. This growth has created a need for clearer analyses of the theory and modeling approaches used in the field. This issue is particularly complex in neuroscience because the field studies phenomena that cross a wide range of scales and often require consideration at varying degrees of abstraction, from precise biophysical interactions to the computations they implement. We argue that a pragmatic perspective of science, in which descriptive, mechanistic, and normative models and theories each play a distinct role in defining and bridging levels of abstraction, will facilitate neuroscientific practice. This analysis leads to methodological suggestions, including selecting a level of abstraction that is appropriate for a given problem, identifying transfer functions to connect models and data, and the use of models themselves as a form of experiment.

Geometric analysis enables biological insight from complex non-identifiable models using simple surrogates

An enduring challenge in computational biology is to balance data quality and quantity with model complexity. Tools such as identifiability analysis and information criterion have been developed to harmonise this juxtaposition, yet cannot always resolve the mismatch between available data and the granularity required in mathematical models to answer important biological questions. Often, it is only simple phenomenological models, such as the logistic and Gompertz growth models, that are identifiable from standard experimental measurements. To draw insights from complex, non-identifiable models that incorporate key biological mechanisms of interest, we study the geometry of a map in parameter space from the complex model to a simple, identifiable, surrogate model. By studying how non-identifiable parameters in the complex model quantitatively relate to identifiable parameters in surrogate, we introduce and exploit a layer of interpretation between the set of non-identifiable parameters and the goodness-of-fit metric or likelihood studied in typical identifiability analysis. We demonstrate our approach by analysing a hierarchy of mathematical models for multicellular tumour spheroid growth experiments. Typical data from tumour spheroid experiments are limited and noisy, and corresponding mathematical models are very often made arbitrarily complex. Our geometric approach is able to predict non-identifiabilities, classify non-identifiable parameter spaces into identifiable parameter combinations that relate to features in the data characterised by parameters in a surrogate model, and overall provide additional biological insight from complex non-identifiable models.

Sloppiness: Fundamental study, new formalism and its application in model assessment

Computational modelling of biological processes poses multiple challenges in each stage of the modelling exercise. Some significant challenges include identifiability, precisely estimating parameters from limited data, informative experiments and anisotropic sensitivity in the parameter space. One of these challenges' crucial but inconspicuous sources is the possible presence of large regions in the parameter space over which model predictions are nearly identical. This property, known as sloppiness, has been reasonably well-addressed in the past decade, studying its possible impacts and remedies. However, certain critical unanswered questions concerning sloppiness, particularly related to its quantification and practical implications in various stages of system identification, still prevail. In this work, we systematically examine sloppiness at a fundamental level and formalise two new theoretical definitions of sloppiness. Using the proposed definitions, we establish a mathematical relationship between the parameter estimates' precision and sloppiness in linear predictors. Further, we develop a novel computational method and a visual tool to assess the goodness of a model around a point in parameter space by identifying local structural identifiability and sloppiness and finding the most sensitive and least sensitive parameters for non-infinitesimal perturbations. We demonstrate the working of our method in benchmark systems biology models of various complexities. The pharmacokinetic HIV infection model analysis identified a new set of biologically relevant parameters that can be used to control the free virus in an active HIV infection.

Information geometry for multiparameter models: new perspectives on the origin of simplicity

Complex models in physics, biology, economics, and engineering are oftensloppy, meaning that the model parameters are not well determined by the model predictions for collective behavior. Many parameter combinations can vary over decades without significant changes in the predictions. This review uses information geometry to explore sloppiness and its deep relation to emergent theories. We introduce themodel manifoldof predictions, whose coordinates are the model parameters. Itshyperribbonstructure explains why only a few parameter combinations matter for the behavior. We review recent rigorous results that connect the hierarchy of hyperribbon widths to approximation theory, and to the smoothness of model predictions under changes of the control variables. We discuss recent geodesic methods to find simpler models on nearby boundaries of the model manifold-emergent theories with fewer parameters that explain the behavior equally well. We discuss a Bayesian prior which optimizes the mutual information between model parameters and experimental data, naturally favoring points on the emergent boundary theories and thus simpler models. We introduce a 'projected maximum likelihood' prior that efficiently approximates this optimal prior, and contrast both to the poor behavior of the traditional Jeffreys prior. We discuss the way the renormalization group coarse-graining in statistical mechanics introduces a flow of the model manifold, and connect stiff and sloppy directions along the model manifold with relevant and irrelevant eigendirections of the renormalization group. Finally, we discuss recently developed 'intensive' embedding methods, allowing one to visualize the predictions of arbitrary probabilistic models as low-dimensional projections of an isometric embedding, and illustrate our method by generating the model manifold of the Ising model.

Arnold tongue entrainment reveals dynamical principles of the embryonic segmentation clock

Living systems exhibit an unmatched complexity, due to countless, entangled interactions across scales. Here, we aim to understand a complex system, that is, segmentation timing in mouse embryos, without a reference to these detailed interactions. To this end, we develop a coarse-grained approach, in which theory guides the experimental identification of the segmentation clock entrainment responses. We demonstrate period- and phase-locking of the segmentation clock across a wide range of entrainment parameters, including higher-order coupling. These quantifications allow to derive the phase response curve (PRC) and Arnold tongues of the segmentation clock, revealing its essential dynamical properties. Our results indicate that the somite segmentation clock has characteristics reminiscent of a highly non-linear oscillator close to an infinite period bifurcation and suggests the presence of long-term feedbacks. Combined, this coarse-grained theoretical-experimental approach reveals how we can derive simple, essential features of a highly complex dynamical system, providing precise experimental control over the pace and rhythm of the somite segmentation clock.

How Modelers Model: the Overlooked Social and Human Dimensions in Model Intercomparison Studies

There is a growing realization that the complexity of model ensemble studies depends not only on the models used but also on the experience and approach used by modelers to calibrate and validate results, which remain a source of uncertainty. Here, we applied a multi-criteria decision-making method to investigate the rationale applied by modelers in a model ensemble study where 12 process-based different biogeochemical model types were compared across five successive calibration stages. The modelers shared a common level of agreement about the importance of the variables used to initialize their models for calibration. However, we found inconsistency among modelers when judging the importance of input variables across different calibration stages. The level of subjective weighting attributed by modelers to calibration data decreased sequentially as the extent and number of variables provided increased. In this context, the perceived importance attributed to variables such as the fertilization rate, irrigation regime, soil texture, pH, and initial levels of soil organic carbon and nitrogen stocks was statistically different when classified according to model types. The importance attributed to input variables such as experimental duration, gross primary production, and net ecosystem exchange varied significantly according to the length of the modeler's experience. We argue that the gradual access to input data across the five calibration stages negatively influenced the consistency of the interpretations made by the modelers, with cognitive bias in "trial-and-error" calibration routines. Our study highlights that overlooking human and social attributes is critical in the outcomes of modeling and model intercomparison studies. While complexity of the processes captured in the model algorithms and parameterization is important, we contend that (1) the modeler's assumptions on the extent to which parameters should be altered and (2) modeler perceptions of the importance of model parameters are just as critical in obtaining a quality model calibration as numerical or analytical details.

How critical is brain criticality?

Criticality is the singular state of complex systems poised at the brink of a phase transition between order and randomness. Such systems display remarkable information-processing capabilities, evoking the compelling hypothesis that the brain may itself be critical. This foundational idea is now drawing renewed interest thanks to high-density data and converging cross-disciplinary knowledge. Together, these lines of inquiry have shed light on the intimate link between criticality, computation, and cognition. Here, we review these emerging trends in criticality neuroscience, highlighting new data pertaining to the edge of chaos and near-criticality, and making a case for the distance to criticality as a useful metric for probing cognitive states and mental illness. This unfolding progress in the field contributes to establishing criticality theory as a powerful mechanistic framework for studying emergent function and its efficiency in both biological and artificial neural networks.

Emergence of dynamic properties in network hypermotifs

Networks are fundamental for our understanding of complex systems. The study of networks has uncovered common principles that underlie the behavior of vastly different fields of study, including physics, biology, sociology, and engineering. One of these common principles is the existence of network motifs-small recurrent patterns that can provide certain features that are important for the specific network. However, it remains unclear how network motifs are joined in real networks to make larger circuits and what properties emerge from interactions between network motifs. Here, we develop a framework to explore the mesoscale-level behavior of complex networks. Considering network motifs as hypernodes, we define the rules for their interaction at the network's next level of organization. We develop a method to infer the favorable arrangements of interactions between network motifs into hypermotifs from real evolved and designed network data. We mathematically explore the emergent properties of these higher-order circuits and their relations to the properties of the individual minimal circuit components they combine. We apply this framework to biological, neuronal, social, linguistic, and electronic networks and find that network motifs are not randomly distributed in real networks but are combined in a way that both maintains autonomy and generates emergent properties. This framework provides a basis for exploring the mesoscale structure and behavior of complex systems where it can be used to reveal intermediate patterns in complex networks and to identify specific nodes and links in the network that are the key drivers of the network's emergent properties.

The sloppy relationship between neural circuit structure and function

Investigating and describing the relationships between the structure of a circuit and its function has a long tradition in neuroscience. Since neural circuits acquire their structure through sophisticated developmental programmes, and memories and experiences are maintained through synaptic modification, it is to be expected that structure is closely linked to function. Recent findings challenge this hypothesis from three different angles: Function does not strongly constrain circuit parameters, many parameters in neural circuits are irrelevant and contribute little to function, and circuit parameters are unstable and subject to constant random drift. At the same time however, recent work also showed that dynamics in neural circuit activity that is related to function are robust over time and across individuals. Here this apparent contradiction is addressed by considering the properties of neural manifolds that restrict circuit activity to functionally relevant subspaces, and it will be suggested that degenerate, anisotropic and unstable parameter spaces are a closely related to the structure and implementation of functionally relevant neural manifolds. Abstract figure legend What are the relationships between noisy and highly variable microscopic neural circuit variables on the one hand and the generation of behaviour on the other? Here it is proposed that an intermediate level of description exists where this relationship can be understood in terms of low-dimensional dynamics. Recordings of neural activity during unconstrained behaviour and the development of new machine learning methods will help to uncover these links. This article is protected by copyright. All rights reserved.

Recovery of neural dynamics criticality in personalized whole-brain models of stroke

The critical brain hypothesis states that biological neuronal networks, because of their structural and functional architecture, work near phase transitions for optimal response to internal and external inputs. Criticality thus provides optimal function and behavioral capabilities. We test this hypothesis by examining the influence of brain injury (strokes) on the criticality of neural dynamics estimated at the level of single participants using directly measured individual structural connectomes and whole-brain models. Lesions engender a sub-critical state that recovers over time in parallel with behavior. The improvement of criticality is associated with the re-modeling of specific white-matter connections. We show that personalized whole-brain dynamical models poised at criticality track neural dynamics, alteration post-stroke, and behavior at the level of single participants.

Maximum entropy models provide functional connectivity estimates in neural networks

Tools to estimate brain connectivity offer the potential to enhance our understanding of brain functioning. The behavior of neuronal networks, including functional connectivity and induced connectivity changes by external stimuli, can be studied using models of cultured neurons. Cultured neurons tend to be active in groups, and pairs of neurons are said to be functionally connected when their firing patterns show significant synchronicity. Methods to infer functional connections are often based on pair-wise cross-correlation between activity patterns of (small groups of) neurons. However, these methods are not very sensitive to detect inhibitory connections, and they were not designed for use during stimulation. Maximum Entropy (MaxEnt) models may provide a conceptually different method to infer functional connectivity. They have the potential benefit to estimate functional connectivity during stimulation, and to infer excitatory as well as inhibitory connections. MaxEnt models do not involve pairwise comparison, but aim to capture probability distributions of sets of neurons that are synchronously active in discrete time bins. We used electrophysiological recordings from in vitro neuronal cultures on micro electrode arrays to investigate the ability of MaxEnt models to infer functional connectivity. Connectivity estimates provided by MaxEnt models correlated well with those obtained by conditional firing probabilities (CFP), an established cross-correlation based method. In addition, stimulus-induced connectivity changes were detected by MaxEnt models, and were of the same magnitude as those detected by CFP. Thus, MaxEnt models provide a potentially powerful new tool to study functional connectivity in neuronal networks.

Bayesian, frequentist, and information geometric approaches to parametric uncertainty quantification of classical empirical interatomic potentials

In this paper, we consider the problem of quantifying parametric uncertainty in classical empirical interatomic potentials (IPs) using both Bayesian (Markov Chain Monte Carlo) and frequentist (profile likelihood) methods. We interface these tools with the Open Knowledgebase of Interatomic Models and study three models based on the Lennard-Jones, Morse, and Stillinger–Weber potentials. We confirm that IPs are typically sloppy, i.e., insensitive to coordinated changes in some parameter combinations. Because the inverse problem in such models is ill-conditioned, parameters are unidentifiable. This presents challenges for traditional statistical methods, as we demonstrate and interpret within both Bayesian and frequentist frameworks. We use information geometry to illuminate the underlying cause of this phenomenon and show that IPs have global properties similar to those of sloppy models from fields, such as systems biology, power systems, and critical phenomena. IPs correspond to bounded manifolds with a hierarchy of widths, leading to low effective dimensionality in the model. We show how information geometry can motivate new, natural parameterizations that improve the stability and interpretation of uncertainty quantification analysis and further suggest simplified, less-sloppy models.

Geometric framework to predict structure from function in neural networks

Neural computation in biological and artificial networks relies on the nonlinear summation of many inputs. The structural connectivity matrix of synaptic weights between neurons is a critical determinant of overall network function, but quantitative links between neural network structure and function are complex and subtle. For example, many networks can give rise to similar functional responses, and the same network can function differently depending on context. Whether certain patterns of synaptic connectivity are required to generate specific network-level computations is largely unknown. Here we introduce a geometric framework for identifying synaptic connections required by steady-state responses in recurrent networks of threshold-linear neurons. Assuming that the number of specified response patterns does not exceed the number of input synapses, we analytically calculate the solution space of all feedforward and recurrent connectivity matrices that can generate the specified responses from the network inputs. A generalization accounting for noise further reveals that the solution space geometry can undergo topological transitions as the allowed error increases, which could provide insight into both neuroscience and machine learning. We ultimately use this geometric characterization to derive certainty conditions guaranteeing a nonzero synapse between neurons. Our theoretical framework could thus be applied to neural activity data to make rigorous anatomical predictions that follow generally from the model architecture.

Discovering sparse control strategies in neural activity

Biological circuits such as neural or gene regulation networks use internal states to map sensory input to an adaptive repertoire of behavior. Characterizing this mapping is a major challenge for systems biology. Though experiments that probe internal states are developing rapidly, organismal complexity presents a fundamental obstacle given the many possible ways internal states could map to behavior. Using C. elegans as an example, we propose a protocol for systematic perturbation of neural states that limits experimental complexity and could eventually help characterize collective aspects of the neural-behavioral map. We consider experimentally motivated small perturbations—ones that are most likely to preserve natural dynamics and are closer to internal control mechanisms—to neural states and their impact on collective neural activity. Then, we connect such perturbations to the local information geometry of collective statistics, which can be fully characterized using pairwise perturbations. Applying the protocol to a minimal model of C. elegans neural activity, we find that collective neural statistics are most sensitive to a few principal perturbative modes. Dominant eigenvalues decay initially as a power law, unveiling a hierarchy that arises from variation in individual neural activity and pairwise interactions. Highest-ranking modes tend to be dominated by a few, “pivotal” neurons that account for most of the system’s sensitivity, suggesting a sparse mechanism of collective control.

Gaussian Information Bottleneck and the Non-Perturbative Renormalization Group

The renormalization group (RG) is a class of theoretical techniques used to explain the collective physics of interacting, many-body systems. It has been suggested that the RG formalism may be useful in finding and interpreting emergent low-dimensional structure in complex systems outside of the traditional physics context, such as in biology or computer science. In such contexts, one common dimensionality-reduction framework already in use is information bottleneck (IB), in which the goal is to compress an "input" signal X while maximizing its mutual information with some stochastic "relevance" variable Y. IB has been applied in the vertebrate and invertebrate processing systems to characterize optimal encoding of the future motion of the external world. Other recent work has shown that the RG scheme for the dimer model could be "discovered" by a neural network attempting to solve an IB-like problem. This manuscript explores whether IB and any existing formulation of RG are formally equivalent. A class of soft-cutoff non-perturbative RG techniques are defined by families of non-deterministic coarsening maps, and hence can be formally mapped onto IB, and vice versa. For concreteness, this discussion is limited entirely to Gaussian statistics (GIB), for which IB has exact, closed-form solutions. Under this constraint, GIB has a semigroup structure, in which successive transformations remain IB-optimal. Further, the RG cutoff scheme associated with GIB can be identified. Our results suggest that IB can be used to impose a notion of "large scale" structure, such as biological function, on an RG procedure.

Glycan processing in the Golgi: optimal information coding and constraints on cisternal number and enzyme specificity

Many proteins that undergo sequential enzymatic modification in the Golgi cisternae are displayed at the plasma membrane as cell identity markers. The modified proteins, called glycans, represent a molecular code. The fidelity of this glycan code is measured by how accurately the glycan synthesis machinery realises the desired target glycan distribution for a particular cell type and niche. In this paper, we construct a simplified chemical synthesis model to quantitatively analyse the tradeoffs between the number of cisternae, and the number and specificity of enzymes, required to synthesize a prescribed target glycan distribution of a certain complexity to within a given fidelity. We find that to synthesize complex distributions, such as those observed in real cells, one needs to have multiple cisternae and precise enzyme partitioning in the Golgi. Additionally, for fixed number of enzymes and cisternae, there is an optimal level of specificity (promiscuity) of enzymes that achieves the target distribution with high fidelity. The geometry of the fidelity landscape in the multidimensional space of the number and specificity of enzymes, inter-cisternal transfer rates, and number of cisternae, provides a measure for robustness and identifies stiff and sloppy directions. Our results show how the complexity of the target glycan distribution and number of glycosylation enzymes places functional constraints on the Golgi cisternal number and enzyme specificity.

Quantifying the regulatory role of individual transcription factors in Escherichia coli

Gene regulation often results from the action of multiple transcription factors (TFs) acting at a promoter, obscuring the individual regulatory effect of each TF on RNA polymerase (RNAP). Here we measure the fundamental regulatory interactions of TFs in E. coli by designing synthetic target genes that isolate individual TFs' regulatory effects. Using a thermodynamic model, each TF's regulatory interactions are decoupled from TF occupancy and interpreted as acting through (de)stabilization of RNAP and (de)acceleration of transcription initiation. We find that the contribution of each mechanism depends on TF identity and binding location; regulation immediately downstream of the promoter is insensitive to TF identity, but the same TFs regulate by distinct mechanisms upstream of the promoter. These two mechanisms are uncoupled and can act coherently, to reinforce the observed regulatory role (activation/repression), or incoherently, wherein the TF regulates two distinct steps with opposing effects.

What the foundations of quantum computer science teach us about chemistry

With the rapid development of quantum technology, one of the leading applications that has been identified is the simulation of chemistry. Interestingly, even before full scale quantum computers are available, quantum computer science has exhibited a remarkable string of results that directly impact what is possible in a chemical simulation with any computer. Some of these results even impact our understanding of chemistry in the real world. In this Perspective, we take the position that direct chemical simulation is best understood as a digital experiment. While on the one hand, this clarifies the power of quantum computers to extend our reach, it also shows us the limitations of taking such an approach too directly. Leveraging results that quantum computers cannot outpace the physical world, we build to the controversial stance that some chemical problems are best viewed as problems for which no algorithm can deliver their solution, in general, known in computer science as undecidable problems. This has implications for the predictive power of thermodynamic models and topics such as the ergodic hypothesis. However, we argue that this Perspective is not defeatist but rather helps shed light on the success of existing chemical models such as transition state theory, molecular orbital theory, and thermodynamics as models that benefit from data. We contextualize recent results, showing that data-augmented models are a more powerful rote simulation. These results help us appreciate the success of traditional chemical theory and anticipate new models learned from experimental data. Not only can quantum computers provide data for such models, but they can also extend the class and power of models that utilize data in fundamental ways. These discussions culminate in speculation on new ways for quantum computing and chemistry to interact and our perspective on the eventual roles of quantum computers in the future of chemistry.

Neural circuit function redundancy in brain disorders

Redundancy is a ubiquitous property of the nervous system. This means that vastly different configurations of cellular and synaptic components can enable the same neural circuit functions. However, until recently, very little brain disorder research has considered the implications of this characteristic when designing experiments or interpreting data. Here, we first summarise the evidence for redundancy in healthy brains, explaining redundancy and three related sub-concepts: sloppiness, dependencies and multiple solutions. We then lay out key implications for brain disorder research, covering recent examples of redundancy effects in experimental studies on psychiatric disorders. Finally, we give predictions for future experiments based on these concepts.

Macroscopic Quantities of Collective Brain Activity during Wakefulness and Anesthesia

The study of states of arousal is key to understand the principles of consciousness. Yet, how different brain states emerge from the collective activity of brain regions remains unknown. Here, we studied the fMRI brain activity of monkeys during wakefulness and anesthesia-induced loss of consciousness. We showed that the coupling between each brain region and the rest of the cortex provides an efficient statistic to classify the two brain states. Based on this and other statistics, we estimated maximum entropy models to derive collective, macroscopic properties that quantify the system's capabilities to produce work, to contain information, and to transmit it, which were all maximized in the awake state. The differences in these properties were consistent with a phase transition from critical dynamics in the awake state to supercritical dynamics in the anesthetized state. Moreover, information-theoretic measures identified those parameters that impacted the most the network dynamics. We found that changes in the state of consciousness primarily depended on changes in network couplings of insular, cingulate, and parietal cortices. Our findings suggest that the brain state transition underlying the loss of consciousness is predominantly driven by the uncoupling of specific brain regions from the rest of the network.

Relevance in the Renormalization Group and in Information Theory

The analysis of complex physical systems hinges on the ability to extract the relevant degrees of freedom from among the many others. Though much hope is placed in machine learning, it also brings challenges, chief of which is interpretability. It is often unclear what relation, if any, the architecture- and training-dependent learned "relevant" features bear to standard objects of physical theory. Here we report on theoretical results which may help to systematically address this issue: we establish equivalence between the field-theoretic relevance of the renormalization group, and an information-theoretic notion of relevance we define using the information bottleneck (IB) formalism of compression theory. We show analytically that for statistical physical systems described by a field theory the relevant degrees of freedom found using IB compression indeed correspond to operators with the lowest scaling dimensions. We confirm our field theoretic predictions numerically. We study dependence of the IB solutions on the physical symmetries of the data. Our findings provide a dictionary connecting two distinct theoretical toolboxes, and an example of constructively incorporating physical interpretability in applications of deep learning in physics.

Thermodynamic Efficiency of Interactions in Self-Organizing Systems

The emergence of global order in complex systems with locally interacting components is most striking at criticality, where small changes in control parameters result in a sudden global reorganization. We study the thermodynamic efficiency of interactions in self-organizing systems, which quantifies the change in the system's order per unit of work carried out on (or extracted from) the system. We analytically derive the thermodynamic efficiency of interactions for the case of quasi-static variations of control parameters in the exactly solvable Curie-Weiss (fully connected) Ising model, and demonstrate that this quantity diverges at the critical point of a second-order phase transition. This divergence is shown for quasi-static perturbations in both control parameters-the external field and the coupling strength. Our analysis formalizes an intuitive understanding of thermodynamic efficiency across diverse self-organizing dynamics in physical, biological, and social domains.

Parameter estimation in fluorescence recovery after photobleaching: quantitative analysis of protein binding reactions and diffusion

Fluorescence recovery after photobleaching (FRAP) is a common experimental method for investigating rates of molecular redistribution in biological systems. Many mathematical models of FRAP have been developed, the purpose of which is usually the estimation of certain biological parameters such as the diffusivity and chemical reaction rates of a protein, this being accomplished by fitting the model to experimental data. In this article, we consider a two species reaction–diffusion FRAP model. Using asymptotic analysis, we derive new FRAP recovery curve approximation formulae, and formally re-derive existing ones. On the basis of these formulae, invoking the concept of Fisher information, we predict, in terms of biological and experimental parameters, sufficient conditions to ensure that the values all model parameters can be estimated from data. We verify our predictions with extensive computational simulations. We also use computational methods to investigate cases in which some or all biological parameters are theoretically inestimable. In these cases, we propose methods which can be used to extract the maximum possible amount of information from the FRAP data.

Advances in systems biology modeling: 10 years of crowdsourcing DREAM challenges

Computational and mathematical models are key to obtain a system-level understanding of biological processes, but their limitations have to be clearly defined to allow their proper application and interpretation. Crowdsourced benchmarks in the form of challenges provide an unbiased assessment of methods, and for the past decade, the Dialogue for Reverse Engineering Assessment and Methods (DREAM) organized more than 15 systems biology challenges. From transcription factor binding to dynamical network models, from signaling networks to gene regulation, from whole-cell models to cell-lineage reconstruction, and from single-cell positioning in a tissue to drug combinations and cell survival, the breadth is broad. To celebrate the 5-year anniversary of Cell Systems, we review the genesis of these systems biology challenges and discuss how interlocking the forward- and reverse-modeling paradigms allows to push the rim of systems biology. This approach will persist for systems levels approaches in biology and medicine.

Efficient population coding depends on stimulus convergence and source of noise

Sensory organs transmit information to downstream brain circuits using a neural code comprised of spikes from multiple neurons. According to the prominent efficient coding framework, the properties of sensory populations have evolved to encode maximum information about stimuli given biophysical constraints. How information coding depends on the way sensory signals from multiple channels converge downstream is still unknown, especially in the presence of noise which corrupts the signal at different points along the pathway. Here, we calculated the optimal information transfer of a population of nonlinear neurons under two scenarios. First, a lumped-coding channel where the information from different inputs converges to a single channel, thus reducing the number of neurons. Second, an independent-coding channel when different inputs contribute independent information without convergence. In each case, we investigated information loss when the sensory signal was corrupted by two sources of noise. We determined critical noise levels at which the optimal number of distinct thresholds of individual neurons in the population changes. Comparing our system to classical physical systems, these changes correspond to first- or second-order phase transitions for the lumped- or the independent-coding channel, respectively. We relate our theoretical predictions to coding in a population of auditory nerve fibers recorded experimentally, and find signatures of efficient coding. Our results yield important insights into the diverse coding strategies used by neural populations to optimally integrate sensory stimuli in the presence of distinct sources of noise.

Inferring phenomenological models of first passage processes

Biochemical processes in cells are governed by complex networks of many chemical species interacting stochastically in diverse ways and on different time scales. Constructing microscopically accurate models of such networks is often infeasible. Instead, here we propose a systematic framework for building phenomenological models of such networks from experimental data, focusing on accurately approximating the time it takes to complete the process, the First Passage (FP) time. Our phenomenological models are mixtures of Gamma distributions, which have a natural biophysical interpretation. The complexity of the models is adapted automatically to account for the amount of available data and its temporal resolution. The framework can be used for predicting behavior of FP systems under varying external conditions. To demonstrate the utility of the approach, we build models for the distribution of inter-spike intervals of a morphologically complex neuron, a Purkinje cell, from experimental and simulated data. We demonstrate that the developed models can not only fit the data, but also make nontrivial predictions. We demonstrate that our coarse-grained models provide constraints on more mechanistically accurate models of the involved phenomena.

Solvable Model for the Linear Separability of Structured Data

Linear separability, a core concept in supervised machine learning, refers to whether the labels of a data set can be captured by the simplest possible machine: a linear classifier. In order to quantify linear separability beyond this single bit of information, one needs models of data structure parameterized by interpretable quantities, and tractable analytically. Here, I address one class of models with these properties, and show how a combinatorial method allows for the computation, in a mean field approximation, of two useful descriptors of linear separability, one of which is closely related to the popular concept of storage capacity. I motivate the need for multiple metrics by quantifying linear separability in a simple synthetic data set with controlled correlations between the points and their labels, as well as in the benchmark data set MNIST, where the capacity alone paints an incomplete picture. The analytical results indicate a high degree of “universality”, or robustness with respect to the microscopic parameters controlling data structure.

Hierarchical Approach for Comparing Collective Behavior Across Scales: Cellular Systems to Honey Bee Colonies

How individuals in a group lead to collective behavior is a fundamental question across biological systems, from cellular systems, to animal groups, to human organizations. Recent technological advancements have enabled an unprecedented increase in our ability to collect, quantify, and analyze how individual responses lead to group behavior. However, despite a wealth of data demonstrating that collective behavior exists across biological scales, it is difficult to make general statements that apply in different systems. In this perspective, we present a cohesive framework for comparing groups across different levels of biological organization, using an intermediate link of “collective mechanisms” that connects individual responses to group behavior. Using this approach we demonstrate that an effective way of comparing different groups is with an analysis hierarchy that asks complementary questions, including how individuals in a group implement various collective mechanisms, and how these various mechanisms are used to achieve group function. We apply this framework to compare two collective systems—cellular systems and honey bee colonies. Using a case study of a response to a disturbance, we compare and contrast collective mechanisms used in each system. We then discuss how inherent differences in group structure and physical constraints lead to different combinations of collective mechanisms to solve a particular problem. Together, we demonstrate how a hierarchical approach can be used to compare and contrast different systems, lead to new hypotheses in each system, and form a basis for common research questions in collective behavior.

Causal Geometry

Information geometry has offered a way to formally study the efficacy of scientific models by quantifying the impact of model parameters on the predicted effects. However, there has been little formal investigation of causation in this framework, despite causal models being a fundamental part of science and explanation. Here, we introduce causal geometry, which formalizes not only how outcomes are impacted by parameters, but also how the parameters of a model can be intervened upon. Therefore, we introduce a geometric version of "effective information"-a known measure of the informativeness of a causal relationship. We show that it is given by the matching between the space of effects and the space of interventions, in the form of their geometric congruence. Therefore, given a fixed intervention capability, an effective causal model is one that is well matched to those interventions. This is a consequence of "causal emergence," wherein macroscopic causal relationships may carry more information than "fundamental" microscopic ones. We thus argue that a coarse-grained model may, paradoxically, be more informative than the microscopic one, especially when it better matches the scale of accessible interventions-as we illustrate on toy examples.

On the Nature of Explanations Offered by Network Science: A Perspective From and for Practicing Neuroscientists

Network neuroscience represents the brain as a collection of regions and inter-regional connections. Given its ability to formalize systems-level models, network neuroscience has generated unique explanations of neural function and behavior. The mechanistic status of these explanations and how they can contribute to and fit within the field of neuroscience as a whole has received careful treatment from philosophers. However, these philosophical contributions have not yet reached many neuroscientists. Here we complement formal philosophical efforts by providing an applied perspective from and for neuroscientists. We discuss the mechanistic status of the explanations offered by network neuroscience and how they contribute to, enhance, and interdigitate with other types of explanations in neuroscience. In doing so, we rely on philosophical work concerning the role of causality, scale, and mechanisms in scientific explanations. In particular, we make the distinction between an explanation and the evidence supporting that explanation, and we argue for a scale-free nature of mechanistic explanations. In the course of these discussions, we hope to provide a useful applied framework in which network neuroscience explanations can be exercised across scales and combined with other fields of neuroscience to gain deeper insights into the brain and behavior.

Exploring the landscape of model representations

The success of any physical model critically depends upon adopting an appropriate representation for the phenomenon of interest. Unfortunately, it remains generally challenging to identify the essential degrees of freedom or, equivalently, the proper order parameters for describing complex phenomena. Here we develop a statistical physics framework for exploring and quantitatively characterizing the space of order parameters for representing physical systems. Specifically, we examine the space of low-resolution representations that correspond to particle-based coarse-grained (CG) models for a simple microscopic model of protein fluctuations. We employ Monte Carlo (MC) methods to sample this space and determine the density of states for CG representations as a function of their ability to preserve the configurational information, I, and large-scale fluctuations, Q, of the microscopic model. These two metrics are uncorrelated in high-resolution representations but become anticorrelated at lower resolutions. Moreover, our MC simulations suggest an emergent length scale for coarse-graining proteins, as well as a qualitative distinction between good and bad representations of proteins. Finally, we relate our work to recent approaches for clustering graphs and detecting communities in networks.

Training deep neural density estimators to identify mechanistic models of neural dynamics

Mechanistic modeling in neuroscience aims to explain observed phenomena in terms of underlying causes. However, determining which model parameters agree with complex and stochastic neural data presents a significant challenge. We address this challenge with a machine learning tool which uses deep neural density estimators-trained using model simulations-to carry out Bayesian inference and retrieve the full space of parameters compatible with raw data or selected data features. Our method is scalable in parameters and data features and can rapidly analyze new data after initial training. We demonstrate the power and flexibility of our approach on receptive fields, ion channels, and Hodgkin-Huxley models. We also characterize the space of circuit configurations giving rise to rhythmic activity in the crustacean stomatogastric ganglion, and use these results to derive hypotheses for underlying compensation mechanisms. Our approach will help close the gap between data-driven and theory-driven models of neural dynamics.

Quantum approximate Bayesian computation for NMR model inference

Recent technological advances may lead to the development of small scale quantum computers capable of solving problems that cannot be tackled with classical computers. A limited number of algorithms has been proposed and their relevance to real world problems is a subject of active investigation. Analysis of many-body quantum system is particularly challenging for classical computers due to the exponential scaling of Hilbert space dimension with the number of particles. Hence, solving problems relevant to chemistry and condensed matter physics are expected to be the first successful applications of quantum computers. In this paper, we propose another class of problems from the quantum realm that can be solved efficiently on quantum computers: model inference for nuclear magnetic resonance (NMR) spectroscopy, which is important for biological and medical research. Our results are based on three interconnected studies. Firstly, we use methods from classical machine learning to analyze a dataset of NMR spectra of small molecules. We perform a stochastic neighborhood embedding and identify clusters of spectra, and demonstrate that these clusters are correlated with the covalent structure of the molecules. Secondly, we propose a simple and efficient method, aided by a quantum simulator, to extract the NMR spectrum of any hypothetical molecule described by a parametric Heisenberg model. Thirdly, we propose a simple variational Bayesian inference procedure for extracting Hamiltonian parameters of experimentally relevant NMR spectra.

Sensitivity of collective outcomes identifies pivotal components

A social system is susceptible to perturbation when its collective properties depend sensitively on a few pivotal components. Using the information geometry of minimal models from statistical physics, we develop an approach to identify pivotal components to which coarse-grained, or aggregate, properties are sensitive. As an example, we introduce our approach on a reduced toy model with a median voter who always votes in the majority. The sensitivity of majority-minority divisions to changing voter behaviour pinpoints the unique role of the median. More generally, the sensitivity identifies pivotal components that precisely determine collective outcomes generated by a complex network of interactions. Using perturbations to target pivotal components in the models, we analyse datasets from political voting, finance and Twitter. Across these systems, we find remarkable variety, from systems dominated by a median-like component to those whose components behave more equally. In the context of political institutions such as courts or legislatures, our methodology can help describe how changes in voters map to new collective voting outcomes. For economic indices, differing system response reflects varying fiscal conditions across time. Thus, our information-geometric approach provides a principled, quantitative framework that may help assess the robustness of collective outcomes to targeted perturbation and compare social institutions, or even biological networks, with one another and across time.

Practical Use of Regularization in Individualizing a Mathematical Model of Cardiovascular Hemodynamics Using Scarce Data

Individualizing physiological models to a patient can enable patient-specific monitoring and treatment in critical care environments. However, this task often presents a unique "practical identifiability" challenge due to the conflict between model complexity and data scarcity. Regularization provides an established framework to cope with this conflict by compensating for data scarcity with prior knowledge. However, regularization has not been widely pursued in individualizing physiological models to facilitate patient-specific critical care. Thus, the goal of this work is to garner potentially generalizable insight into the practical use of regularization in individualizing a complex physiological model using scarce data by investigating its effect in a clinically significant critical care case study of blood volume kinetics and cardiovascular hemodynamics in hemorrhage and circulatory resuscitation. We construct a population-average model as prior knowledge and individualize the physiological model via regularization to illustrate that regularization can be effective in individualizing a physiological model to learn salient individual-specific characteristics (resulting in the goodness of fit to individual-specific data) while restricting unnecessary deviations from the population-average model (achieving practical identifiability). We also illustrate that regularization yields parsimonious individualization of only sensitive parameters as well as adequate physiological plausibility and relevance in predicting internal physiological states.

Cortical state transitions and stimulus response evolve along stiff and sloppy parameter dimensions, respectively

Previous research showed that spontaneous neuronal activity presents sloppiness: the collective behavior is strongly determined by a small number of parameter combinations, defined as ‘stiff’ dimensions, while it is insensitive to many others (‘sloppy’ dimensions). Here, we analyzed neural population activity from the auditory cortex of anesthetized rats while the brain spontaneously transited through different synchronized and desynchronized states and intermittently received sensory inputs. We showed that cortical state transitions were determined by changes in stiff parameters associated with the activity of a core of neurons with low responses to stimuli and high centrality within the observed network. In contrast, stimulus-evoked responses evolved along sloppy dimensions associated with the activity of neurons with low centrality and displaying large ongoing and stimulus-evoked fluctuations without affecting the integrity of the network. Our results shed light on the interplay among stability, flexibility, and responsiveness of neuronal collective dynamics during intrinsic and induced activity.