1
|
Mattingly HH, Kamino K, Ong J, Kottou R, Emonet T, Machta BB. E. coli do not count single molecules. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.07.09.602750. [PMID: 39026702 PMCID: PMC11257612 DOI: 10.1101/2024.07.09.602750] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/20/2024]
Abstract
Organisms must perform sensory-motor behaviors to survive. What bounds or constraints limit behavioral performance? Previously, we found that the gradient-climbing speed of a chemotaxing Escherichia coli is near a bound set by the limited information they acquire from their chemical environments (1). Here we ask what limits their sensory accuracy. Past theoretical analyses have shown that the stochasticity of single molecule arrivals sets a fundamental limit on the precision of chemical sensing (2). Although it has been argued that bacteria approach this limit, direct evidence is lacking. Here, using information theory and quantitative experiments, we find that E. coli's chemosensing is not limited by the physics of particle counting. First, we derive the physical limit on the behaviorally-relevant information that any sensor can get about a changing chemical concentration, assuming that every molecule arriving at the sensor is recorded. Then, we derive and measure how much information E. coli's signaling pathway encodes during chemotaxis. We find that E. coli encode two orders of magnitude less information than an ideal sensor limited only by shot noise in particle arrivals. These results strongly suggest that constraints other than particle arrival noise limit E. coli's sensory fidelity.
Collapse
Affiliation(s)
| | | | - Jude Ong
- Molecular, Cellular, and Developmental Biology, Yale University
| | - Rafaela Kottou
- Molecular, Cellular, and Developmental Biology, Yale University
| | - Thierry Emonet
- Molecular, Cellular, and Developmental Biology, Yale University
- Physics, Yale University
- QBio Institute, Yale University
| | | |
Collapse
|
2
|
Moore JJ, Genkin A, Tournoy M, Pughe-Sanford JL, de Ruyter van Steveninck RR, Chklovskii DB. The neuron as a direct data-driven controller. Proc Natl Acad Sci U S A 2024; 121:e2311893121. [PMID: 38913890 PMCID: PMC11228465 DOI: 10.1073/pnas.2311893121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 04/12/2024] [Indexed: 06/26/2024] Open
Abstract
In the quest to model neuronal function amid gaps in physiological data, a promising strategy is to develop a normative theory that interprets neuronal physiology as optimizing a computational objective. This study extends current normative models, which primarily optimize prediction, by conceptualizing neurons as optimal feedback controllers. We posit that neurons, especially those beyond early sensory areas, steer their environment toward a specific desired state through their output. This environment comprises both synaptically interlinked neurons and external motor sensory feedback loops, enabling neurons to evaluate the effectiveness of their control via synaptic feedback. To model neurons as biologically feasible controllers which implicitly identify loop dynamics, infer latent states, and optimize control we utilize the contemporary direct data-driven control (DD-DC) framework. Our DD-DC neuron model explains various neurophysiological phenomena: the shift from potentiation to depression in spike-timing-dependent plasticity with its asymmetry, the duration and adaptive nature of feedforward and feedback neuronal filters, the imprecision in spike generation under constant stimulation, and the characteristic operational variability and noise in the brain. Our model presents a significant departure from the traditional, feedforward, instant-response McCulloch-Pitts-Rosenblatt neuron, offering a modern, biologically informed fundamental unit for constructing neural networks.
Collapse
Affiliation(s)
- Jason J Moore
- Neuroscience Institute, New York University Grossman School of Medicine, New York City, NY 10016
- Center for Computational Neuroscience, Flatiron Institute, New York City, NY 10010
| | - Alexander Genkin
- Center for Computational Neuroscience, Flatiron Institute, New York City, NY 10010
| | - Magnus Tournoy
- Center for Computational Neuroscience, Flatiron Institute, New York City, NY 10010
| | | | | | - Dmitri B Chklovskii
- Neuroscience Institute, New York University Grossman School of Medicine, New York City, NY 10016
- Center for Computational Neuroscience, Flatiron Institute, New York City, NY 10010
| |
Collapse
|
3
|
Hansel C. Contiguity in perception: origins in cellular associative computations. Trends Neurosci 2024; 47:170-180. [PMID: 38310022 PMCID: PMC10939850 DOI: 10.1016/j.tins.2024.01.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Revised: 11/30/2023] [Accepted: 01/05/2024] [Indexed: 02/05/2024]
Abstract
Our brains are good at detecting and learning associative structures; according to some linguistic theories, this capacity even constitutes a prerequisite for the development of syntax and compositionality in language and verbalized thought. I will argue that the search for associative motifs in input patterns is an evolutionary old brain function that enables contiguity in sensory perception and orientation in time and space. It has its origins in an elementary material property of cells that is particularly evident at chemical synapses: input-assigned calcium influx that activates calcium sensor proteins involved in memory storage. This machinery for the detection and learning of associative motifs generates knowledge about input relationships and integrates this knowledge into existing networks through updates in connectivity patterns.
Collapse
Affiliation(s)
- Christian Hansel
- Department of Neurobiology, University of Chicago, Chicago, IL 60637, USA.
| |
Collapse
|
4
|
Charvin H, Catenacci Volpi N, Polani D. Exact and Soft Successive Refinement of the Information Bottleneck. ENTROPY (BASEL, SWITZERLAND) 2023; 25:1355. [PMID: 37761653 PMCID: PMC10528077 DOI: 10.3390/e25091355] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 09/08/2023] [Accepted: 09/13/2023] [Indexed: 09/29/2023]
Abstract
The information bottleneck (IB) framework formalises the essential requirement for efficient information processing systems to achieve an optimal balance between the complexity of their representation and the amount of information extracted about relevant features. However, since the representation complexity affordable by real-world systems may vary in time, the processing cost of updating the representations should also be taken into account. A crucial question is thus the extent to which adaptive systems can leverage the information content of already existing IB-optimal representations for producing new ones, which target the same relevant features but at a different granularity. We investigate the information-theoretic optimal limits of this process by studying and extending, within the IB framework, the notion of successive refinement, which describes the ideal situation where no information needs to be discarded for adapting an IB-optimal representation's granularity. Thanks in particular to a new geometric characterisation, we analytically derive the successive refinability of some specific IB problems (for binary variables, for jointly Gaussian variables, and for the relevancy variable being a deterministic function of the source variable), and provide a linear-programming-based tool to numerically investigate, in the discrete case, the successive refinement of the IB. We then soften this notion into a quantification of the loss of information optimality induced by several-stage processing through an existing measure of unique information. Simple numerical experiments suggest that this quantity is typically low, though not entirely negligible. These results could have important implications for (i) the structure and efficiency of incremental learning in biological and artificial agents, (ii) the comparison of IB-optimal observation channels in statistical decision problems, and (iii) the IB theory of deep neural networks.
Collapse
Affiliation(s)
- Hippolyte Charvin
- School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield AL10 9AB, UK; (N.C.V.); (D.P.)
| | | | | |
Collapse
|
5
|
Gutierrez GJ, Wang S. Gap junctions: The missing piece of the connectome. Curr Biol 2023; 33:R819-R822. [PMID: 37552951 DOI: 10.1016/j.cub.2023.06.058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/10/2023]
Abstract
The central pattern generator that controls flying power in Drosophila requires desynchronized firing to drive a steady wingbeat frequency. A new study reveals how gap junctions are the key to desynchronizing the motor neurons.
Collapse
Affiliation(s)
- Gabrielle J Gutierrez
- Department of Neuroscience and Behavior, Barnard College, 3009 Broadway, New York, NY 10027, USA.
| | - Siwei Wang
- Department of Organismal Biology and Anatomy, University of Chicago, 1027 E 57th Street, Chicago, IL 60637, USA.
| |
Collapse
|
6
|
Talley J, Pusdekar S, Feltenberger A, Ketner N, Evers J, Liu M, Gosh A, Palmer SE, Wardill TJ, Gonzalez-Bellido PT. Predictive saccades and decision making in the beetle-predating saffron robber fly. Curr Biol 2023:S0960-9822(23)00770-4. [PMID: 37379842 DOI: 10.1016/j.cub.2023.06.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 04/28/2023] [Accepted: 06/06/2023] [Indexed: 06/30/2023]
Abstract
Internal predictions about the sensory consequences of self-motion, encoded by corollary discharge, are ubiquitous in the animal kingdom, including for fruit flies, dragonflies, and humans. In contrast, predicting the future location of an independently moving external target requires an internal model. With the use of internal models for predictive gaze control, vertebrate predatory species compensate for their sluggish visual systems and long sensorimotor latencies. This ability is crucial for the timely and accurate decisions that underpin a successful attack. Here, we directly demonstrate that the robber fly Laphria saffrana, a specialized beetle predator, also uses predictive gaze control when head tracking potential prey. Laphria uses this predictive ability to perform the difficult categorization and perceptual decision task of differentiating a beetle from other flying insects with a low spatial resolution retina. Specifically, we show that (1) this predictive behavior is part of a saccade-and-fixate strategy, (2) the relative target angular position and velocity, acquired during fixation, inform the subsequent predictive saccade, and (3) the predictive saccade provides Laphria with additional fixation time to sample the frequency of the prey's specular wing reflections. We also demonstrate that Laphria uses such wing reflections as a proxy for the wingbeat frequency of the potential prey and that consecutively flashing LEDs to produce apparent motion elicits attacks when the LED flicker frequency matches that of the beetle's wingbeat cycle.
Collapse
Affiliation(s)
- Jennifer Talley
- Air Force Research Laboratory, Munitions Directorate, Eglin AFB, FL 32542, USA.
| | - Siddhant Pusdekar
- Department of Ecology, Evolution and Behavior, University of Minnesota, Saint Paul, MN 55108, USA
| | - Aaron Feltenberger
- Air Force Research Laboratory, Munitions Directorate, Eglin AFB, FL 32542, USA
| | - Natalie Ketner
- Air Force Research Laboratory, Munitions Directorate, Eglin AFB, FL 32542, USA
| | - Johnny Evers
- Air Force Research Laboratory, Munitions Directorate, Eglin AFB, FL 32542, USA
| | - Molly Liu
- Department of Ecology, Evolution and Behavior, University of Minnesota, Saint Paul, MN 55108, USA
| | - Atishya Gosh
- Department of Ecology, Evolution and Behavior, University of Minnesota, Saint Paul, MN 55108, USA; Department of Biomedical Informatics and Computational Biology, University of Minnesota, Minneapolis, MN 55455, USA
| | - Stephanie E Palmer
- Department of Organismal Biology and Anatomy, The University of Chicago, Chicago, IL 60637, USA
| | - Trevor J Wardill
- Department of Ecology, Evolution and Behavior, University of Minnesota, Saint Paul, MN 55108, USA; Department of Biomedical Informatics and Computational Biology, University of Minnesota, Minneapolis, MN 55455, USA
| | - Paloma T Gonzalez-Bellido
- Department of Ecology, Evolution and Behavior, University of Minnesota, Saint Paul, MN 55108, USA; Department of Biomedical Informatics and Computational Biology, University of Minnesota, Minneapolis, MN 55455, USA.
| |
Collapse
|
7
|
Ngampruetikorn V, Schwab DJ. Information bottleneck theory of high-dimensional regression: relevancy, efficiency and optimality. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 2022; 35:9784-9796. [PMID: 37332888 PMCID: PMC10275337] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/20/2023]
Abstract
Avoiding overfitting is a central challenge in machine learning, yet many large neural networks readily achieve zero training loss. This puzzling contradiction necessitates new approaches to the study of overfitting. Here we quantify overfitting via residual information, defined as the bits in fitted models that encode noise in training data. Information efficient learning algorithms minimize residual information while maximizing the relevant bits, which are predictive of the unknown generative models. We solve this optimization to obtain the information content of optimal algorithms for a linear regression problem and compare it to that of randomized ridge regression. Our results demonstrate the fundamental trade-off between residual and relevant information and characterize the relative information efficiency of randomized regression with respect to optimal algorithms. Finally, using results from random matrix theory, we reveal the information complexity of learning a linear map in high dimensions and unveil information-theoretic analogs of double and multiple descent phenomena.
Collapse
Affiliation(s)
| | - David J. Schwab
- Initiative for the Theoretical Sciences, The Graduate Center, CUNY
| |
Collapse
|
8
|
Price BH, Gavornik JP. Efficient Temporal Coding in the Early Visual System: Existing Evidence and Future Directions. Front Comput Neurosci 2022; 16:929348. [PMID: 35874317 PMCID: PMC9298461 DOI: 10.3389/fncom.2022.929348] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2022] [Accepted: 06/13/2022] [Indexed: 01/16/2023] Open
Abstract
While it is universally accepted that the brain makes predictions, there is little agreement about how this is accomplished and under which conditions. Accurate prediction requires neural circuits to learn and store spatiotemporal patterns observed in the natural environment, but it is not obvious how such information should be stored, or encoded. Information theory provides a mathematical formalism that can be used to measure the efficiency and utility of different coding schemes for data transfer and storage. This theory shows that codes become efficient when they remove predictable, redundant spatial and temporal information. Efficient coding has been used to understand retinal computations and may also be relevant to understanding more complicated temporal processing in visual cortex. However, the literature on efficient coding in cortex is varied and can be confusing since the same terms are used to mean different things in different experimental and theoretical contexts. In this work, we attempt to provide a clear summary of the theoretical relationship between efficient coding and temporal prediction, and review evidence that efficient coding principles explain computations in the retina. We then apply the same framework to computations occurring in early visuocortical areas, arguing that data from rodents is largely consistent with the predictions of this model. Finally, we review and respond to criticisms of efficient coding and suggest ways that this theory might be used to design future experiments, with particular focus on understanding the extent to which neural circuits make predictions from efficient representations of environmental statistics.
Collapse
|
9
|
Kline AG, Palmer SE. Gaussian Information Bottleneck and the Non-Perturbative Renormalization Group. NEW JOURNAL OF PHYSICS 2022; 24:033007. [PMID: 35368649 PMCID: PMC8967309 DOI: 10.1088/1367-2630/ac395d] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
The renormalization group (RG) is a class of theoretical techniques used to explain the collective physics of interacting, many-body systems. It has been suggested that the RG formalism may be useful in finding and interpreting emergent low-dimensional structure in complex systems outside of the traditional physics context, such as in biology or computer science. In such contexts, one common dimensionality-reduction framework already in use is information bottleneck (IB), in which the goal is to compress an "input" signal X while maximizing its mutual information with some stochastic "relevance" variable Y. IB has been applied in the vertebrate and invertebrate processing systems to characterize optimal encoding of the future motion of the external world. Other recent work has shown that the RG scheme for the dimer model could be "discovered" by a neural network attempting to solve an IB-like problem. This manuscript explores whether IB and any existing formulation of RG are formally equivalent. A class of soft-cutoff non-perturbative RG techniques are defined by families of non-deterministic coarsening maps, and hence can be formally mapped onto IB, and vice versa. For concreteness, this discussion is limited entirely to Gaussian statistics (GIB), for which IB has exact, closed-form solutions. Under this constraint, GIB has a semigroup structure, in which successive transformations remain IB-optimal. Further, the RG cutoff scheme associated with GIB can be identified. Our results suggest that IB can be used to impose a notion of "large scale" structure, such as biological function, on an RG procedure.
Collapse
Affiliation(s)
- Adam G Kline
- Department of Physics, The University of Chicago, Chicago IL 60637
| | - Stephanie E Palmer
- Department of Organismal Biology and Anatomy and Department of Physics, The University of Chicago, Chicago IL 60637
| |
Collapse
|