1
|
Wang B, Audette NJ, Schneider DM, Aljadeff J. Desegregation of neuronal predictive processing. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.08.05.606684. [PMID: 39149380 PMCID: PMC11326200 DOI: 10.1101/2024.08.05.606684] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 08/17/2024]
Abstract
Neural circuits construct internal 'world-models' to guide behavior. The predictive processing framework posits that neural activity signaling sensory predictions and concurrently computing prediction-errors is a signature of those internal models. Here, to understand how the brain generates predictions for complex sensorimotor signals, we investigate the emergence of high-dimensional, multi-modal predictive representations in recurrent networks. We find that robust predictive processing arises in a network with loose excitatory/inhibitory balance. Contrary to previous proposals of functionally specialized cell-types, the network exhibits desegregation of stimulus and prediction-error representations. We confirmed these model predictions by experimentally probing predictive-coding circuits using a rich stimulus-set to violate learned expectations. When constrained by data, our model further reveals and makes concrete testable experimental predictions for the distinct functional roles of excitatory and inhibitory neurons, and of neurons in different layers along a laminar hierarchy, in computing multi-modal predictions. These results together imply that in natural conditions, neural representations of internal models are highly distributed, yet structured to allow flexible readout of behaviorally-relevant information. The generality of our model advances the understanding of computation of internal models across species, by incorporating different types of predictive computations into a unified framework.
Collapse
Affiliation(s)
- Bin Wang
- Department of Physics, University of California San Diego, La Jolla, CA, 92093, USA
| | | | - David M Schneider
- Center for Neural Science, New York University, New York, NY 10003, USA
| | - Johnatan Aljadeff
- Department of Neurobiology, University of California San Diego, La Jolla, CA, 92093, USA
| |
Collapse
|
2
|
Anand K, Khedair J, Kühn R. Structural model for fluctuations in financial markets. Phys Rev E 2018; 97:052312. [PMID: 29906875 DOI: 10.1103/physreve.97.052312] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2017] [Indexed: 11/07/2022]
Abstract
In this paper we provide a comprehensive analysis of a structural model for the dynamics of prices of assets traded in a market which takes the form of an interacting generalization of the geometric Brownian motion model. It is formally equivalent to a model describing the stochastic dynamics of a system of analog neurons, which is expected to exhibit glassy properties and thus many metastable states in a large portion of its parameter space. We perform a generating functional analysis, introducing a slow driving of the dynamics to mimic the effect of slowly varying macroeconomic conditions. Distributions of asset returns over various time separations are evaluated analytically and are found to be fat-tailed in a manner broadly in line with empirical observations. Our model also allows us to identify collective, interaction-mediated properties of pricing distributions and it predicts pricing distributions which are significantly broader than their noninteracting counterparts, if interactions between prices in the model contain a ferromagnetic bias. Using simulations, we are able to substantiate one of the main hypotheses underlying the original modeling, viz., that the phenomenon of volatility clustering can be rationalized in terms of an interplay between the dynamics within metastable states and the dynamics of occasional transitions between them.
Collapse
Affiliation(s)
- Kartik Anand
- Deutsche Bundesbank, Wilhelm-Epstein-Strasse 14, 60431 Frankfurt am Main, Germany
| | - Jonathan Khedair
- Department of Mathematics, King's College London, Strand, London WC2R 2LS, United Kingdom
| | - Reimer Kühn
- Department of Mathematics, King's College London, Strand, London WC2R 2LS, United Kingdom
| |
Collapse
|
3
|
Shiino M, Yamana M. Statistical mechanics of stochastic neural networks: Relationship between the self-consistent signal-to-noise analysis, Thouless-Anderson-Palmer equation, and replica symmetric calculation approaches. PHYSICAL REVIEW. E, STATISTICAL, NONLINEAR, AND SOFT MATTER PHYSICS 2004; 69:011904. [PMID: 14995644 DOI: 10.1103/physreve.69.011904] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/01/2003] [Indexed: 05/24/2023]
Abstract
We study the statistical mechanical aspects of stochastic analog neural network models for associative memory with correlation type learning. We take three approaches to derive the set of the order parameter equations for investigating statistical properties of retrieval states: the self-consistent signal-to-noise analysis (SCSNA), the Thouless-Anderson-Palmer (TAP) equation, and the replica symmetric calculation. On the basis of the cavity method the SCSNA can be generalized to deal with stochastic networks. We establish the close connection between the TAP equation and the SCSNA to elucidate the relationship between the Onsager reaction term of the TAP equation and the output proportional term of the SCSNA that appear in the expressions for the local fields.
Collapse
Affiliation(s)
- Masatoshi Shiino
- Department of Applied Physics, Faculty of Science, Tokyo Institute of Technology, 2-12-1 Ohokayama Meguro-ku Tokyo, Japan.
| | | |
Collapse
|
4
|
McGraw PN, Menzinger M. Bistable gradient networks. I. Attractors and pattern retrieval at low loading in the thermodynamic limit. PHYSICAL REVIEW. E, STATISTICAL, NONLINEAR, AND SOFT MATTER PHYSICS 2003; 67:016118. [PMID: 12636575 DOI: 10.1103/physreve.67.016118] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/26/2002] [Indexed: 05/24/2023]
Abstract
We examine the large-network, low-loading behavior of an attractor neural network, the so-called bistable gradient network (BGN), and compare it with that of the Hopfield network (HN). We use analytical and numerical methods to characterize the attractor states of the network and their basins of attraction. The energy landscape of BGN is more complex than that of the HN and depends on the strength of the coupling among units. At weak coupling, the BGN acts as a highly selective associative memory; the input must be close to the one of the stored patterns in order to be recognized. A category of spurious attractors occurs which is not present in the HN. Stronger coupling results in a transition to a more Hopfield-like regime with large basins of attraction. The basins of attraction for spurious attractors are noticeably suppressed compared to the Hopfield case, even though the Hebbian synaptic structure is the same and there is no stochastic noise.
Collapse
Affiliation(s)
- Patrick N McGraw
- Department of Chemistry, University of Toronto, Toronto, Ontario, Canada M5S 3H6
| | | |
Collapse
|
5
|
Yoshioka M. Linear stability analysis of retrieval state in associative memory neural networks of spiking neurons. PHYSICAL REVIEW. E, STATISTICAL, NONLINEAR, AND SOFT MATTER PHYSICS 2002; 66:061913. [PMID: 12613448 DOI: 10.1103/physreve.66.061913] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
We study associative memory neural networks of the Hodgkin-Huxley type of spiking neurons in which multiple periodic spatiotemporal patterns of spike timing are memorized as limit-cycle-type attractors. In encoding the spatiotemporal patterns, we assume the spike-timing-dependent synaptic plasticity with the asymmetric time window. Analysis for periodic solution of retrieval state reveals that if the area of the negative part of the time window is equivalent to the positive part, then crosstalk among encoded patterns vanishes. Phase transition due to the loss of the stability of periodic solution is observed when we assume fast function for direct interaction among neurons. In order to evaluate the critical point of this phase transition, we employ Floquet theory in which the stability problem of the infinite number of spiking neurons interacting with function is reduced to the eigenvalue problem with the finite size of matrix. Numerical integration of the single-body dynamics yields the explicit value of the matrix, which enables us to determine the critical point of the phase transition with a high degree of precision.
Collapse
Affiliation(s)
- Masahiko Yoshioka
- Brain Science Institute, RIKEN, Hirosawa 2-1, Wako-shi, Saitama 351-0198, Japan.
| |
Collapse
|
6
|
Yoshioka M. Spike-timing-dependent learning rule to encode spatiotemporal patterns in a network of spiking neurons. PHYSICAL REVIEW. E, STATISTICAL, NONLINEAR, AND SOFT MATTER PHYSICS 2002; 65:011903. [PMID: 11800714 DOI: 10.1103/physreve.65.011903] [Citation(s) in RCA: 11] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/09/2001] [Indexed: 05/23/2023]
Abstract
We study associative memory neural networks based on the Hodgkin-Huxley type of spiking neurons. We introduce the spike-timing-dependent learning rule, in which the time window with the negative part as well as the positive part is used to describe the biologically plausible synaptic plasticity. The learning rule is applied to encode a number of periodical spatiotemporal patterns, which are successfully reproduced in the periodical firing pattern of spiking neurons in the process of memory retrieval. The global inhibition is incorporated into the model so as to induce the gamma oscillation. The occurrence of gamma oscillation turns out to give appropriate spike timings for memory retrieval of discrete type of spatiotemporal pattern. The theoretical analysis to elucidate the stationary properties of perfect retrieval state is conducted in the limit of an infinite number of neurons and shows the good agreement with the result of numerical simulations. The result of this analysis indicates that the presence of the negative and positive parts in the form of the time window contributes to reduce the size of crosstalk term, implying that the time window with the negative and positive parts is suitable to encode a number of spatiotemporal patterns. We draw some phase diagrams, in which we find various types of phase transitions with change of the intensity of global inhibition.
Collapse
Affiliation(s)
- Masahiko Yoshioka
- Brain Science Institute, RIKEN, Hirosawa 2-1, Wako-shi, Saitama 351-0198, Japan.
| |
Collapse
|
7
|
Shiino M, Yoshida K. Chaos-nonchaos phase transitions induced by external noise in ensembles of nonlinearly coupled oscillators. PHYSICAL REVIEW. E, STATISTICAL, NONLINEAR, AND SOFT MATTER PHYSICS 2001; 63:026210. [PMID: 11308561 DOI: 10.1103/physreve.63.026210] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/25/2000] [Indexed: 05/23/2023]
Abstract
Nonlinear dynamical behaviors of ensembles of nonlinearly coupled oscillators subjected to external noise are studied on the basis of nonlinear Fokker-Planck equations. The effects of two kinds of noise, the Langevin noise and the noise introduced in the coupling strength, are investigated, and phase transitions involving chaos-nonchaos bifurcations are found to occur as the noise level is changed. An H theorem is proposed for the nonlinear Fokker-Planck equation to ensure stability of the Gaussian type solution that is approached for large times.
Collapse
Affiliation(s)
- M Shiino
- Department of Applied Physics, Faculty of Science, Tokyo Institute of Technology, 2-12-1 Ohokayama, Meguro-ku, Tokyo, Japan
| | | |
Collapse
|
8
|
Yoshioka M, Shiino M. Associative memory storing an extensive number of patterns based on a network of oscillators with distributed natural frequencies in the presence of external white noise. PHYSICAL REVIEW. E, STATISTICAL PHYSICS, PLASMAS, FLUIDS, AND RELATED INTERDISCIPLINARY TOPICS 2000; 61:4732-4744. [PMID: 11031513 DOI: 10.1103/physreve.61.4732] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/24/1999] [Indexed: 05/23/2023]
Abstract
We study associative memory based on temporal coding in which successful retrieval is realized as an entrainment in a network of simple phase oscillators with distributed natural frequencies under the influence of white noise. The memory patterns are assumed to be given by uniformly distributed random numbers on [0, 2 pi) so that the patterns encode the phase differences of the oscillators. To derive the macroscopic order parameter equations for the network with an extensive number of stored patterns, we introduce an effective transfer function by assuming a fixed-point equation of the form of the Thouless-Anderson-Palmer equation, which describes the time-averaged output as a function of the effective time-averaged local field. Properties of the networks associated with synchronization phenomena for a discrete symmetric natural frequency distribution with three frequency components are studied based on the order parameter equations, and are shown to be in good agreement with the results of numerical simulations. Two types of retrieval states are found to occur with respect to the degree of synchronization, when the size of the width of the natural frequency distribution is changed.
Collapse
Affiliation(s)
- M Yoshioka
- Department of Applied Physics, Tokyo Institute of Technology, Ohokayama, Japan.
| | | |
Collapse
|
9
|
Erichsen R, Theumann WK, Dominguez DR. Categorization in fully connected multistate neural network models. PHYSICAL REVIEW. E, STATISTICAL PHYSICS, PLASMAS, FLUIDS, AND RELATED INTERDISCIPLINARY TOPICS 1999; 60:7321-31. [PMID: 11970677 DOI: 10.1103/physreve.60.7321] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/22/1999] [Indexed: 11/07/2022]
Abstract
The categorization ability of fully connected neural network models, with either discrete or continuous Q-state units, is studied in this work in replica symmetric mean-field theory. Hierarchically correlated multistate patterns in a two level structure of ancestors and descendents (examples) are embedded in the network and the categorization task consists in recognizing the ancestors when the network is trained exclusively with their descendents. Explicit results for the dependence of the equilibrium properties of a Q=3-state model and a Q=infinity-state model are obtained in the form of phase diagrams and categorization curves. A strong improvement of the categorization ability is found when the network is trained with examples of low activity. The categorization ability is found to be robust to finite threshold and synaptic noise. The Almeida-Thouless lines that limit the validity of the replica-symmetric results, are also obtained.
Collapse
Affiliation(s)
- R Erichsen
- Instituto de Física, Universidade Federal do Rio Grande do Sul, Caixa Postal 15051, CEP 91501-970, Porto Alegre, RS, Brazil
| | | | | |
Collapse
|
10
|
Bollé D, Kozlowski P. Statics and dynamics of an Ashkin-Teller neural network with low loading. ACTA ACUST UNITED AC 1999. [DOI: 10.1088/0305-4470/31/30/002] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
11
|
Yamana M, Shiino M, Yoshioka M. Oscillator neural network model with distributed native frequencies. ACTA ACUST UNITED AC 1999. [DOI: 10.1088/0305-4470/32/19/305] [Citation(s) in RCA: 21] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
|
12
|
Simmen MW, Treves A, Rolls ET. Pattern retrieval in threshold-linear associative nets. NETWORK (BRISTOL, ENGLAND) 1996; 7:109-122. [PMID: 29480145 DOI: 10.1080/0954898x.1996.11978657] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Networks of threshold-linear neurons have previously been introduced and analysed as distributed associative memory systems. Here, results from simulations of pattern retrieval in a large-scale, sparsely connected network are presented. The storage capacity lies near a = 0.8 and 1.2 for binary and ternary patterns respectively, in reasonable accordance with theoretical estimates. The system is capable of retrieving states strongly correlated with one of the stored patterns even when the initial state is a highly degraded version of one of these patterns. This pattern completion ability holds for an extensive number of memory patterns, up to α ≈ αc/2, thereby increasing the credibility of the model as an effective associative memory.
Collapse
Affiliation(s)
- Martin W Simmen
- a Centre for Cognitive Science , University of Edinburgh , 2 Buccleuch Place, Edinburgh EH8 9LW , UK
| | - Alessandro Treves
- b Biophysics and Cognitive Science , SISSA, via Beirut 2, 34013 Trieste , Italy
| | - Edmund T Rolls
- c Department of Experimental Psychology , University of Oxford , South Parks Road, Oxford 0X1 3UD , UK
| |
Collapse
|
13
|
|
14
|
Brunel N, Zecchina R. Response functions improving performance in analog attractor neural networks. PHYSICAL REVIEW. E, STATISTICAL PHYSICS, PLASMAS, FLUIDS, AND RELATED INTERDISCIPLINARY TOPICS 1994; 49:R1823-R1826. [PMID: 9961512 DOI: 10.1103/physreve.49.r1823] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2023]
|