Exponential Capacity of Dense Associative Memories

Recent generalizations of the Hopfield model of associative memories are able to store a number P of random patterns that grows exponentially with the number N of neurons, P=exp(αN). Besides the huge storage capacity, another interesting feature of these networks is their connection to the attention mechanism which is part of the Transformer architecture widely applied in deep learning. In this work, we study a generic family of pattern ensembles using a statistical mechanics analysis which gives exact asymptotic thresholds for the retrieval of a typical pattern, α_{1}, and lower bounds for the maximum of the load α for which all patterns can be retrieved, α_{c}, as well as sizes of attraction basins. We discuss in detail the cases of Gaussian and spherical patterns, and show that they display rich and qualitatively different phase diagrams.

Dynamics at the edge for independent diffusing particles

We study the dynamics of the outliers for a large number of independent Brownian particles in one dimension. We derive the multitime joint distribution of the position of the rightmost particle, by two different methods. We obtain the two-time joint distribution of the maximum and second maximum positions, and we study the counting statistics at the edge. Finally, we derive the multitime joint distribution of the running maximum, as well as the joint distribution of the arrival time of the first particle at several space points.

A central limit theorem and a law of the iterated logarithm for the Biggins martingale of the supercritical branching random walk

Let (Wn(θ))n∈ℕ0 be the Biggins martingale associated with a supercritical branching random walk, and denote by W_∞(θ) its limit. Assuming essentially that the martingale (Wn(2θ))n∈ℕ0 is uniformly integrable and that var W1(θ) is finite, we prove a functional central limit theorem for the tail process (W∞(θ)-Wn+r(θ))r∈ℕ0 and a law of the iterated logarithm for W∞(θ)-Wn(θ) as n→∞.

Phase transition in the Jarzynski estimator of free energy differences

The transition between a regime in which thermodynamic relations apply only to ensembles of small systems coupled to a large environment and a regime in which they can be used to characterize individual macroscopic systems is analyzed in terms of the change in behavior of the Jarzynski estimator of equilibrium free energy differences from nonequilibrium work measurements. Given a fixed number of measurements, the Jarzynski estimator is unbiased for sufficiently small systems. In these systems the directionality of time is poorly defined and the configurations that dominate the empirical average, but which are in fact typical of the reverse process, are sufficiently well sampled. As the system size increases the arrow of time becomes better defined. The dominant atypical fluctuations become rare and eventually cannot be sampled with the limited resources that are available. Asymptotically, only typical work values are measured. The Jarzynski estimator becomes maximally biased and approaches the exponential of minus the average work, which is the result that is expected from standard macroscopic thermodynamics. In the proper scaling limit, this regime change has been recently described in terms of a phase transition in variants of the random energy model. In this paper this correspondence is further demonstrated in two examples of physical interest: the sudden compression of an ideal gas and adiabatic quasistatic volume changes in a dilute real gas.

Improving free-energy estimates from unidirectional work measurements: theory and experiment

We derive analytical expressions for the bias of the Jarzynski free-energy estimator from N nonequilibrium work measurements, for a generic work distribution. To achieve this, we map the estimator onto the random energy model in a suitable scaling limit parametrized by (logN)/μ, where μ measures the width of the lower tail of the work distribution, and then compute the finite-N corrections to this limit with different approaches for different regimes of (logN)/μ. We show that these expressions describe accurately the bias for a wide class of work distributions and exploit them to build an improved free-energy estimator from unidirectional work measurements. We apply the method to optical tweezers unfolding and refolding experiments on DNA hairpins of varying loop size and dissipation, displaying both near-Gaussian and non-Gaussian work distributions.

Critical weight statistics of the random energy model and of the directed polymer on the Cayley tree

We consider the critical point of two mean-field disordered models: (i) the random energy model (REM), introduced by Derrida as a mean-field spin-glass model of N spins and (ii) the directed polymer of length N on a Cayley Tree (DPCT) with random bond energies. Both models are known to exhibit a freezing transition between a high-temperature phase where the entropy is extensive and a low-temperature phase of finite entropy, where the weight statistics coincides with the weight statistics of Lévy sums with index mu=TT{c}<1 . In this paper, we study the weight statistics at criticality via the entropy S=Sigma w{i}lnw{i} and the generalized moments Y{k}= Sigma w{i}{k} , where the w{i} are the Boltzmann weights of the 2{N} configurations. In the REM, we find that the critical weight statistics is governed by the finite-size exponent nu=2 : the entropy scales as S[over]{N}(T{c}) approximately N{12} , the typical values e{lnY{k}[over]} decay as N{-k2} , and the disorder-averaged values Y{k}[over] are governed by rare events and decay as N{-12} for any k>1 . For the DPCT, we find that the entropy scales similarly as S[over]{N}(T{c}) approximately N{12} , whereas another exponent nu'=1 governs the Y{k} statistics: the typical values e{lnY{k}[over]} decay as N{-k} , and the disorder-averaged values Y{k}[over] decay as N{-1} for any k>1 . As a consequence, the asymptotic probability distribution pi[over]{N=infinity}(q) of the overlap q , in addition to the delta function delta(q) , which bears the whole normalization, contains an isolated point at q=1 , as a memory of the delta peak (1-TT{c})delta(q-1) of the low-temperature phase T<T{c} . The associated value pi[over]{N=infinity}(q=1) is finite for the DPCT, and diverges as pi[over]{N=infinity}(q=1) approximately N{12} for the REM.