Cluster variation method in statistical physics and probabilistic graphical models

Tensor Network Message Passing

When studying interacting systems, computing their statistical properties is a fundamental problem in various fields such as physics, applied mathematics, and machine learning. However, this task can be quite challenging due to the exponential growth of the state space as the system size increases. Many standard methods have significant weaknesses. For instance, message-passing algorithms can be inaccurate and even fail to converge due to short loops, while tensor network methods can have exponential computational complexity in large graphs due to long loops. In this Letter, we propose a new method called "tensor network message passing." This approach allows us to compute local observables like marginal probabilities and correlations by combining the strengths of tensor networks in contracting small subgraphs with many short loops and the strengths of message-passing methods in globally sparse graphs, thus addressing the crucial weaknesses of both approaches. Our algorithm is exact for systems that are globally treelike and locally dense-connected when the dense local graphs have a limited tree width. We have conducted numerical experiments on synthetic and real-world graphs to compute magnetizations of Ising models and spin glasses, and have demonstrated the superiority of our approach over standard belief propagation and the recently proposed loopy message-passing algorithm. In addition, we discuss the potential applications of our method in inference problems in networks, combinatorial optimization problems, and decoding problems in quantum error correction.

Discrete Modeling Approach for Cluster-Based Excess Gibbs-Energy of Molecular Liquids

The excess Gibbs-energy of a two-component liquid molecular mixture is modeled based on discrete clusters of molecules. These clusters preserve the three-dimensional geometric information about local molecule neighborhoods that inform the interaction energies of the clusters. In terms of a discrete Markov-chain, the clusters are used to hypothetically construct the mixture using sequential insertion steps. Each insertion step and, therefore, cluster is assigned a probability of occurring in an equilibrium system that is determined via the constrained minimization of the Helmholtz free energy. For this, informational Shannon entropy based on these probabilities is used synonymously with thermodynamic entropy. A first approach for coupling the model to real molecules is introduced in the form of a molecular sampling algorithm, which utilizes a force-field approach to determine the energetic interactions within a cluster. An exemplary application to four mixtures shows promising results regarding the description of a variety of excess Gibbs-energy curves, including the ability to distinguish between structural isomers.

Data-driven optimal closures for mean-cluster models: Beyond the classical pair approximation

This study concerns the mean-clustering approach to modeling the evolution of lattice dynamics. Instead of tracking the state of individual lattice sites, this approach describes the time evolution of the concentrations of different cluster types. It leads to an infinite hierarchy of ordinary differential equations which must be closed by truncation using a so-called closure condition. This condition approximates the concentrations of higher-order clusters in terms of the concentrations of lower-order ones. The pair approximation is the most common form of closure. Here, we consider its generalization, termed the "optimal approximation," which we calibrate using a robust data-driven strategy. To fix attention, we focus on a recently proposed structured lattice model for a nickel-based oxide, similar to that used as cathode material in modern commercial Li-ion batteries. The form of the obtained optimal approximation allows us to deduce a simple sparse closure model. In addition to being more accurate than the classical pair approximation, this "sparse approximation" is also physically interpretable which allows us to a posteriori refine the hypotheses underlying construction of this class of closure models. Moreover, the mean-cluster model closed with this sparse approximation is linear and hence analytically solvable such that its parametrization is straightforward, although it offers a good approximation of the actual time evolution of the cluster concentrations on short timescales only. On the other hand, parametrization of the mean-cluster model closed with the pair approximation is shown to lead to an ill-posed inverse problem.

The Wako-Saitô-Muñoz-Eaton Model for Predicting Protein Folding and Dynamics

Despite the recent advances in the prediction of protein structures by deep neutral networks, the elucidation of protein-folding mechanisms remains challenging. A promising theory for describing protein folding is a coarse-grained statistical mechanical model called the Wako-Saitô-Muñoz-Eaton (WSME) model. The model can calculate the free-energy landscapes of proteins based on a three-dimensional structure with low computational complexity, thereby providing a comprehensive understanding of the folding pathways and the structure and stability of the intermediates and transition states involved in the folding reaction. In this review, we summarize previous and recent studies on protein folding and dynamics performed using the WSME model and discuss future challenges and prospects. The WSME model successfully predicted the folding mechanisms of small single-domain proteins and the effects of amino-acid substitutions on protein stability and folding in a manner that was consistent with experimental results. Furthermore, extended versions of the WSME model were applied to predict the folding mechanisms of multi-domain proteins and the conformational changes associated with protein function. Thus, the WSME model may contribute significantly to solving the protein-folding problem and is expected to be useful for predicting protein folding, stability, and dynamics in basic research and in industrial and medical applications.

Frustration-induced complexity in order-disorder transitions of the J_{1}-J_{2}-J_{3} Ising model on the square lattice

We revisit the field-free Ising model on a square lattice with up to third-neighbor (NNNN) interactions, also known as the J_{1}-J_{2}-J_{3} model, in the mean-field approximation. Using a systematic enumeration procedure, we show that the region of phase space in which the high-temperature disordered phase is stable against all modes representing periodic magnetization patterns up to a given size is a convex polytope that can be obtained by solving a standard vertex enumeration problem. Each face of this polytope corresponds to a set of coupling constants for which a single set of modes, equivalent up to a symmetry of the lattice, bifurcates from the disordered solution. While the structure of this polytope is simple in the half-space J_{3}>0, where the NNNN interaction is ferromagnetic, it becomes increasingly complex in the half-space J_{3}<0, where the antiferromagnetic NNNN interaction induces strong frustration. We then pass to the limit N→∞ giving a closed-form description of the order-disorder surface in the thermodynamic limit, which shows that for J_{3}<0, the emergent ordered phases will have a "devil's surface"-like mode structure. Finally, using Monte Carlo simulations, we show that for small periodic systems, the mean-field analysis correctly predicts the dominant modes of the ordered phases that develop for coupling constants associated with the centroid of the faces of the disorder polytope.

Emergence of effective temperatures in an out-of-equilibrium model of biopolymer folding

We investigate the possibility of extending the notion of temperature in a stochastic model for the RNA or protein folding driven out of equilibrium. We simulate the dynamics of a small RNA hairpin subject to an external pulling force, which is time-dependent. First, we consider a fluctuation-dissipation relation (FDR) whereby we verify that various effective temperatures can be obtained for different observables, only when the slowest intrinsic relaxation timescale of the system regulates the dynamics of the system. Then, we introduce a different nonequilibrium temperature, which is defined from the rate of heat exchanged with a weakly interacting thermal bath. Notably, this "kinetic" temperature can be defined for any frequency of the external switching force. We also discuss and compare the behavior of these two emerging parameters, by discriminating the time-delayed nature of the FDR temperature from the instantaneous character of the kinetic temperature. The validity of our numerics are corroborated by a simple four-state Markov model which describes the long-time behavior of the RNA molecule.

Cancer Niches and Their Kikuchi Free Energy

Biological forms depend on a progressive specialization of pluripotent stem cells. The differentiation of these cells in their spatial and functional environment defines the organism itself; however, cellular mutations may disrupt the mutual balance between a cell and its niche, where cell proliferation and specialization are released from their autopoietic homeostasis. This induces the construction of cancer niches and maintains their survival. In this paper, we characterise cancer niche construction as a direct consequence of interactions between clusters of cancer and healthy cells. Explicitly, we evaluate these higher-order interactions between niches of cancer and healthy cells using Kikuchi approximations to the free energy. Kikuchi's free energy is measured in terms of changes to the sum of energies of baseline clusters of cells (or nodes) minus the energies of overcounted cluster intersections (and interactions of interactions, etc.). We posit that these changes in energy node clusters correspond to a long-term reduction in the complexity of the system conducive to cancer niche survival. We validate this formulation through numerical simulations of apoptosis, local cancer growth, and metastasis, and highlight its implications for a computational understanding of the etiopathology of cancer.

Belief propagation for networks with loops

Belief propagation is a widely used message passing method for the solution of probabilistic models on networks such as epidemic models, spin models, and Bayesian graphical models, but it suffers from the serious shortcoming that it works poorly in the common case of networks that contain short loops. Here, we provide a solution to this long-standing problem, deriving a belief propagation method that allows for fast calculation of probability distributions in systems with short loops, potentially with high density, as well as giving expressions for the entropy and partition function, which are notoriously difficult quantities to compute. Using the Ising model as an example, we show that our approach gives excellent results on both real and synthetic networks, improving substantially on standard message passing methods. We also discuss potential applications of our method to a variety of other problems.

The 2-D Cluster Variation Method: Topography Illustrations and Their Enthalpy Parameter Correlations

One of the biggest challenges in characterizing 2-D image topographies is finding a low-dimensional parameter set that can succinctly describe, not so much image patterns themselves, but the nature of these patterns. The 2-D cluster variation method (CVM), introduced by Kikuchi in 1951, can characterize very local image pattern distributions using configuration variables, identifying nearest-neighbor, next-nearest-neighbor, and triplet configurations. Using the 2-D CVM, we can characterize 2-D topographies using just two parameters; the activation enthalpy (ε0) and the interaction enthalpy (ε1). Two different initial topographies (“scale-free-like” and “extreme rich club-like”) were each computationally brought to a CVM free energy minimum, for the case where the activation enthalpy was zero and different values were used for the interaction enthalpy. The results are: (1) the computational configuration variable results differ significantly from the analytically-predicted values well before ε1 approaches the known divergence as ε1→0.881, (2) the range of potentially useful parameter values, favoring clustering of like-with-like units, is limited to the region where ε0<3 and ε1<0.25, and (3) the topographies in the systems that are brought to a free energy minimum show interesting visual features, such as extended “spider legs” connecting previously unconnected “islands,” and as well as evolution of “peninsulas” in what were previously solid masses.

Cluster-Based Thermodynamics of Interacting Dice in a Lattice

In this paper, a model for two-component systems of six-sided dice in a simple cubic lattice is developed, based on a basic cluster approach previously proposed. The model represents a simplified picture of liquid mixtures of molecules with different interaction sites on their surfaces, where each interaction site can be assigned an individual energetic property to account for cooperative effects. Based on probabilities that characterize the sequential construction of the lattice using clusters, explicit expressions for the Shannon entropy, synonymously used as thermodynamic entropy, and the internal energy of the system are derived. The latter are used to formulate the Helmholtz free energy that is minimized to determine thermodynamic bulk properties of the system in equilibrium. The model is exemplarily applied to mixtures that contain distinct isomeric configurations of molecules, and the results are compared with the Monte-Carlo simulation results as a benchmark. The comparison shows that the model can be applied to distinguish between isomeric configurations, which suggests that it can be further developed towards an excess Gibbs-energy, respectively, activity coefficient model for chemical engineering applications.

Entropy Multiparticle Correlation Expansion for a Crystal

As first shown by H. S. Green in 1952, the entropy of a classical fluid of identical particles can be written as a sum of many-particle contributions, each of them being a distinctive functional of all spatial distribution functions up to a given order. By revisiting the combinatorial derivation of the entropy formula, we argue that a similar correlation expansion holds for the entropy of a crystalline system. We discuss how one- and two-body entropies scale with the size of the crystal, and provide fresh numerical data to check the expectation, grounded in theoretical arguments, that both entropies are extensive quantities.

State estimation of power flows for smart grids via belief propagation

Belief propagation is an algorithm that is known from statistical physics and computer science. It provides an efficient way of calculating marginals that involve large sums of products which are efficiently rearranged into nested products of sums to approximate the marginals. It allows a reliable estimation of the state and its variance of power grids that is needed for the control and forecast of power grid management. At prototypical examples of IEEE grids we show that belief propagation not only scales linearly with the grid size for the state estimation itself but also facilitates and accelerates the retrieval of missing data and allows an optimized positioning of measurement units. Based on belief propagation, we give a criterion for how to assess whether other algorithms, using only local information, are adequate for state estimation for a given grid. We also demonstrate how belief propagation can be utilized for coarse-graining power grids toward representations that reduce the computational effort when the coarse-grained version is integrated into a larger grid. It provides a criterion for partitioning power grids into areas in order to minimize the error of flow estimates between different areas.

Modules or Mean-Fields?

The segregation of neural processing into distinct streams has been interpreted by some as evidence in favour of a modular view of brain function. This implies a set of specialised ‘modules’, each of which performs a specific kind of computation in isolation of other brain systems, before sharing the result of this operation with other modules. In light of a modern understanding of stochastic non-equilibrium systems, like the brain, a simpler and more parsimonious explanation presents itself. Formulating the evolution of a non-equilibrium steady state system in terms of its density dynamics reveals that such systems appear on average to perform a gradient ascent on their steady state density. If this steady state implies a sufficiently sparse conditional independency structure, this endorses a mean-field dynamical formulation. This decomposes the density over all states in a system into the product of marginal probabilities for those states. This factorisation lends the system a modular appearance, in the sense that we can interpret the dynamics of each factor independently. However, the argument here is that it is factorisation, as opposed to modularisation, that gives rise to the functional anatomy of the brain or, indeed, any sentient system. In the following, we briefly overview mean-field theory and its applications to stochastic dynamical systems. We then unpack the consequences of this factorisation through simple numerical simulations and highlight the implications for neuronal message passing and the computational architecture of sentience.

Dynamical Transitions in a One-Dimensional Katz–Lebowitz–Spohn Model

Dynamical transitions, already found in the high- and low-density phases of the Totally Asymmetric Simple Exclusion Process and a couple of its generalizations, are singularities in the rate of relaxation towards the Non-Equilibrium Stationary State (NESS), which do not correspond to any transition in the NESS itself. We investigate dynamical transitions in the one-dimensional Katz–Lebowitz–Spohn model, a further generalization of the Totally Asymmetric Simple Exclusion Process where the hopping rate depends on the occupation state of the 2 nodes adjacent to the nodes affected by the hop. Following previous work, we choose Glauber rates and bulk-adapted boundary conditions. In particular, we consider a value of the repulsion which parameterizes the Glauber rates such that the fundamental diagram of the model exhibits 2 maxima and a minimum, and the NESS phase diagram is especially rich. We provide evidence, based on pair approximation, domain wall theory and exact finite size results, that dynamical transitions also occur in the one-dimensional Katz–Lebowitz–Spohn model, and discuss 2 new phenomena which are peculiar to this model.

Nonconvex image reconstruction via expectation propagation

The problem of efficiently reconstructing tomographic images can be mapped into a Bayesian inference problem over the space of pixels densities. Solutions to this problem are given by pixels assignments that are compatible with tomographic measurements and maximize a posterior probability density. This maximization can be performed with standard local optimization tools when the log-posterior is a convex function, but it is generally intractable when introducing realistic nonconcave priors that reflect typical images features such as smoothness or sharpness. We introduce a new method to reconstruct images obtained from Radon projections by using expectation propagation, which allows us to approximate the intractable posterior. We show, by means of extensive simulations, that, compared to state-of-the-art algorithms for this task, expectation propagation paired with very simple but non-log-concave priors is often able to reconstruct images up to a smaller error while using a lower amount of information per pixel. We provide estimates for the critical rate of information per pixel above which recovery is error-free by means of simulations on ensembles of phantom and real images.

Loop Corrections in Spin Models through Density Consistency

Computing marginal distributions of discrete or semidiscrete Markov random fields (MRFs) is a fundamental, generally intractable problem with a vast number of applications in virtually all fields of science. We present a new family of computational schemes to approximately calculate the marginals of discrete MRFs. This method shares some desirable properties with belief propagation, in particular, providing exact marginals on acyclic graphs, but it differs with the latter in that it includes some loop corrections; i.e., it takes into account correlations coming from all cycles in the factor graph. It is also similar to the adaptive Thouless-Anderson-Palmer method, but it differs with the latter in that the consistency is not on the first two moments of the distribution but rather on the value of its density on a subset of values. The results on finite-dimensional Isinglike models show a significant improvement with respect to the Bethe-Peierls (tree) approximation in all cases and with respect to the plaquette cluster variational method approximation in many cases. In particular, for the critical inverse temperature β_{c} of the homogeneous hypercubic lattice, the expansion of (dβ_{c})^{-1} around d=∞ of the proposed scheme is exact up to d^{-4} order, whereas the latter two are exact only up to d^{-2} order.

The Stochastic Complexity of Spin Models: Are Pairwise Models Really Simple?

Models can be simple for different reasons: because they yield a simple and computationally efficient interpretation of a generic dataset (e.g., in terms of pairwise dependencies)—as in statistical learning—or because they capture the laws of a specific phenomenon—as e.g., in physics—leading to non-trivial falsifiable predictions. In information theory, the simplicity of a model is quantified by the stochastic complexity, which measures the number of bits needed to encode its parameters. In order to understand how simple models look like, we study the stochastic complexity of spin models with interactions of arbitrary order. We show that bijections within the space of possible interactions preserve the stochastic complexity, which allows to partition the space of all models into equivalence classes. We thus found that the simplicity of a model is not determined by the order of the interactions, but rather by their mutual arrangements. Models where statistical dependencies are localized on non-overlapping groups of few variables are simple, affording predictions on independencies that are easy to falsify. On the contrary, fully connected pairwise models, which are often used in statistical learning, appear to be highly complex, because of their extended set of interactions, and they are hard to falsify.

The Cluster Variation Method: A Primer for Neuroscientists

Effective Brain-Computer Interfaces (BCIs) require that the time-varying activation patterns of 2-D neural ensembles be modelled. The cluster variation method (CVM) offers a means for the characterization of 2-D local pattern distributions. This paper provides neuroscientists and BCI researchers with a CVM tutorial that will help them to understand how the CVM statistical thermodynamics formulation can model 2-D pattern distributions expressing structural and functional dynamics in the brain. The premise is that local-in-time free energy minimization works alongside neural connectivity adaptation, supporting the development and stabilization of consistent stimulus-specific responsive activation patterns. The equilibrium distribution of local patterns, or configuration variables, is defined in terms of a single interaction enthalpy parameter (h) for the case of an equiprobable distribution of bistate (neural/neural ensemble) units. Thus, either one enthalpy parameter (or two, for the case of non-equiprobable distribution) yields equilibrium configuration variable values. Modeling 2-D neural activation distribution patterns with the representational layer of a computational engine, we can thus correlate variational free energy minimization with specific configuration variable distributions. The CVM triplet configuration variables also map well to the notion of a M = 3 functional motif. This paper addresses the special case of an equiprobable unit distribution, for which an analytic solution can be found.

Statistical analysis of loopy belief propagation in random fields

Loopy belief propagation (LBP), which is equivalent to the Bethe approximation in statistical mechanics, is a message-passing-type inference method that is widely used to analyze systems based on Markov random fields (MRFs). In this paper, we propose a message-passing-type method to analytically evaluate the quenched average of LBP in random fields by using the replica cluster variation method. The proposed analytical method is applicable to general pairwise MRFs with random fields whose distributions differ from each other and can give the quenched averages of the Bethe free energies over random fields, which are consistent with numerical results. The order of its computational cost is equivalent to that of standard LBP. In the latter part of this paper, we describe the application of the proposed method to Bayesian image restoration, in which we observed that our theoretical results are in good agreement with the numerical results for natural images.

Statistical physics of loopy interactions: independent-loop approximation and beyond

We consider an interacting system of spin variables on a loopy interaction graph, identified by a tree graph and a set of loopy interactions. We start from a high-temperature expansion for loopy interactions represented by a sum of non-negative contributions from all the possible frustration-free loop configurations. We then compute the loop corrections using different approximations for the nonlocal loop interactions induced by the spin correlations in the tree graph. For distant loopy interactions, we can exploit the exponential decay of correlations in the tree interaction graph to compute loop corrections within an independent-loop approximation. Higher orders of the approximation are obtained by considering the correlations between the nearby loopy interactions involving larger number of spin variables. In particular, the sum over the loop configurations can be computed "exactly" by the belief propagation algorithm in the low orders of the approximation as long as the loopy interactions have a tree structure. These results might be useful in developing more accurate and convergent message-passing algorithms exploiting the structure of loopy interactions.

Bethe free-energy approximations for disordered quantum systems

Given a locally consistent set of reduced density matrices, we construct approximate density matrices which are globally consistent with the local density matrices we started from when the trial density matrix has a tree structure. We employ the cavity method of statistical physics to find the optimal density matrix representation by slowly decreasing the temperature in an annealing algorithm, or by minimizing an approximate Bethe free energy depending on the reduced density matrices and some cavity messages originated from the Bethe approximation of the entropy. We obtain the classical Bethe expression for the entropy within a naive (mean-field) approximation of the cavity messages, which is expected to work well at high temperatures. In the next order of the approximation, we obtain another expression for the Bethe entropy depending only on the diagonal elements of the reduced density matrices. In principle, we can improve the entropy approximation by considering more accurate cavity messages in the Bethe approximation of the entropy. We compare the annealing algorithm and the naive approximation of the Bethe entropy with exact and approximate numerical simulations for small and large samples of the random transverse Ising model on random regular graphs.

Quantifying disorder through conditional entropy: an application to fluid mixing

In this paper, we present a method to quantify the extent of disorder in a system by using conditional entropies. Our approach is especially useful when other global, or mean field, measures of disorder fail. The method is equally suited for both continuum and lattice models, and it can be made rigorous for the latter. We apply it to mixing and demixing in multicomponent fluid membranes, and show that it has advantages over previous measures based on Shannon entropies, such as a much diminished dependence on binning and the ability to capture local correlations. Further potential applications are very diverse, and could include the study of local and global order in fluid mixtures, liquid crystals, magnetic materials, and particularly biomolecular systems.

Computing loop corrections by message passing

Any spanning tree in a loopy interaction graph can be used for communicating the effect of the loopy interactions by introducing messages that are passed along the edges in the spanning tree. This defines an exact mapping of the problem on the loopy interaction graph onto an extended problem on a tree interaction graph, where the thermodynamic quantities can be computed by a message-passing algorithm based on the Bethe equations. We propose an approximation loop correction algorithm for the Ising model relying on the above representation of the problem. The algorithm deals at the same time with the short and long loops, and can be used to obtain upper and lower bounds for the free energy.

Mean-field method with correlations determined by linear response

We introduce a mean-field approximation based on the reconciliation of maximum entropy and linear response for correlations in the cluster variation method. Within a general formalism that includes previous mean-field methods, we derive formulas improving on, e.g., the Bethe approximation and the Sessak-Monasson result at high temperature. Applying the method to direct and inverse Ising problems, we find improvements over standard implementations.

Inference algorithm for finite-dimensional spin glasses: belief propagation on the dual lattice

Starting from a cluster variational method, and inspired by the correctness of the paramagnetic ansatz [at high temperatures in general, and at any temperature in the two-dimensional (2D) Edwards-Anderson (EA) model] we propose a message-passing algorithm--the dual algorithm--to estimate the marginal probabilities of spin glasses on finite-dimensional lattices. We use the EA models in 2D and 3D as benchmarks. The dual algorithm improves the Bethe approximation, and we show that in a wide range of temperatures (compared to the Bethe critical temperature) our algorithm compares very well with Monte Carlo simulations, with the double-loop algorithm, and with exact calculation of the ground state of 2D systems with bimodal and Gaussian interactions. Moreover, it is usually 100 times faster than other provably convergent methods, as the double-loop algorithm. In 2D and 3D the quality of the inference deteriorates only where the correlation length becomes very large, i.e., at low temperatures in 2D and close to the critical temperature in 3D.

Belief-propagation algorithm and the Ising model on networks with arbitrary distributions of motifs

We generalize the belief-propagation algorithm to sparse random networks with arbitrary distributions of motifs (triangles, loops, etc.). Each vertex in these networks belongs to a given set of motifs (generalization of the configuration model). These networks can be treated as sparse uncorrelated hypergraphs in which hyperedges represent motifs. Here a hypergraph is a generalization of a graph, where a hyperedge can connect any number of vertices. These uncorrelated hypergraphs are treelike (hypertrees), which crucially simplifies the problem and allows us to apply the belief-propagation algorithm to these loopy networks with arbitrary motifs. As natural examples, we consider motifs in the form of finite loops and cliques. We apply the belief-propagation algorithm to the ferromagnetic Ising model with pairwise interactions on the resulting random networks and obtain an exact solution of this model. We find an exact critical temperature of the ferromagnetic phase transition and demonstrate that with increasing the clustering coefficient and the loop size, the critical temperature increases compared to ordinary treelike complex networks. However, weak clustering does not change the critical behavior qualitatively. Our solution also gives the birth point of the giant connected component in these loopy networks.

Adaptive cluster expansion for inferring boltzmann machines with noisy data

We introduce a procedure to infer the interactions among a set of binary variables, based on their sampled frequencies and pairwise correlations. The algorithm builds the clusters of variables contributing most to the entropy of the inferred Ising model and rejects the small contributions due to the sampling noise. Our procedure successfully recovers benchmark Ising models even at criticality and in the low temperature phase, and is applied to neurobiological data.

Revisiting waterlike network-forming lattice models

We revisit different three-dimensional network-forming lattice models proposed in the literature to investigate water anomalies. We perform a semianalytical calculation based on a cluster-variation technique, showing a quite good agreement with independent Monte Carlo results. The method allows us to clarify the structure of the phase diagrams, which turn out to exhibit different kinds of orientationally ordered phases. We point out that certain "waterlike" thermodynamic anomalies, claimed by previous studies, are indeed artifacts of a homogeneity assumption made in the analytical treatment. We argue that such a difficulty is common to a whole class of lattice models for water and suggest a possible way to overcome the problem in terms of "equivalent" models defined on random lattices.

Computational protein design with side-chain conformational entropy

Recent advances in modeling protein structures at the atomic level have made it possible to tackle "de novo" computational protein design. Most procedures are based on combinatorial optimization using a scoring function that estimates the folding free energy of a protein sequence on a given main-chain structure. However, the computation of the conformational entropy in the folded state is generally an intractable problem, and its contribution to the free energy is not properly evaluated. In this article, we propose a new automated protein design methodology that incorporates such conformational entropy based on statistical mechanics principles. We define the free energy of a protein sequence by the corresponding partition function over rotamer states. The free energy is written in variational form in a pairwise approximation and minimized using the Belief Propagation algorithm. In this way, a free energy is associated to each amino acid sequence: we use this insight to rescore the results obtained with a standard minimization method, with the energy as the cost function. Then, we set up a design method that directly uses the free energy as a cost function in combination with a stochastic search in the sequence space. We validate the methods on the design of three superficial sites of a small SH3 domain, and then apply them to the complete redesign of 27 proteins. Our results indicate that accounting for entropic contribution in the score function affects the outcome in a highly nontrivial way, and might improve current computational design techniques based on protein stability.

Haplotype inference in general pedigrees using the cluster variation method

We present CVMHAPLO, a probabilistic method for haplotyping in general pedigrees with many markers. CVMHAPLO reconstructs the haplotypes by assigning in every iteration a fixed number of the ordered genotypes with the highest marginal probability, conditioned on the marker data and ordered genotypes assigned in previous iterations. CVMHAPLO makes use of the cluster variation method (CVM) to efficiently estimate the marginal probabilities. We focused on single-nucleotide polymorphism (SNP) markers in the evaluation of our approach. In simulated data sets where exact computation was feasible, we found that the accuracy of CVMHAPLO was high and similar to that of maximum-likelihood methods. In simulated data sets where exact computation of the maximum-likelihood haplotype configuration was not feasible, the accuracy of CVMHAPLO was similar to that of state of the art Markov chain Monte Carlo (MCMC) maximum-likelihood approximations when all ordered genotypes were assigned and higher when only a subset of the ordered genotypes was assigned. CVMHAPLO was faster than the MCMC approach and provided more detailed information about the uncertainty in the inferred haplotypes. We conclude that CVMHAPLO is a practical tool for the inference of haplotypes in large complex pedigrees.

Three-dimensional Ising model with nearest- and next-nearest-neighbor interactions

The phase diagram of the Ising model in the presence of nearest- and next-nearest-neighbor interactions on a simple cubic lattice is studied within the framework of the differential operator technique. The Hamiltonian is solved by employing an effective-field theory with finite clusters consisting of a pair of spins. A functional form is also proposed for the free energy, similar to the Landau expansion, in order to obtain the phase diagram of the model. The transition from the ferromagnetic (or antiferromagnetic) phase to the disordered paramagnetic phase is of second order. On the other hand, a first-order transition is obtained from the lamellar phase to the paramagnetic phase, as well as from the lamellar phase to the ferromagnetic (or antiferromagnetic) phase, with the presence of a critical end point. An expected singular behavior of the first-order line at the critical end point is also obtained.

Cavity approximation for graphical models

We reformulate the cavity approximation (CA), a class of algorithms recently introduced for improving the Bethe approximation estimates of marginals in graphical models. In our formulation, which allows for the treatment of multivalued variables, a further generalization to factor graphs with arbitrary order of interaction factors is explicitly carried out, and a message passing algorithm that implements the first order correction to the Bethe approximation is described. Furthermore, we investigate an implementation of the CA for pairwise interactions. In all cases considered we could confirm that CA[k] with increasing k provides a sequence of approximations of markedly increasing precision. Furthermore, in some cases we could also confirm the general expectation that the approximation of order k , whose computational complexity is O(N(k+1)) has an error that scales as 1/N(k+1) with the size of the system. We discuss the relation between this approach and some recent developments in the field.

Edge-to-site reduction of Bethe-Peierls approximation for nearest neighbor exclusion cubic lattice particle systems and thermodynamic modeling of liquid silicates

We study an interacting particle system on the simple cubic lattice satisfying the nearest neighbor exclusion (NNE) which forbids any two nearest sites to be simultaneously occupied. Under the constraint, we develop an edge-to-site reduction of the Bethe-Peierls entropy approximation of the cluster variation method. The resulting NNE-corrected Bragg-Williams approximation is applied to statistical mechanical modeling of a liquid silicate formed by silica and a univalent network modifier, for which we derive the molar Gibbs energy of mixing and enthalpy of mixing and compare the predictions with available thermodynamic data.

Kinetics of the Wako-Saitô-Muñoz-Eaton model of protein folding

We consider a simplified model of protein folding, with binary degrees of freedom, whose equilibrium thermodynamics is exactly solvable. Based on this exact solution, the kinetics is studied in the framework of a local equilibrium approach, for which we prove that (i) the free energy decreases with time, (ii) the exact equilibrium is recovered in the infinite time limit, and (iii) the equilibration rate is an upper bound of the exact one. The kinetics is compared to the exact one for a small peptide and to Monte Carlo simulations for a longer protein; then rates are studied for a real protein and a model structure.