Galliano L, Rende R, Coslovich D. Policy-guided Monte Carlo on general state spaces: Application to glass-forming mixtures.
J Chem Phys 2024;
161:064503. [PMID:
39132794 DOI:
10.1063/5.0221221]
[Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2024] [Accepted: 07/23/2024] [Indexed: 08/13/2024] Open
Abstract
Policy-guided Monte Carlo is an adaptive method to simulate classical interacting systems. It adjusts the proposal distribution of the Metropolis-Hastings algorithm to maximize the sampling efficiency, using a formalism inspired by reinforcement learning. In this work, we first extend the policy-guided method to deal with a general state space, comprising, for instance, both discrete and continuous degrees of freedom, and then apply it to a few paradigmatic models of glass-forming mixtures. We assess the efficiency of a set of physically inspired moves whose proposal distributions are optimized through on-policy learning. Compared to conventional Monte Carlo methods, the optimized proposals are two orders of magnitude faster for an additive soft sphere mixture but yield a much more limited speed-up for the well-studied Kob-Andersen model. We discuss the current limitations of the method and suggest possible ways to improve it.
Collapse