1
|
Drissi M, Keeble JWT, Rozalén Sarmiento J, Rios A. Second-order optimization strategies for neural network quantum states. PHILOSOPHICAL TRANSACTIONS. SERIES A, MATHEMATICAL, PHYSICAL, AND ENGINEERING SCIENCES 2024; 382:20240057. [PMID: 38910393 DOI: 10.1098/rsta.2024.0057] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/31/2024] [Accepted: 05/13/2024] [Indexed: 06/25/2024]
Abstract
The Variational Monte Carlo (VMC) method has recently seen important advances through the use of neural network quantum states. While more and more sophisticated ansatze have been designed to tackle a wide variety of quantum many-body problems, modest progress has been made on the associated optimization algorithms. In this work, we revisit the Kronecker-Factored Approximate Curvature (KFAC), an optimizer that has been used extensively in a variety of simulations. We suggest improvements in the scaling and the direction of this optimizer and find that they substantially increase its performance at a negligible additional cost. We also reformulate the VMC approach in a game theory framework, to propose a novel optimizer based on decision geometry. We find that on a practical test case for continuous systems, this new optimizer consistently outperforms any of the KFAC improvements in terms of stability, accuracy and speed of convergence. Beyond VMC, the versatility of this approach suggests that decision geometry could provide a solid foundation for accelerating a broad class of machine learning algorithms. This article is part of the theme issue 'The liminal position of Nuclear Physics: from hadrons to neutron stars'.
Collapse
Affiliation(s)
- M Drissi
- TRIUMF , Vancouver, British Columbia V6T 2A3, Canada
| | - J W T Keeble
- Department of Physics, University of Surrey , Guildford, GU2 7XH, UK
| | - J Rozalén Sarmiento
- Departament de Física Quàntica i Astrofísica, Universitat de Barcelona (UB) , c. Martí i Franquès 1, Barcelona E08028, Spain
- Institut de Ciències del Cosmos (ICCUB), Universitat de Barcelona (UB) , Barcelona, Spain
| | - A Rios
- Department of Physics, University of Surrey , Guildford, GU2 7XH, UK
- Departament de Física Quàntica i Astrofísica, Universitat de Barcelona (UB) , c. Martí i Franquès 1, Barcelona E08028, Spain
- Institut de Ciències del Cosmos (ICCUB), Universitat de Barcelona (UB) , Barcelona, Spain
| |
Collapse
|
2
|
Li X, Huang JC, Zhang GZ, Li HE, Shen ZP, Zhao C, Li J, Hu HS. Improved optimization for the neural-network quantum states and tests on the chromium dimer. J Chem Phys 2024; 160:234102. [PMID: 38884396 DOI: 10.1063/5.0214150] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2024] [Accepted: 05/30/2024] [Indexed: 06/18/2024] Open
Abstract
The advent of Neural-network Quantum States (NQS) has significantly advanced wave function ansatz research, sparking a resurgence in orbital space variational Monte Carlo (VMC) exploration. This work introduces three algorithmic enhancements to reduce computational demands of VMC optimization using NQS: an adaptive learning rate algorithm, constrained optimization, and block optimization. We evaluate the refined algorithm on complex multireference bond stretches of H2O and N2 within the cc-pVDZ basis set and calculate the ground-state energy of the strongly correlated chromium dimer (Cr2) in the Ahlrichs SV basis set. Our results achieve superior accuracy compared to coupled cluster theory at a relatively modest CPU cost. This work demonstrates how to enhance optimization efficiency and robustness using these strategies, opening a new path to optimize large-scale restricted Boltzmann machine-based NQS more effectively and marking a substantial advancement in NQS's practical quantum chemistry applications.
Collapse
Affiliation(s)
- Xiang Li
- Department of Chemistry and Engineering Research Center of Advanced Rare-Earth Materials of Ministry of Education, Tsinghua University, Beijing 100084, China
| | - Jia-Cheng Huang
- Department of Chemistry and Engineering Research Center of Advanced Rare-Earth Materials of Ministry of Education, Tsinghua University, Beijing 100084, China
| | - Guang-Ze Zhang
- Department of Chemistry and Engineering Research Center of Advanced Rare-Earth Materials of Ministry of Education, Tsinghua University, Beijing 100084, China
| | - Hao-En Li
- Department of Chemistry and Engineering Research Center of Advanced Rare-Earth Materials of Ministry of Education, Tsinghua University, Beijing 100084, China
| | - Zhu-Ping Shen
- Department of Chemistry and Engineering Research Center of Advanced Rare-Earth Materials of Ministry of Education, Tsinghua University, Beijing 100084, China
| | - Chen Zhao
- Department of Chemistry and Engineering Research Center of Advanced Rare-Earth Materials of Ministry of Education, Tsinghua University, Beijing 100084, China
| | - Jun Li
- Department of Chemistry and Engineering Research Center of Advanced Rare-Earth Materials of Ministry of Education, Tsinghua University, Beijing 100084, China
- Department of Chemistry and Guangdong Provincial Key Laboratory of Catalytic Chemistry, Southern University of Science and Technology, Shenzhen 518055, China
- Fundamental Science Center of Rare Earths, Ganjiang Innovation Academy, Chinese Academy of Sciences, Ganzhou 341000, China
| | - Han-Shi Hu
- Department of Chemistry and Engineering Research Center of Advanced Rare-Earth Materials of Ministry of Education, Tsinghua University, Beijing 100084, China
| |
Collapse
|
3
|
Hermann J, Spencer J, Choo K, Mezzacapo A, Foulkes WMC, Pfau D, Carleo G, Noé F. Ab initio quantum chemistry with neural-network wavefunctions. Nat Rev Chem 2023; 7:692-709. [PMID: 37558761 DOI: 10.1038/s41570-023-00516-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/16/2023] [Indexed: 08/11/2023]
Abstract
Deep learning methods outperform human capabilities in pattern recognition and data processing problems and now have an increasingly important role in scientific discovery. A key application of machine learning in molecular science is to learn potential energy surfaces or force fields from ab initio solutions of the electronic Schrödinger equation using data sets obtained with density functional theory, coupled cluster or other quantum chemistry (QC) methods. In this Review, we discuss a complementary approach using machine learning to aid the direct solution of QC problems from first principles. Specifically, we focus on quantum Monte Carlo methods that use neural-network ansatzes to solve the electronic Schrödinger equation, in first and second quantization, computing ground and excited states and generalizing over multiple nuclear configurations. Although still at their infancy, these methods can already generate virtually exact solutions of the electronic Schrödinger equation for small systems and rival advanced conventional QC methods for systems with up to a few dozen electrons.
Collapse
Affiliation(s)
- Jan Hermann
- Microsoft Research AI4Science, Berlin, Germany
- FU Berlin, Department of Mathematics and Computer Science, Berlin, Germany
| | | | - Kenny Choo
- Department of Physics, University of Zurich, Zurich, Switzerland
- IBM Quantum, IBM Research Zurich, Ruschlikon, Switzerland
| | | | - W M C Foulkes
- Imperial College London, Department of Physics, London, UK
| | - David Pfau
- DeepMind, London, UK.
- Imperial College London, Department of Physics, London, UK.
| | | | - Frank Noé
- Microsoft Research AI4Science, Berlin, Germany.
- FU Berlin, Department of Mathematics and Computer Science, Berlin, Germany.
- FU Berlin, Department of Physics, Berlin, Germany.
- Department of Chemistry,Rice University, Houston, TX, USA.
| |
Collapse
|
4
|
Martyn JM, Najafi K, Luo D. Variational Neural-Network Ansatz for Continuum Quantum Field Theory. PHYSICAL REVIEW LETTERS 2023; 131:081601. [PMID: 37683171 DOI: 10.1103/physrevlett.131.081601] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Revised: 07/21/2023] [Accepted: 07/25/2023] [Indexed: 09/10/2023]
Abstract
Physicists dating back to Feynman have lamented the difficulties of applying the variational principle to quantum field theories. In nonrelativistic quantum field theories, the challenge is to parametrize and optimize over the infinitely many n-particle wave functions comprising the state's Fock-space representation. Here we approach this problem by introducing neural-network quantum field states, a deep learning ansatz that enables application of the variational principle to nonrelativistic quantum field theories in the continuum. Our ansatz uses the Deep Sets neural network architecture to simultaneously parametrize all of the n-particle wave functions comprising a quantum field state. We employ our ansatz to approximate ground states of various field theories, including an inhomogeneous system and a system with long-range interactions, thus demonstrating a powerful new tool for probing quantum field theories.
Collapse
Affiliation(s)
- John M Martyn
- Center for Theoretical Physics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA
- The NSF AI Institute for Artificial Intelligence and Fundamental Interactions
| | - Khadijeh Najafi
- IBM Quantum, IBM T. J. Watson Research Center, Yorktown Heights, New York 10598, USA
- MIT-IBM Watson AI Lab, Cambridge, Massachusetts 02142, USA
| | - Di Luo
- Center for Theoretical Physics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA
- The NSF AI Institute for Artificial Intelligence and Fundamental Interactions
- Department of Physics, Harvard University, Cambridge, Massachusetts 02138, USA
| |
Collapse
|
5
|
Atanasova H, Bernheimer L, Cohen G. Stochastic representation of many-body quantum states. Nat Commun 2023; 14:3601. [PMID: 37328458 DOI: 10.1038/s41467-023-39244-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Accepted: 06/02/2023] [Indexed: 06/18/2023] Open
Abstract
The quantum many-body problem is ultimately a curse of dimensionality: the state of a system with many particles is determined by a function with many dimensions, which rapidly becomes difficult to efficiently store, evaluate and manipulate numerically. On the other hand, modern machine learning models like deep neural networks can express highly correlated functions in extremely large-dimensional spaces, including those describing quantum mechanical problems. We show that if one represents wavefunctions as a stochastically generated set of sample points, the problem of finding ground states can be reduced to one where the most technically challenging step is that of performing regression-a standard supervised learning task. In the stochastic representation the (anti)symmetric property of fermionic/bosonic wavefunction can be used for data augmentation and learned rather than explicitly enforced. We further demonstrate that propagation of an ansatz towards the ground state can then be performed in a more robust and computationally scalable fashion than traditional variational approaches allow.
Collapse
Affiliation(s)
| | - Liam Bernheimer
- School of Chemistry, Tel Aviv University, Tel Aviv, 6997801, Israel
| | - Guy Cohen
- School of Chemistry, Tel Aviv University, Tel Aviv, 6997801, Israel.
- The Raymond and Beverley Sackler Center for Computational Molecular and Materials Science, Tel Aviv University, Tel Aviv, 6997801, Israel.
| |
Collapse
|
6
|
Zhang Z, Liu J, Hu J, Wang Q, Meißner UG. Revealing the nature of hidden charm pentaquarks with machine learning. Sci Bull (Beijing) 2023:S2095-9273(23)00258-X. [PMID: 37147206 DOI: 10.1016/j.scib.2023.04.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2023] [Revised: 03/16/2023] [Accepted: 04/06/2023] [Indexed: 05/07/2023]
Abstract
We study the nature of the hidden charm pentaquarks, i.e., the Pc4312,Pc4440 and Pc(4457), with a neural network approach in pionless effective field theory. In this framework, the normal χ2 fitting approach cannot distinguish the quantum numbers of the Pc(4440) and Pc(4457). In contrast to that, the neural network-based approach can discriminate them, which still cannot be seen as a proof of the spin of the states since pion exchange is not considered in the approach. In addition, we also illustrate the role of each experimental data bin of the invariant J/ψp mass distribution on the underlying physics in both neural network and fitting methods. Their similarities and differences demonstrate that neural network methods can use data information more effectively and directly. This study provides more insights about how the neural network-based approach predicts the nature of exotic states from the mass spectrum.
Collapse
Affiliation(s)
- Zhenyu Zhang
- Guangdong Provincial Key Laboratory of Nuclear Science, Institute of Quantum Matter, South China Normal University, Guangzhou 510006, China; Guangdong-Hong Kong Joint Laboratory of Quantum Matter, Southern Nuclear Science Computing Center, South China Normal University, Guangzhou 510006, China
| | - Jiahao Liu
- Guangdong Provincial Key Laboratory of Nuclear Science, Institute of Quantum Matter, South China Normal University, Guangzhou 510006, China; Guangdong-Hong Kong Joint Laboratory of Quantum Matter, Southern Nuclear Science Computing Center, South China Normal University, Guangzhou 510006, China
| | - Jifeng Hu
- Guangdong Provincial Key Laboratory of Nuclear Science, Institute of Quantum Matter, South China Normal University, Guangzhou 510006, China; Guangdong-Hong Kong Joint Laboratory of Quantum Matter, Southern Nuclear Science Computing Center, South China Normal University, Guangzhou 510006, China.
| | - Qian Wang
- Guangdong Provincial Key Laboratory of Nuclear Science, Institute of Quantum Matter, South China Normal University, Guangzhou 510006, China; Guangdong-Hong Kong Joint Laboratory of Quantum Matter, Southern Nuclear Science Computing Center, South China Normal University, Guangzhou 510006, China.
| | - Ulf-G Meißner
- Helmholtz-Institut für Strahlen- und Kernphysik and Bethe Center for Theoretical Physics, Universität Bonn, Bonn D-53115, Germany; Institute for Advanced Simulation, Institut für Kernphysik and Jülich Center for Hadron Physics, Forschungszentrum Jülich, Jïlich D-52425, Germany; Tbilisi State University, Tbilisi 0186, Georgia.
| |
Collapse
|
7
|
Rigo M, Hall B, Hjorth-Jensen M, Lovato A, Pederiva F. Solving the nuclear pairing model with neural network quantum states. Phys Rev E 2023; 107:025310. [PMID: 36932590 DOI: 10.1103/physreve.107.025310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Accepted: 02/10/2023] [Indexed: 06/18/2023]
Abstract
We present a variational Monte Carlo method that solves the nuclear many-body problem in the occupation number formalism exploiting an artificial neural network representation of the ground-state wave function. A memory-efficient version of the stochastic reconfiguration algorithm is developed to train the network by minimizing the expectation value of the Hamiltonian. We benchmark this approach against widely used nuclear many-body methods by solving a model used to describe pairing in nuclei for different types of interaction and different values of the interaction strength. Despite its polynomial computational cost, our method outperforms coupled-cluster and provides energies that are in excellent agreement with the numerically exact full configuration-interaction values.
Collapse
Affiliation(s)
- Mauro Rigo
- Physics Department, University of Trento, via Sommarive 14, I-38123 Trento, Italy
| | - Benjamin Hall
- Department of Physics and Astronomy and Facility for Rare Isotope Beams, Michigan State University, East Lansing, Michigan 48824, USA
- Physics Division, Argonne National Laboratory, Argonne, Illinois 60439, USA
| | - Morten Hjorth-Jensen
- Department of Physics and Astronomy and Facility for Rare Isotope Beams, Michigan State University, East Lansing, Michigan 48824, USA
- Department of Physics and Center for Computing in Science Education, University of Oslo, N-0316 Oslo, Norway
| | - Alessandro Lovato
- Physics Division, Argonne National Laboratory, Argonne, Illinois 60439, USA
- Computational Science Division, Argonne National Laboratory, Argonne, Illinois 60439, USA
- INFN-TIFPA Trento Institute for Fundamental Physics and Applications, Via Sommarive, 14, 38123 Trento, Italy
| | - Francesco Pederiva
- Physics Department, University of Trento, via Sommarive 14, I-38123 Trento, Italy
- INFN-TIFPA Trento Institute for Fundamental Physics and Applications, Via Sommarive, 14, 38123 Trento, Italy
| |
Collapse
|
8
|
Shmilovich K, Willmott D, Batalov I, Kornbluth M, Mailoa J, Kolter JZ. Orbital Mixer: Using Atomic Orbital Features for Basis-Dependent Prediction of Molecular Wavefunctions. J Chem Theory Comput 2022; 18:6021-6030. [PMID: 36122312 DOI: 10.1021/acs.jctc.2c00555] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
Leveraging ab initio data at scale has enabled the development of machine learning models capable of extremely accurate and fast molecular property prediction. A central paradigm of many previous studies focuses on generating predictions for only a fixed set of properties. Recent lines of research instead aim to explicitly learn the electronic structure via molecular wavefunctions, from which other quantum chemical properties can be directly derived. While previous methods generate predictions as a function of only the atomic configuration, in this work we present an alternate approach that directly purposes basis-dependent information to predict molecular electronic structure. Our model, Orbital Mixer, is composed entirely of multi-layer perceptrons (MLPs) using MLP-Mixer layers within a simple, intuitive, and scalable architecture that achieves competitive Hamiltonian and molecular orbital energy and coefficient prediction accuracies compared to the state-of-the-art.
Collapse
Affiliation(s)
- Kirill Shmilovich
- Pritzker School of Molecular Engineering, University of Chicago, Chicago, Illinois 60637, United States
| | - Devin Willmott
- Bosch Center for Artificial Intelligence, Pittsburgh, Pennsylvania 15222, United States
| | - Ivan Batalov
- Bosch Center for Artificial Intelligence, Pittsburgh, Pennsylvania 15222, United States
| | - Mordechai Kornbluth
- Bosch Research and Technology Center, Cambridge, Massachusetts 02139, United States
| | - Jonathan Mailoa
- Tencent Quantum Laboratory, Shenzhen, Guangdong 518057, China
| | - J Zico Kolter
- Bosch Center for Artificial Intelligence, Pittsburgh, Pennsylvania 15222, United States.,Carnegie Mellon University, Pittsburgh, Pennsylvania 15222, United States
| |
Collapse
|
9
|
Pei MY, Clark SR. Neural-Network Quantum States for Spin-1 Systems: Spin-Basis and Parameterization Effects on Compactness of Representations. ENTROPY (BASEL, SWITZERLAND) 2021; 23:879. [PMID: 34356420 PMCID: PMC8304762 DOI: 10.3390/e23070879] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/21/2021] [Revised: 07/07/2021] [Accepted: 07/07/2021] [Indexed: 11/16/2022]
Abstract
Neural network quantum states (NQS) have been widely applied to spin-1/2 systems, where they have proven to be highly effective. The application to systems with larger on-site dimension, such as spin-1 or bosonic systems, has been explored less and predominantly using spin-1/2 Restricted Boltzmann Machines (RBMs) with a one-hot/unary encoding. Here, we propose a more direct generalization of RBMs for spin-1 that retains the key properties of the standard spin-1/2 RBM, specifically trivial product states representations, labeling freedom for the visible variables and gauge equivalence to the tensor network formulation. To test this new approach, we present variational Monte Carlo (VMC) calculations for the spin-1 anti-ferromagnetic Heisenberg (AFH) model and benchmark it against the one-hot/unary encoded RBM demonstrating that it achieves the same accuracy with substantially fewer variational parameters. Furthermore, we investigate how the hidden unit complexity of NQS depend on the local single-spin basis used. Exploiting the tensor network version of our RBM we construct an analytic NQS representation of the Affleck-Kennedy-Lieb-Tasaki (AKLT) state in the xyz spin-1 basis using only M=2N hidden units, compared to M∼O(N2) required in the Sz basis. Additional VMC calculations provide strong evidence that the AKLT state in fact possesses an exact compact NQS representation in the xyz basis with only M=N hidden units. These insights help to further unravel how to most effectively adapt the NQS framework for more complex quantum systems.
Collapse
Affiliation(s)
- Michael Y. Pei
- H.H. Wills Physics Laboratory, University of Bristol, Bristol BS8 1TL, UK
| | - Stephen R. Clark
- H.H. Wills Physics Laboratory, University of Bristol, Bristol BS8 1TL, UK
| |
Collapse
|
10
|
Schätzle Z, Hermann J, Noé F. Convergence to the fixed-node limit in deep variational Monte Carlo. J Chem Phys 2021; 154:124108. [PMID: 33810658 DOI: 10.1063/5.0032836] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
Variational quantum Monte Carlo (QMC) is an ab initio method for solving the electronic Schrödinger equation that is exact in principle, but limited by the flexibility of the available Ansätze in practice. The recently introduced deep QMC approach, specifically two deep-neural-network Ansätze PauliNet and FermiNet, allows variational QMC to reach the accuracy of diffusion QMC, but little is understood about the convergence behavior of such Ansätze. Here, we analyze how deep variational QMC approaches the fixed-node limit with increasing network size. First, we demonstrate that a deep neural network can overcome the limitations of a small basis set and reach the mean-field (MF) complete-basis-set limit. Moving to electron correlation, we then perform an extensive hyperparameter scan of a deep Jastrow factor for LiH and H4 and find that variational energies at the fixed-node limit can be obtained with a sufficiently large network. Finally, we benchmark MF and many-body Ansätze on H2O, increasing the fraction of recovered fixed-node correlation energy of single-determinant Slater-Jastrow-type Ansätze by half an order of magnitude compared to previous variational QMC results, and demonstrate that a single-determinant Slater-Jastrow-backflow version of the Ansatz overcomes the fixed-node limitations. This analysis helps understand the superb accuracy of deep variational Ansätze in comparison to the traditional trial wavefunctions at the respective level of theory and will guide future improvements of the neural-network architectures in deep QMC.
Collapse
Affiliation(s)
- Z Schätzle
- FU Berlin, Department of Mathematics and Computer Science, Arnimallee 6, 14195 Berlin, Germany
| | - J Hermann
- FU Berlin, Department of Mathematics and Computer Science, Arnimallee 6, 14195 Berlin, Germany
| | - F Noé
- FU Berlin, Department of Mathematics and Computer Science, Arnimallee 6, 14195 Berlin, Germany
| |
Collapse
|