1
|
Asadchev A, Valeev EF. 3-center and 4-center 2-particle Gaussian AO integrals on modern accelerated processors. J Chem Phys 2024; 160:244109. [PMID: 38934632 DOI: 10.1063/5.0217001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2024] [Accepted: 06/04/2024] [Indexed: 06/28/2024] Open
Abstract
We report an implementation of the McMurchie-Davidson (MD) algorithm for 3-center and 4-center 2-particle integrals over Gaussian atomic orbitals (AOs) with low and high angular momenta l and varying degrees of contraction for graphical processing units (GPUs). This work builds upon our recent implementation of a matrix form of the MD algorithm that is efficient for GPU evaluation of 4-center 2-particle integrals over Gaussian AOs of high angular momenta (l ≥ 4) [A. Asadchev and E. F. Valeev, J. Phys. Chem. A 127, 10889-10895 (2023)]. The use of unconventional data layouts and three variants of the MD algorithm allow for the evaluation of integrals with double precision and sustained performance between 25% and 70% of the theoretical hardware peak. Performance assessment includes integrals over AOs with l ≤ 6 (a higher l is supported). Preliminary implementation of the Hartree-Fock exchange operator is presented and assessed for computations with up to a quadruple-zeta basis and more than 20 000 AOs. The corresponding C++ code is part of the experimental open-source LibintX library available at https://github.com/ValeevGroup/libintx.
Collapse
Affiliation(s)
- Andrey Asadchev
- Department of Chemistry, Virginia Tech, Blacksburg, Virginia 24061, USA
| | - Edward F Valeev
- Department of Chemistry, Virginia Tech, Blacksburg, Virginia 24061, USA
| |
Collapse
|
2
|
Tao Y, Giese TJ, Ekesan Ş, Zeng J, Aradi B, Hourahine B, Aktulga HM, Götz AW, Merz KM, York DM. Amber free energy tools: Interoperable software for free energy simulations using generalized quantum mechanical/molecular mechanical and machine learning potentials. J Chem Phys 2024; 160:224104. [PMID: 38856060 DOI: 10.1063/5.0211276] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2024] [Accepted: 05/15/2024] [Indexed: 06/11/2024] Open
Abstract
We report the development and testing of new integrated cyberinfrastructure for performing free energy simulations with generalized hybrid quantum mechanical/molecular mechanical (QM/MM) and machine learning potentials (MLPs) in Amber. The Sander molecular dynamics program has been extended to leverage fast, density-functional tight-binding models implemented in the DFTB+ and xTB packages, and an interface to the DeePMD-kit software enables the use of MLPs. The software is integrated through application program interfaces that circumvent the need to perform "system calls" and enable the incorporation of long-range Ewald electrostatics into the external software's self-consistent field procedure. The infrastructure provides access to QM/MM models that may serve as the foundation for QM/MM-ΔMLP potentials, which supplement the semiempirical QM/MM model with a MLP correction trained to reproduce ab initio QM/MM energies and forces. Efficient optimization of minimum free energy pathways is enabled through a new surface-accelerated finite-temperature string method implemented in the FE-ToolKit package. Furthermore, we interfaced Sander with the i-PI software by implementing the socket communication protocol used in the i-PI client-server model. The new interface with i-PI allows for the treatment of nuclear quantum effects with semiempirical QM/MM-ΔMLP models. The modular interoperable software is demonstrated on proton transfer reactions in guanine-thymine mispairs in a B-form deoxyribonucleic acid helix. The current work represents a considerable advance in the development of modular software for performing free energy simulations of chemical reactions that are important in a wide range of applications.
Collapse
Affiliation(s)
- Yujun Tao
- Laboratory for Biomolecular Simulation Research, Institute for Quantitative Biomedicine and Department of Chemistry and Chemical Biology, Rutgers University, Piscataway, New Jersey 08854, USA
| | - Timothy J Giese
- Laboratory for Biomolecular Simulation Research, Institute for Quantitative Biomedicine and Department of Chemistry and Chemical Biology, Rutgers University, Piscataway, New Jersey 08854, USA
| | - Şölen Ekesan
- Laboratory for Biomolecular Simulation Research, Institute for Quantitative Biomedicine and Department of Chemistry and Chemical Biology, Rutgers University, Piscataway, New Jersey 08854, USA
| | - Jinzhe Zeng
- Laboratory for Biomolecular Simulation Research, Institute for Quantitative Biomedicine and Department of Chemistry and Chemical Biology, Rutgers University, Piscataway, New Jersey 08854, USA
| | - Bálint Aradi
- Bremen Center for Computational Materials Science, University of Bremen, D-28334 Bremen, Germany
| | - Ben Hourahine
- SUPA, Department of Physics, University of Strathclyde, Glasgow G4 0NG, United Kingdom
| | - Hasan Metin Aktulga
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, USA
| | - Andreas W Götz
- San Diego Supercomputer Center, University of California San Diego, La Jolla, California 92093, USA
| | - Kenneth M Merz
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, USA
| | - Darrin M York
- Laboratory for Biomolecular Simulation Research, Institute for Quantitative Biomedicine and Department of Chemistry and Chemical Biology, Rutgers University, Piscataway, New Jersey 08854, USA
| |
Collapse
|
3
|
Hicks CB, Martinez TJ. Massively scalable workflows for quantum chemistry: BigChem and ChemCloud. J Chem Phys 2024; 160:142501. [PMID: 38591672 DOI: 10.1063/5.0190834] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 03/14/2024] [Indexed: 04/10/2024] Open
Abstract
Electronic structure theory, i.e., quantum chemistry, is the fundamental building block for many problems in computational chemistry. We present a new distributed computing framework (BigChem), which allows for an efficient solution of many quantum chemistry problems in parallel. BigChem is designed to be easily composable and leverages industry-standard middleware (e.g., Celery, RabbitMQ, and Redis) for distributed approaches to large scale problems. BigChem can harness any collection of worker nodes, including ones on cloud providers (such as AWS or Azure), local clusters, or supercomputer centers (and any mixture of these). BigChem builds upon MolSSI packages, such as QCEngine to standardize the operation of numerous computational chemistry programs, demonstrated here with Psi4, xtb, geomeTRIC, and TeraChem. BigChem delivers full utilization of compute resources at scale, offers a programable canvas for designing sophisticated quantum chemistry workflows, and is fault tolerant to node failures and network disruptions. We demonstrate linear scalability of BigChem running computational chemistry workloads on up to 125 GPUs. Finally, we present ChemCloud, a web API to BigChem and successor to TeraChem Cloud. ChemCloud delivers scalable and secure access to BigChem over the Internet.
Collapse
Affiliation(s)
- Colton B Hicks
- Department of Chemistry and The PULSE Institute, Stanford University, Stanford, California 94305, USA and SLAC National Accelerator Laboratory, Menlo Park, California 94025, USA
| | - Todd J Martinez
- Department of Chemistry and The PULSE Institute, Stanford University, Stanford, California 94305, USA and SLAC National Accelerator Laboratory, Menlo Park, California 94025, USA
| |
Collapse
|
4
|
Shajan A, Manathunga M, Götz AW, Merz KM. Geometry Optimization: A Comparison of Different Open-Source Geometry Optimizers. J Chem Theory Comput 2023; 19:7533-7541. [PMID: 37870541 DOI: 10.1021/acs.jctc.3c00188] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2023]
Abstract
Based on a series of energy minimizations with starting structures obtained from the Baker test set of 30 organic molecules, a comparison is made between various open-source geometry optimization codes that are interfaced with the open-source QUantum Interaction Computational Kernel (QUICK) program for gradient and energy calculations. The findings demonstrate how the choice of the coordinate system influences the optimization process to reach an equilibrium structure. With fewer steps, internal coordinates outperform Cartesian coordinates, while the choice of the initial Hessian and Hessian update method in quasi-Newton approaches made by different optimization algorithms also contributes to the rate of convergence. Furthermore, an available open-source machine learning method based on Gaussian process regression (GPR) was evaluated for energy minimizations over surrogate potential energy surfaces with both Cartesian and internal coordinates with internal coordinates outperforming Cartesian. Overall, geomeTRIC and DL-FIND with their default optimization method as well as with the GPR-based model using Hartree-Fock theory with the 6-31G** basis set needed a comparable number of geometry optimization steps to the approach of Baker using a unit matrix as the initial Hessian to reach the optimized geometry. On the other hand, the Berny and Sella offerings in ASE outperformed the other algorithms. Based on this, we recommend using the file-based approaches, ASE/Berny and ASE/Sella, for large-scale optimization efforts, while if using a single executable is preferable, we now distribute QUICK integrated with DL-FIND.
Collapse
Affiliation(s)
- Akhil Shajan
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, United States
| | - Madushanka Manathunga
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, United States
- Department of Biochemistry and Molecular Biology, Michigan State University, East Lansing, Michigan 48824, United States
| | - Andreas W Götz
- San Diego Supercomputer Center, University of California San Diego, La Jolla, California 92093-0505, United States
| | - Kenneth M Merz
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, United States
- Department of Biochemistry and Molecular Biology, Michigan State University, East Lansing, Michigan 48824, United States
| |
Collapse
|
5
|
Zahariev F, Xu P, Westheimer BM, Webb S, Galvez Vallejo J, Tiwari A, Sundriyal V, Sosonkina M, Shen J, Schoendorff G, Schlinsog M, Sattasathuchana T, Ruedenberg K, Roskop LB, Rendell AP, Poole D, Piecuch P, Pham BQ, Mironov V, Mato J, Leonard S, Leang SS, Ivanic J, Hayes J, Harville T, Gururangan K, Guidez E, Gerasimov IS, Friedl C, Ferreras KN, Elliott G, Datta D, Cruz DDA, Carrington L, Bertoni C, Barca GMJ, Alkan M, Gordon MS. The General Atomic and Molecular Electronic Structure System (GAMESS): Novel Methods on Novel Architectures. J Chem Theory Comput 2023; 19:7031-7055. [PMID: 37793073 DOI: 10.1021/acs.jctc.3c00379] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/06/2023]
Abstract
The primary focus of GAMESS over the last 5 years has been the development of new high-performance codes that are able to take effective and efficient advantage of the most advanced computer architectures, both CPU and accelerators. These efforts include employing density fitting and fragmentation methods to reduce the high scaling of well-correlated (e.g., coupled-cluster) methods as well as developing novel codes that can take optimal advantage of graphical processing units and other modern accelerators. Because accurate wave functions can be very complex, an important new functionality in GAMESS is the quasi-atomic orbital analysis, an unbiased approach to the understanding of covalent bonds embedded in the wave function. Best practices for the maintenance and distribution of GAMESS are also discussed.
Collapse
Affiliation(s)
- Federico Zahariev
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Peng Xu
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Bryce M Westheimer
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Simon Webb
- VeraChem LLC, 12850 Middlebrook Road, Suite 205, Germantown, Maryland 20874-5244, United States
| | - Jorge Galvez Vallejo
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
- Research School of Computer Science, Australian National University, Canberra, ACT 2601, Australia
| | - Ananta Tiwari
- EP Analytics, Inc., 9909 Mira Mesa Boulevard, Suite 230, San Diego, California 92131, United States
| | - Vaibhav Sundriyal
- Department of Computational Modeling and Simulation Engineering, Old Dominion University, Norfolk, Virginia 23529, United States
| | - Masha Sosonkina
- Department of Computational Modeling and Simulation Engineering, Old Dominion University, Norfolk, Virginia 23529, United States
| | - Jun Shen
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, United States
| | - George Schoendorff
- Propellants Branch, Rocket Propulsion Division, Aerospace Systems Directorate, Air Force Research Laboratory, AFRL/RQRP, Edwards Air Force Base, California 93524, United States
| | - Megan Schlinsog
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Tosaporn Sattasathuchana
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Klaus Ruedenberg
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Luke B Roskop
- Hewlett-Packard Enterprise, 2131 Lindau Lane #1000, Bloomington, Minnesota 55425, United States
| | | | - David Poole
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
- School of Chemistry & Biochemistry, Georgia Institute of Technology, Athens, Georgia 30332, United States
| | - Piotr Piecuch
- Department of Chemistry and Department of Physics and Astronomy, Michigan State University, East Lansing, Michigan 48824, United States
| | - Buu Q Pham
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Vladimir Mironov
- Department of Chemistry, Kyungpook National University, Daegu 41566, South Korea
| | - Joani Mato
- Physical Sciences Division, Pacific Northwest National Laboratory, 902 Battelle Boulevard, P.O. Box 999, MS K1-83, Richland, Washington 99352, United States
| | - Sam Leonard
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Sarom S Leang
- EP Analytics, Inc., 9909 Mira Mesa Boulevard, Suite 230, San Diego, California 92131, United States
| | - Joe Ivanic
- Advanced Biomedical Computational Science, Frederick National Laboratory for Cancer Research, Frederick, Maryland 21702, United States
| | - Jackson Hayes
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Taylor Harville
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Karthik Gururangan
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, United States
| | - Emilie Guidez
- Department of Chemistry, University of Colorado Denver, Denver, Colorado 80217, United States
| | - Igor S Gerasimov
- Department of Chemistry, Kyungpook National University, Daegu 41566, South Korea
| | - Christian Friedl
- Institut für Theoretische Physik, Johannes Kepler Universität Linz, Altenberger Str. 69, 4040 Linz, Austria
| | - Katherine N Ferreras
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - George Elliott
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Dipayan Datta
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Daniel Del Angel Cruz
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Laura Carrington
- EP Analytics, Inc., 9909 Mira Mesa Boulevard, Suite 230, San Diego, California 92131, United States
| | - Colleen Bertoni
- Argonne Leadership Computing Facility, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - Giuseppe M J Barca
- Research School of Computer Science, Australian National University, Canberra, ACT 2601, Australia
| | - Melisa Alkan
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| |
Collapse
|
6
|
Case D, Aktulga HM, Belfon K, Cerutti DS, Cisneros GA, Cruzeiro VD, Forouzesh N, Giese TJ, Götz AW, Gohlke H, Izadi S, Kasavajhala K, Kaymak MC, King E, Kurtzman T, Lee TS, Li P, Liu J, Luchko T, Luo R, Manathunga M, Machado MR, Nguyen HM, O’Hearn KA, Onufriev AV, Pan F, Pantano S, Qi R, Rahnamoun A, Risheh A, Schott-Verdugo S, Shajan A, Swails J, Wang J, Wei H, Wu X, Wu Y, Zhang S, Zhao S, Zhu Q, Cheatham TE, Roe DR, Roitberg A, Simmerling C, York DM, Nagan MC, Merz KM. AmberTools. J Chem Inf Model 2023; 63:6183-6191. [PMID: 37805934 PMCID: PMC10598796 DOI: 10.1021/acs.jcim.3c01153] [Citation(s) in RCA: 144] [Impact Index Per Article: 144.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Indexed: 10/10/2023]
Abstract
AmberTools is a free and open-source collection of programs used to set up, run, and analyze molecular simulations. The newer features contained within AmberTools23 are briefly described in this Application note.
Collapse
Affiliation(s)
- David
A. Case
- Department
of Chemistry and Chemical Biology, Rutgers
University, Piscataway 08854, New Jersey, United States
| | - Hasan Metin Aktulga
- Department
of Computer Science and Engineering, Michigan
State University, East Lansing 48824-1322, Michigan, United States
| | - Kellon Belfon
- FOG
Pharmaceuticals Inc., Cambridge 02140, Massachusetts, United States
| | - David S. Cerutti
- Psivant, 451 D Street, Suite 205, Boston 02210, Massachusetts, United States
| | - G. Andrés Cisneros
- Department
of Physics, Department of Chemistry and Biochemistry, University of Texas at Dallas, Richardson 75801, Texas, United States
| | - Vinícius
Wilian D. Cruzeiro
- Department
of Chemistry and The PULSE Institute, Stanford
University, Stanford 94305, California, United States
| | - Negin Forouzesh
- Department
of Computer Science, California State University, Los Angeles 90032, California, United States
| | - Timothy J. Giese
- Laboratory
for Biomolecular Simulation Research, Institute for Quantitative Biomedicine
and Department of Chemistry and Chemical Biology, Rutgers University, Piscataway 08854, New Jersey, United States
| | - Andreas W. Götz
- San
Diego Supercomputer Center, University of
California San Diego, La Jolla 92093-0505, California, United States
| | - Holger Gohlke
- Institute
for Pharmaceutical and Medicinal Chemistry, Heinrich Heine University Düsseldorf, Düsseldorf 40225, Germany
- Institute
of Bio- and Geosciences (IBG-4: Bioinformatics), Forschungszentrum Jülich GmbH, Jülich 52425, Germany
| | - Saeed Izadi
- Pharmaceutical
Development, Genentech, Inc., South San Francisco 94080, California, United
States
| | - Koushik Kasavajhala
- Laufer
Center for Physical and Quantitative Biology, Department of Chemistry, Stony Brook University, Stony Brook 11794, New York, United States
| | - Mehmet C. Kaymak
- Department
of Computer Science and Engineering, Michigan
State University, East Lansing 48824-1322, Michigan, United States
| | - Edward King
- Departments
of Molecular Biology and Biochemistry, Chemical and Biomolecular Engineering,
Materials Science and Engineering, and Biomedical Engineering, Graduate
Program in Chemical and Materials Physics, University of California, Irvine 92697, California, United States
| | - Tom Kurtzman
- Ph.D.
Programs in Chemistry, Biochemistry, and Biology, The Graduate Center of the City University of New York, 365 Fifth Avenue, New York 10016, New York, United States
- Department
of Chemistry, Lehman College, 250 Bedford Park Blvd West, Bronx 10468, New York, United States
| | - Tai-Sung Lee
- Laboratory
for Biomolecular Simulation Research, Institute for Quantitative Biomedicine
and Department of Chemistry and Chemical Biology, Rutgers University, Piscataway 08854, New Jersey, United States
| | - Pengfei Li
- Department
of Chemistry and Biochemistry, Loyola University
Chicago, Chicago 60660, Illinois, United States
| | - Jian Liu
- Beijing
National Laboratory for Molecular Sciences, Institute of Theoretical
and Computational Chemistry, College of Chemistry and Molecular Engineering, Peking University, Beijing 100871, China
| | - Tyler Luchko
- Department
of Physics and Astronomy, California State
University, Northridge, Northridge 91330, California, United States
| | - Ray Luo
- Departments
of Molecular Biology and Biochemistry, Chemical and Biomolecular Engineering,
Materials Science and Engineering, and Biomedical Engineering, Graduate
Program in Chemical and Materials Physics, University of California, Irvine 92697, California, United States
| | - Madushanka Manathunga
- Department
of Chemistry and Department of Biochemistry and Molecular Biology, Michigan State University, East Lansing 48824-1322, Michigan, United States
| | | | - Hai Minh Nguyen
- Department
of Chemistry and Chemical Biology, Rutgers
University, Piscataway 08854, New Jersey, United States
| | - Kurt A. O’Hearn
- Department
of Computer Science and Engineering, Michigan
State University, East Lansing 48824-1322, Michigan, United States
| | - Alexey V. Onufriev
- Departments
of Computer Science and Physics, Virginia
Tech, Blacksburg 24061, Virginia, United
States
| | - Feng Pan
- Department
of Statistics, Florida State University, Tallahassee 32304, Florida, United States
| | - Sergio Pantano
- Institut Pasteur de Montevideo, Montevideo 11400, Uruguay
| | - Ruxi Qi
- Cryo-EM
Center, Southern University of Science and
Technology, Shenzhen 518055, China
| | - Ali Rahnamoun
- Department
of Computer Science and Engineering, Michigan
State University, East Lansing 48824-1322, Michigan, United States
| | - Ali Risheh
- Department
of Computer Science, California State University, Los Angeles 90032, California, United States
| | - Stephan Schott-Verdugo
- Institute
of Bio- and Geosciences (IBG-4: Bioinformatics), Forschungszentrum Jülich GmbH, Jülich 52425, Germany
| | - Akhil Shajan
- Department
of Chemistry and Department of Biochemistry and Molecular Biology, Michigan State University, East Lansing 48824-1322, Michigan, United States
| | - Jason Swails
- Entos, 4470 W Sunset
Blvd, Suite 107, Los Angeles 90027, California, United States
| | - Junmei Wang
- Department
of Pharmaceutical Sciences, School of Pharmacy, University of Pittsburgh, Pittsburgh 15261, Pennsylvania, United States
| | - Haixin Wei
- Departments
of Molecular Biology and Biochemistry, Chemical and Biomolecular Engineering,
Materials Science and Engineering, and Biomedical Engineering, Graduate
Program in Chemical and Materials Physics, University of California, Irvine 92697, California, United States
| | - Xiongwu Wu
- Laboratory
of Computational Biology, NHLBI, NIH, Bethesda 20892, Maryland, United States
| | - Yongxian Wu
- Departments
of Molecular Biology and Biochemistry, Chemical and Biomolecular Engineering,
Materials Science and Engineering, and Biomedical Engineering, Graduate
Program in Chemical and Materials Physics, University of California, Irvine 92697, California, United States
| | - Shi Zhang
- Laboratory
for Biomolecular Simulation Research, Institute for Quantitative Biomedicine
and Department of Chemistry and Chemical Biology, Rutgers University, Piscataway 08854, New Jersey, United States
| | - Shiji Zhao
- Departments
of Molecular Biology and Biochemistry, Chemical and Biomolecular Engineering,
Materials Science and Engineering, and Biomedical Engineering, Graduate
Program in Chemical and Materials Physics, University of California, Irvine 92697, California, United States
- Nurix Therapeutics, Inc., San Francisco 94158, California, United States
| | - Qiang Zhu
- Departments
of Molecular Biology and Biochemistry, Chemical and Biomolecular Engineering,
Materials Science and Engineering, and Biomedical Engineering, Graduate
Program in Chemical and Materials Physics, University of California, Irvine 92697, California, United States
| | - Thomas E. Cheatham
- Department
of Medicinal Chemistry, The University of
Utah, 30 South 2000 East, Salt Lake City 84112, Utah, United
States
| | - Daniel R. Roe
- Laboratory
of Computational Biology, National Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda 20892, Maryland, United States
| | - Adrian Roitberg
- Department
of Chemistry, The University of Florida, 440 Leigh Hall, Gainesville 32611-7200, Florida, United States
| | - Carlos Simmerling
- Laufer
Center for Physical and Quantitative Biology, Department of Chemistry, Stony Brook University, Stony Brook 11794, New York, United States
| | - Darrin M. York
- Laboratory
for Biomolecular Simulation Research, Institute for Quantitative Biomedicine
and Department of Chemistry and Chemical Biology, Rutgers University, Piscataway 08854, New Jersey, United States
| | - Maria C. Nagan
- Department
of Chemistry, Stony Brook University, Stony Brook 11794, New York, United States
| | - Kenneth M. Merz
- Department
of Chemistry and Department of Biochemistry and Molecular Biology, Michigan State University, East Lansing 48824-1322, Michigan, United States
| |
Collapse
|
7
|
Galvez Vallejo JL, Snowdon C, Stocks R, Kazemian F, Yan Yu FC, Seidl C, Seeger Z, Alkan M, Poole D, Westheimer BM, Basha M, De La Pierre M, Rendell A, Izgorodina EI, Gordon MS, Barca GMJ. Toward an extreme-scale electronic structure system. J Chem Phys 2023; 159:044112. [PMID: 37497819 DOI: 10.1063/5.0156399] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Accepted: 07/03/2023] [Indexed: 07/28/2023] Open
Abstract
Electronic structure calculations have the potential to predict key matter transformations for applications of strategic technological importance, from drug discovery to material science and catalysis. However, a predictive physicochemical characterization of these processes often requires accurate quantum chemical modeling of complex molecular systems with hundreds to thousands of atoms. Due to the computationally demanding nature of electronic structure calculations and the complexity of modern high-performance computing hardware, quantum chemistry software has historically failed to operate at such large molecular scales with accuracy and speed that are useful in practice. In this paper, novel algorithms and software are presented that enable extreme-scale quantum chemistry capabilities with particular emphasis on exascale calculations. This includes the development and application of the multi-Graphics Processing Unit (GPU) library LibCChem 2.0 as part of the General Atomic and Molecular Electronic Structure System package and of the standalone Extreme-scale Electronic Structure System (EXESS), designed from the ground up for scaling on thousands of GPUs to perform high-performance accurate quantum chemistry calculations at unprecedented speed and molecular scales. Among various results, we report that the EXESS implementation enables Hartree-Fock/cc-pVDZ plus RI-MP2/cc-pVDZ/cc-pVDZ-RIFIT calculations on an ionic liquid system with 623 016 electrons and 146 592 atoms in less than 45 min using 27 600 GPUs on the Summit supercomputer with a 94.6% parallel efficiency.
Collapse
Affiliation(s)
| | - Calum Snowdon
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Ryan Stocks
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Fazeleh Kazemian
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Fiona Chuo Yan Yu
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Christopher Seidl
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Zoe Seeger
- School of Chemistry, Monash University, Clayton 3800, VIC, Australia
| | - Melisa Alkan
- Department of Chemistry, Iowa State University, Ames, Iowa 50011-3111, USA
| | - David Poole
- School of Chemistry and Biochemistry, Georgia Institute of Technology, Atlanta, Georgia 30332, USA
| | - Bryce M Westheimer
- Department of Chemistry, Iowa State University, Ames, Iowa 50011-3111, USA
| | - Mehaboob Basha
- Pawsey Supercomputing Research Centre, Kensington, WA 6151, Australia
| | | | - Alistair Rendell
- College of Science and Engineering, Flinders University, Adelaide, SA 5042, Australia
| | | | | | - Giuseppe M J Barca
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| |
Collapse
|
8
|
Williams-Young DB, Asadchev A, Popovici DT, Clark D, Waldrop J, Windus TL, Valeev EF, de Jong WA. Distributed memory, GPU accelerated Fock construction for hybrid, Gaussian basis density functional theory. J Chem Phys 2023; 158:234104. [PMID: 37326157 DOI: 10.1063/5.0151070] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2023] [Accepted: 05/26/2023] [Indexed: 06/17/2023] Open
Abstract
With the growing reliance of modern supercomputers on accelerator-based architecture such a graphics processing units (GPUs), the development and optimization of electronic structure methods to exploit these massively parallel resources has become a recent priority. While significant strides have been made in the development GPU accelerated, distributed memory algorithms for many modern electronic structure methods, the primary focus of GPU development for Gaussian basis atomic orbital methods has been for shared memory systems with only a handful of examples pursing massive parallelism. In the present work, we present a set of distributed memory algorithms for the evaluation of the Coulomb and exact exchange matrices for hybrid Kohn-Sham DFT with Gaussian basis sets via direct density-fitted (DF-J-Engine) and seminumerical (sn-K) methods, respectively. The absolute performance and strong scalability of the developed methods are demonstrated on systems ranging from a few hundred to over one thousand atoms using up to 128 NVIDIA A100 GPUs on the Perlmutter supercomputer.
Collapse
Affiliation(s)
- David B Williams-Young
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA
| | - Andrey Asadchev
- Department of Chemistry, Virginia Tech, Blacksburg, Virginia 24061, USA
| | - Doru Thom Popovici
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA
| | - David Clark
- NVIDIA Corporation, Santa Clara, California 95051, USA
| | - Jonathan Waldrop
- Chemical and Biological Sciences Division, Ames National Laboratory, Ames, Iowa 50011, USA
| | - Theresa L Windus
- Chemical and Biological Sciences Division, Ames National Laboratory, Ames, Iowa 50011, USA
- Department of Chemistry, Iowa State University, Ames, Iowa 50011, USA
| | - Edward F Valeev
- Department of Chemistry, Virginia Tech, Blacksburg, Virginia 24061, USA
| | - Wibe A de Jong
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA
| |
Collapse
|
9
|
Woo J, Kim S, Kim WY. Dynamic Precision Approach for Accelerating Large-Scale Eigenvalue Solvers in Electronic Structure Calculations on Graphics Processing Units. J Chem Theory Comput 2023; 19:1457-1465. [PMID: 36812094 DOI: 10.1021/acs.jctc.2c00983] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/24/2023]
Abstract
Single precision (SP) arithmetic can be greatly accelerated as compared to double precision (DP) arithmetic on graphics processing units (GPUs). However, the use of SP in the whole process of electronic structure calculations is inappropriate for the required accuracy. We propose a 3-fold dynamic precision approach for accelerated calculations but still with the accuracy of DP. Here, SP, DP, and mixed precision are dynamically switched during an iterative diagonalization process. We applied this approach to the locally optimal block preconditioned conjugate gradient method to accelerate a large-scale eigenvalue solver for the Kohn-Sham equation. We determined a proper threshold for switching each precision scheme by examining the convergence pattern on the eigenvalue solver only with the kinetic energy operator of the Kohn-Sham Hamiltonian. As a result, we achieved up to 8.53× and 6.60× speedups for band structure and self-consistent field calculations, respectively, for test systems under various boundary conditions on NVIDIA GPUs.
Collapse
Affiliation(s)
- Jeheon Woo
- Department of Chemistry, KAIST, 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| | - Seonghwan Kim
- Department of Chemistry, KAIST, 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| | - Woo Youn Kim
- Department of Chemistry, KAIST, 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| |
Collapse
|
10
|
Manathunga M, Aktulga HM, Götz AW, Merz KM. Quantum Mechanics/Molecular Mechanics Simulations on NVIDIA and AMD Graphics Processing Units. J Chem Inf Model 2023; 63:711-717. [PMID: 36720086 DOI: 10.1021/acs.jcim.2c01505] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
Abstract
We have ported and optimized the graphics processing unit (GPU)-accelerated QUICK and AMBER-based ab initio quantum mechanics/molecular mechanics (QM/MM) implementation on AMD GPUs. This encompasses the entire Fock matrix build and force calculation in QUICK including one-electron integrals, two-electron repulsion integrals, exchange-correlation quadrature, and linear algebra operations. General performance improvements to the QUICK GPU code are also presented. Benchmarks carried out on NVIDIA V100 and AMD MI100 cards display similar performance on both hardware for standalone HF/DFT calculations with QUICK and QM/MM molecular dynamics simulations with QUICK/AMBER. Furthermore, with respect to the QUICK/AMBER release version 21, significant speedups are observed for QM/MM molecular dynamics simulations. This significantly increases the range of scientific problems that can be addressed with open-source QM/MM software on state-of-the-art computer hardware.
Collapse
Affiliation(s)
- Madushanka Manathunga
- Department of Chemistry and Department of Biochemistry and Molecular Biology, Michigan State University, East Lansing, Michigan48824-1322, United States
| | - Hasan Metin Aktulga
- Department of Computer Science and Engineering, Michigan State University, East Lansing, Michigan48824-1322, United States
| | - Andreas W Götz
- San Diego Supercomputer Center, University of California San Diego, La Jolla, California92093-0505, United States
| | - Kenneth M Merz
- Department of Chemistry and Department of Biochemistry and Molecular Biology, Michigan State University, East Lansing, Michigan48824-1322, United States
| |
Collapse
|
11
|
Johnson KG, Mirchandaney S, Hoag E, Heirich A, Aiken A, Martínez TJ. Multinode Multi-GPU Two-Electron Integrals: Code Generation Using the Regent Language. J Chem Theory Comput 2022; 18:6522-6536. [PMID: 36200649 DOI: 10.1021/acs.jctc.2c00414] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
The computation of two-electron repulsion integrals (ERIs) is often the most expensive step of integral-direct self-consistent field methods. Formally it scales as O(N4), where N is the number of Gaussian basis functions used to represent the molecular wave function. In practice, this scaling can be reduced to O(N2) or less by neglecting small integrals with screening methods. The contributions of the ERIs to the Fock matrix are of Coulomb (J) and exchange (K) type and require separate algorithms to compute matrix elements efficiently. We previously implemented highly efficient GPU-accelerated J-matrix and K-matrix algorithms in the electronic structure code TeraChem. Although these implementations supported the use of multiple GPUs on a node, they did not support the use of multiple nodes. This presents a key bottleneck to cutting-edge ab initio simulations of large systems, e.g., excited state dynamics of photoactive proteins. We present our implementation of multinode multi-GPU J- and K-matrix algorithms in TeraChem using the Regent programming language. Regent directly supports distributed computation in a task-based model and can generate code for a variety of architectures, including NVIDIA GPUs. We demonstrate multinode scaling up to 45 GPUs (3 nodes) and benchmark against hand-coded TeraChem integral code. We also outline our metaprogrammed Regent implementation, which enables flexible code generation for integrals of different angular momenta.
Collapse
Affiliation(s)
- K Grace Johnson
- Department of Chemistry and The PULSE Institute, Stanford University, Stanford, California94305, United States.,SLAC National Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park, California94025, United States
| | - Seema Mirchandaney
- SLAC National Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park, California94025, United States
| | - Ellis Hoag
- Department of Computer Science, Stanford University, Stanford, California94305, United States
| | - Alan Heirich
- SLAC National Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park, California94025, United States
| | - Alex Aiken
- SLAC National Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park, California94025, United States.,Department of Computer Science, Stanford University, Stanford, California94305, United States
| | - Todd J Martínez
- Department of Chemistry and The PULSE Institute, Stanford University, Stanford, California94305, United States.,SLAC National Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park, California94025, United States
| |
Collapse
|
12
|
Galvez Vallejo JL, Barca GM, Gordon MS. High-performance GPU-accelerated evaluation of electron repulsion integrals. Mol Phys 2022. [DOI: 10.1080/00268976.2022.2112987] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/15/2022]
Affiliation(s)
| | - Giuseppe M.J. Barca
- Department of Computer Science, Australian National University, Canberra, Australia
| | - Mark S. Gordon
- Department of Chemistry, Iowa State University, Ames, United States
| |
Collapse
|
13
|
Manathunga M, Götz AW, Merz KM. Computer-aided drug design, quantum-mechanical methods for biological problems. Curr Opin Struct Biol 2022; 75:102417. [PMID: 35779437 DOI: 10.1016/j.sbi.2022.102417] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Revised: 05/14/2022] [Accepted: 05/16/2022] [Indexed: 11/28/2022]
Abstract
Quantum chemistry enables to study systems with chemical accuracy (<1 kcal/mol from experiment) but is restricted to a handful of atoms due to its computational expense. This has led to ongoing interest to optimize and simplify these methods while retaining accuracy. Implementing quantum mechanical (QM) methods on modern hardware such as multiple-GPUs is one example of how the field is optimizing performance. Multiscale approaches like the so-called QM/molecular mechanical method are gaining popularity in drug discovery because they focus the application of QM methods on the region of choice (e.g., the binding site), while using efficient MM models to represent less relevant areas. The creation of simplified QM methods is another example, including the use of machine learning to create ultra-fast and accurate QM models. Herein, we summarize recent advancements in the development of optimized QM methods that enhance our ability to use these methods in computer aided drug discovery.
Collapse
Affiliation(s)
- Madushanka Manathunga
- Department of Chemistry and Department of Biochemistry and Molecular Biology, Michigan State University, East Lansing, MI 48824, United States. https://twitter.com/@MaduManathunga
| | - Andreas W Götz
- San Diego Supercomputer Center, University of California San Diego, La Jolla, CA 92093, United States. https://twitter.com/@awgoetz
| | - Kenneth M Merz
- Department of Chemistry and Department of Biochemistry and Molecular Biology, Michigan State University, East Lansing, MI 48824, United States.
| |
Collapse
|
14
|
Seidl C, Barca GMJ. Q-Next: A Fast, Parallel, and Diagonalization-Free Alternative to Direct Inversion of the Iterative Subspace. J Chem Theory Comput 2022; 18:4164-4176. [PMID: 35748512 DOI: 10.1021/acs.jctc.2c00073] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
As computer systems dedicated to scientific calculations become massively parallel, the poor parallel performance of the Fock matrix diagonalization becomes a major impediment to achieving larger molecular sizes in self-consistent field (SCF) calculations. In this Article, a novel, highly parallel, and diagonalization-free algorithm for the accelerated convergence of the SCF procedure is presented. The algorithm, called Q-Next, draws on the second-order SCF, quadratically convergent SCF, and direct inversion of the iterative subspace (DIIS) approaches to enable fast convergence while replacing the Fock matrix diagonalization SCF bottleneck with higher parallel efficiency matrix multiplications. Performance results on both parallel multicore CPU and GPU hardware for a variety of test molecules and basis sets are presented, showing that Q-Next achieves a convergence rate comparable to the DIIS method while being, on average, one order of magnitude faster.
Collapse
Affiliation(s)
- Christopher Seidl
- School of Computing, The Australian National University, Canberra, Australian Capital Territory 2601, Australia
| | - Giuseppe M J Barca
- School of Computing, The Australian National University, Canberra, Australian Capital Territory 2601, Australia
| |
Collapse
|
15
|
Barca GMJ, Alkan M, Galvez-Vallejo JL, Poole DL, Rendell AP, Gordon MS. Faster Self-Consistent Field (SCF) Calculations on GPU Clusters. J Chem Theory Comput 2021; 17:7486-7503. [PMID: 34780186 DOI: 10.1021/acs.jctc.1c00720] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
A novel implementation of the self-consistent field (SCF) procedure specifically designed for high-performance execution on multiple graphics processing units (GPUs) is presented. The algorithm offloads to GPUs the three major computational stages of the SCF, namely, the calculation of one-electron integrals, the calculation and digestion of electron repulsion integrals, and the diagonalization of the Fock matrix, including SCF acceleration via DIIS. Performance results for a variety of test molecules and basis sets show remarkable speedups with respect to the state-of-the-art parallel GAMESS CPU code and relative to other widely used GPU codes for both single and multi-GPU execution. The new code outperforms all existing multi-GPU implementations when using eight V100 GPUs, with speedups relative to Terachem ranging from 1.2× to 3.3× and speedups of up to 28× over QUICK on one GPU and 15× using eight GPUs. Strong scaling calculations show nearly ideal scalability up to 8 GPUs while retaining high parallel efficiency for up to 18 GPUs.
Collapse
Affiliation(s)
- Giuseppe M J Barca
- School of Computing, Australian National University, Canberra, ACT 2601, Australia
| | - Melisa Alkan
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, United States
| | - Jorge L Galvez-Vallejo
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, United States
| | - David L Poole
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, United States
| | - Alistair P Rendell
- College of Science and Engineering, Flinders University, Adelaide, SA 5042, Australia
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, United States
| |
Collapse
|