1
|
Pham BQ, Carrington L, Tiwari A, Leang SS, Alkan M, Bertoni C, Datta D, Sattasathuchana T, Xu P, Gordon MS. Porting fragmentation methods to GPUs using an OpenMP API: Offloading the resolution-of-the-identity second-order Møller-Plesset perturbation method. J Chem Phys 2023; 158:2887208. [PMID: 37114705 DOI: 10.1063/5.0143424] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Accepted: 04/10/2023] [Indexed: 04/29/2023] Open
Abstract
Using an OpenMP Application Programming Interface, the resolution-of-the-identity second-order Møller-Plesset perturbation (RI-MP2) method has been off-loaded onto graphical processing units (GPUs), both as a standalone method in the GAMESS electronic structure program and as an electron correlation energy component in the effective fragment molecular orbital (EFMO) framework. First, a new scheme has been proposed to maximize data digestion on GPUs that subsequently linearizes data transfer from central processing units (CPUs) to GPUs. Second, the GAMESS Fortran code has been interfaced with GPU numerical libraries (e.g., NVIDIA cuBLAS and cuSOLVER) for efficient matrix operations (e.g., matrix multiplication, matrix decomposition, and matrix inversion). The standalone GPU RI-MP2 code shows an increasing speedup of up to 7.5× using one NVIDIA V100 GPU with one IBM 42-core P9 CPU for calculations on fullerenes of increasing size from 40 to 260 carbon atoms using the 6-31G(d)/cc-pVDZ-RI basis sets. A single Summit node with six V100s can compute the RI-MP2 correlation energy of a cluster of 175 water molecules using the correlation consistent basis sets cc-pVDZ/cc-pVDZ-RI containing 4375 atomic orbitals and 14 700 auxiliary basis functions in ∼0.85 h. In the EFMO framework, the GPU RI-MP2 component shows near linear scaling for a large number of V100s when computing the energy of an 1800-atom mesoporous silica nanoparticle in a bath of 4000 water molecules. The parallel efficiencies of the GPU RI-MP2 component with 2304 and 4608 V100s are 98.0% and 96.1%, respectively.
Collapse
Affiliation(s)
- Buu Q Pham
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | | | | | | | - Melisa Alkan
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Colleen Bertoni
- Leadership Computing Facility, Argonne National Laboratory, Lemont, Illinois 60439, USA
| | - Dipayan Datta
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | | | - Peng Xu
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| |
Collapse
|
2
|
Pham BQ, Alkan M, Gordon MS. Porting Fragmentation Methods to Graphical Processing Units Using an OpenMP Application Programming Interface: Offloading the Fock Build for Low Angular Momentum Functions. J Chem Theory Comput 2023; 19:2213-2221. [PMID: 37011288 DOI: 10.1021/acs.jctc.2c01137] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/05/2023]
Abstract
A framework to offload four-index two-electron repulsion integrals to graphical processing units (GPUs) using OpenMP is discussed. The method has been applied to the Fock build for low angular momentum s and p functions in both the restricted Hartree-Fock (RHF) and in the effective fragment molecular orbital (EFMO) framework. Benchmark calculations for the GPU code for the pure RHF method show an increasing speedup relative to the existing OpenMP CPU code in GAMESS from 1.04 to 52× for clusters of 70-569 water molecules. The parallel efficiency on 24 NVIDIA V100 GPU boards also increases when increasing the system size: from 75 to 94% for water clusters that contain 303-1120 molecules. In the EFMO framework, the GPU Fock build shows a high linear scalability up to 4608 V100s with a parallel efficiency of 96% for calculations on a solvated mesoporous silica nanoparticle system with ∼67,000 basis functions.
Collapse
Affiliation(s)
- Buu Q Pham
- Department of Chemistry, Iowa State University, Ames, Iowa 50011, United States
| | - Melisa Alkan
- Department of Chemistry, Iowa State University, Ames, Iowa 50011, United States
| | - Mark S Gordon
- Department of Chemistry, Iowa State University, Ames, Iowa 50011, United States
| |
Collapse
|
3
|
Pham BQ, Gordon MS. Development of the FMO/RI-MP2 Fully Analytic Gradient Using a Hybrid-Distributed/Shared Memory Programming Model. J Chem Theory Comput 2020; 16:1039-1054. [PMID: 31899632 DOI: 10.1021/acs.jctc.9b01082] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
Abstract
The fully analytic gradient of the second-order Møller-Plesset perturbation theory (MP2) with the resolution-of-the-identity (RI) approximation in the fragment molecular orbital (FMO) framework is derived and implemented using a hybrid multilevel parallel programming model, a combination of the general distributed data interface (GDDI) and the OpenMP API. The FMO/MP2 analytic gradient contains three parts, i.e., the internal fragment component, the electrostatic potential (ESP) component, and the response terms. The RI approximation is applied to the internal fragment MP2 gradient term, whose MP2 densities and monomer MP2 Lagrangians are shared with the ESP and the response terms. The FMO/RI-MP2 analytic gradient implementation is validated against the numerical gradient (with errors ∼10-6-10-5 Hartree/Bohr) and the energy conservation in molecular dynamics (MD) simulations using NVE ensembles. The RI approximation introduces an error of ∼10-5 Hartree/Bohr with a speedup of 4.0-8.0× compared with the currently available GDDI FMO/MP2 gradient. The node linear scaling of the fragmentation framework due to multilevel parallelism is well-preserved and is demonstrated in single-point gradient calculations of large water clusters (e.g., 1120 and 2165 molecules) using 300-800 KNL compute nodes with a parallel efficiency of more than 90%.
Collapse
Affiliation(s)
- Buu Q Pham
- Department of Chemistry and Ames Laboratory , Iowa State University , Ames , Iowa 50011 , United States
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory , Iowa State University , Ames , Iowa 50011 , United States
| |
Collapse
|
4
|
Fedorov DG. The fragment molecular orbital method: theoretical development, implementation in
GAMESS
, and applications. WILEY INTERDISCIPLINARY REVIEWS-COMPUTATIONAL MOLECULAR SCIENCE 2017. [DOI: 10.1002/wcms.1322] [Citation(s) in RCA: 71] [Impact Index Per Article: 10.1] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Affiliation(s)
- Dmitri G. Fedorov
- Research Center for Computational Design of Advanced Functional Materials (CD‐FMat)National Institute of Advanced Industrial Science and Technology (AIST)TsukubaJapan
| |
Collapse
|
5
|
Pruitt SR, Steinmann C. Mapping Interaction Energies in Chorismate Mutase with the Fragment Molecular Orbital Method. J Phys Chem A 2017; 121:1797-1807. [DOI: 10.1021/acs.jpca.6b12830] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Affiliation(s)
- Spencer R. Pruitt
- Academic & Research Computing, Worcester Polytechnic Institute, Worcester, Massachusetts 01602, United States
| | - Casper Steinmann
- Centre
for Computational Chemistry, School of Chemistry, University of Bristol, Bristol BS8 1TS, United Kingdom
| |
Collapse
|
6
|
Nakata H, Fedorov DG. Efficient Geometry Optimization of Large Molecular Systems in Solution Using the Fragment Molecular Orbital Method. J Phys Chem A 2016; 120:9794-9804. [DOI: 10.1021/acs.jpca.6b09743] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Affiliation(s)
- Hiroya Nakata
- Department
of Fundamental Technology Research, R and D Center Kagoshima, Kyocera, 1-4 Kokubu Yamashita-cho, Kirishima-shi, Kagoshima 899-4312, Japan
| | - Dmitri G. Fedorov
- Research
Center for Computational Design of Advanced Functional Materials, National Institute of Advanced Industrial Science and Technology, 1-1-1
Umezono, Tsukuba, Ibaraki 305-8568, Japan
| |
Collapse
|
7
|
Nakata H, Fedorov DG, Nagata T, Kitaura K, Nakamura S. Simulations of Chemical Reactions with the Frozen Domain Formulation of the Fragment Molecular Orbital Method. J Chem Theory Comput 2015; 11:3053-64. [DOI: 10.1021/acs.jctc.5b00277] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Affiliation(s)
- Hiroya Nakata
- Department
of Biomolecular Engineering, Tokyo Institute of Technology, 4259 Nagatsuta-cho, Midori-ku, Yokohama, Kanagawa 226-8501, Japan
- Research
Cluster for Innovation, RIKEN, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan
- Japan Society for the Promotion of Science, Kojimachi Business Center Building, 5-3-1 Kojimachi, Chiyoda-ku, Tokyo 102-0083, Japan
| | - Dmitri G. Fedorov
- Nanosystem
Research Institute, National Institute of Advanced Industrial Science and Technology, 1-1-1 Umezono, Tsukuba, Ibaraki 305-8568, Japan
| | - Takeshi Nagata
- Nanosystem
Research Institute, National Institute of Advanced Industrial Science and Technology, 1-1-1 Umezono, Tsukuba, Ibaraki 305-8568, Japan
- Graduate
School of System Informatics, Kobe University, 1-1 Rokkodai-cho, Nada-ku, Kobe 657-8501, Japan
| | - Kazuo Kitaura
- Graduate
School of System Informatics, Kobe University, 1-1 Rokkodai-cho, Nada-ku, Kobe 657-8501, Japan
| | - Shinichiro Nakamura
- Research
Cluster for Innovation, RIKEN, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan
| |
Collapse
|
8
|
Fedorov DG, Asada N, Nakanishi I, Kitaura K. The use of many-body expansions and geometry optimizations in fragment-based methods. Acc Chem Res 2014; 47:2846-56. [PMID: 25144610 DOI: 10.1021/ar500224r] [Citation(s) in RCA: 52] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Conspectus Chemists routinely work with complex molecular systems: solutions, biochemical molecules, and amorphous and composite materials provide some typical examples. The questions one often asks are what are the driving forces for a chemical phenomenon? How reasonable are our views of chemical systems in terms of subunits, such as functional groups and individual molecules? How can one quantify the difference in physicochemical properties of functional units found in a different chemical environment? Are various effects on functional units in molecular systems additive? Can they be represented by pairwise potentials? Are there effects that cannot be represented in a simple picture of pairwise interactions? How can we obtain quantitative values for these effects? Many of these questions can be formulated in the language of many-body effects. They quantify the properties of subunits (fragments), referred to as one-body properties, pairwise interactions (two-body properties), couplings of two-body interactions described by three-body properties, and so on. By introducing the notion of fragments in the framework of quantum chemistry, one obtains two immense benefits: (a) chemists can finally relate to quantum chemistry, which now speaks their language, by discussing chemically interesting subunits and their interactions and (b) calculations become much faster due to a reduced computational scaling. For instance, the somewhat academic sounding question of the importance of three-body effects in water clusters is actually another way of asking how two hydrogen bonds affect each other, when they involve three water molecules. One aspect of this is the many-body charge transfer (CT), because the charge transfers in the two hydrogen bonds are coupled to each other (not independent). In this work, we provide a generalized view on the use of many-body expansions in fragment-based methods, focusing on the general aspects of the property expansion and a contraction of a many-body expansion in a formally two-body series, as exemplified in the development of the fragment molecular orbital (FMO) method. Fragment-based methods have been very successful in delivering the properties of fragments, as well as the fragment interactions, providing insights into complex chemical processes in large molecular systems. We briefly review geometry optimizations performed with fragment-based methods and present an efficient geometry optimization method based on the combination of FMO with molecular mechanics (MM), applied to the complex of a subunit of protein kinase 2 (CK2) with a ligand. FMO results are discussed in comparison with experimental and MM-optimized structures.
Collapse
Affiliation(s)
- Dmitri G. Fedorov
- NRI, National Institute of Advanced Industrial Science and Technology (AIST), Central 2, Umezono 1-1-1, Tsukuba, 305-8568, Japan
| | - Naoya Asada
- Graduate School of Pharmaceutical Sciences, Kyoto University, Sakyo-ku, Kyoto 606-8501, Japan
| | - Isao Nakanishi
- Department of Pharmaceutical Sciences, Kinki University, 3-4-1,
Kowakae, Higashi-Osaka, Osaka 577-8502, Japan
| | - Kazuo Kitaura
- Graduate
School of System Informatics, Kobe University, 1-1 Rokkodai-cho, Nada-ku, Kobe 657-8501, Japan
| |
Collapse
|