1
|
Snowdon C, Barca GMJ. An Efficient RI-MP2 Algorithm for Distributed Many-GPU Architectures. J Chem Theory Comput 2024; 20:9394-9406. [PMID: 39422609 DOI: 10.1021/acs.jctc.4c00814] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2024]
Abstract
Second-order Møller-Plesset perturbation theory (MP2) using the Resolution of the Identity approximation (RI-MP2) is a widely used method for computing molecular energies beyond the Hartree-Fock mean-field approximation. However, its high computational cost and lack of efficient algorithms for modern supercomputing architectures limit its applicability to large molecules. In this paper, we present the first distributed-memory many-GPU RI-MP2 algorithm explicitly designed to utilize hundreds of GPU accelerators for every step of the computation. Our novel algorithm achieves near-peak performance on GPU-based supercomputers through the development of a distributed memory algorithm for forming RI-MP2 intermediate tensors with zero internode communication, except for a single O ( N 2 ) asynchronous broadcast, and a distributed memory algorithm for the O ( N 5 ) energy reduction step, capable of sustaining near-peak performance on clusters with several hundred GPUs. Comparative analysis shows our implementation outperforms state-of-the-art quantum chemistry software by over 3.5 times in speed while achieving an 8-fold reduction in computational power consumption. Benchmarking on the Perlmutter supercomputer, our algorithm achieves 11.8 PFLOP/s (83% of peak performance) performing and the RI-MP2 energy calculation on a 314-water cluster with 7850 primary and 30,144 auxiliary basis functions in 4 min on 180 nodes and 720 A100 GPUs. This performance represents a substantial improvement over traditional CPU-based methods, demonstrating significant time-to-solution and power consumption benefits of leveraging modern GPU-accelerated computing environments for quantum chemistry calculations.
Collapse
Affiliation(s)
- Calum Snowdon
- School of Computing, Australian National University, Canberra 2600, Australia
| | - Giuseppe M J Barca
- School of Computing and Information Systems, University of Melbourne, Melbourne 3010, Australia
| |
Collapse
|
2
|
Shi T, Wang Z, Aldossary A, Liu Y, Li XS, Head-Gordon M. Local Second Order Mo̷ller-Plesset Theory with a Single Threshold Using Orthogonal Virtual Orbitals: A Distributed Memory Implementation. J Chem Theory Comput 2024. [PMID: 39221855 DOI: 10.1021/acs.jctc.4c01016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/04/2024]
Abstract
In order to alleviate the computational burden associated with superlinear compute scalings with molecular size in electron correlation methods, researchers have developed local correlation methods that wisely treat relatively small contributions as zeros but still yield accurate energy approximation. Such local correlation techniques can also be combined with parallel computing resources to obtain further efficiency and scalability. This work focuses on the distributed memory parallel implementation of a local correlation method for second order Mo̷ller-Plesset (MP2) theory. This method also only has a single threshold to control the dropping of terms and accuracy of different computing kernels in the algorithm. The process partitioning strategy and distributed parallel implementation with the message passing interface (MPI) are discussed. In particular, the algorithm relies on a fixed sparsity pattern matrix multiplication and a corresponding distributed conjugate gradient solver, which exhibits almost linear scaling in both strong and weak scaling analyses. Numerical experiments on a range of molecules, including linear chains and molecules with 2 and 3-dimensional characters, are reported. For example, with only 32 MPI ranks, this MP2 implementation can calculate the correlation energy of vancomycin in def2-TZVP basis within 0.003% accuracy (10-6.5 threshold) in half an hour, where the same problem is unfeasible to solve with sequential or pure shared memory implementations.
Collapse
Affiliation(s)
- Tianyi Shi
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, United States
| | - Zhenling Wang
- Department of Chemistry, University of California, Berkeley, California 94720, United States
- Chemical Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, United States
| | | | - Yang Liu
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, United States
| | - Xiaoye S Li
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, United States
| | - Martin Head-Gordon
- Department of Chemistry, University of California, Berkeley, California 94720, United States
- Chemical Sciences Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, United States
| |
Collapse
|
3
|
Panchagnula K, Graf D, Johnson ER, Thom AJW. Targeting spectroscopic accuracy for dispersion bound systems from ab initio techniques: Translational eigenstates of Ne@C70. J Chem Phys 2024; 161:054308. [PMID: 39092939 DOI: 10.1063/5.0223298] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2024] [Accepted: 07/17/2024] [Indexed: 08/04/2024] Open
Abstract
We investigate the endofullerene system Ne@C70 by constructing a three-dimensional Potential Energy Surface (PES) describing the translational motion of the Ne atom. This is constructed from electronic structure calculations from a plethora of methods, including MP2, SCS-MP2, SOS-MP2, RPA@PBE, and C(HF)-RPA, which were previously used for He@C60 in Panchagnula et al. [J. Chem. Phys. 160, 104303 (2024)], alongside B86bPBE-25X-XDM and B86bPBE-50X-XDM. The reduction in symmetry moving from C60 to C70 introduces a double well potential along the anisotropic direction, which forms a test of the sensitivity and effectiveness of the electronic structure methods. The nuclear Hamiltonian is diagonalized using a symmetrized double minimum basis set outlined in Panchagnula and Thom [J. Chem. Phys. 159, 164308 (2023)], with translational energies having error bars ±1 and ±2 cm-1. We find no consistency between electronic structure methods as they find a range of barrier heights and minima positions of the double well and different translational eigenspectra, which also differ from the Lennard-Jones (LJ) PES given in Mandziuk and Bačić [J. Chem. Phys. 101, 2126-2140 (1994)]. We find that generating effective LJ parameters for each electronic structure method cannot reproduce the full PES nor recreate the eigenstates, and this suggests that the LJ form of the PES, while simple, may not be best suited to describe these systems. Even though MP2 and RPA@PBE performed best for He@C60, due to the lack of concordance between all electronic structure methods, we require more experimental data in order to properly validate the choice.
Collapse
Affiliation(s)
- K Panchagnula
- Yusuf Hamied Department of Chemistry, University of Cambridge, Cambridge, United Kingdom
| | - D Graf
- Yusuf Hamied Department of Chemistry, University of Cambridge, Cambridge, United Kingdom
- Department of Chemistry, University of Munich (LMU), Munich, Germany
| | - E R Johnson
- Yusuf Hamied Department of Chemistry, University of Cambridge, Cambridge, United Kingdom
- Department of Chemistry, Dalhousie University, 6243 Alumni Crescent, Halifax, Nova Scotia B3H 4R2, Canada
| | - A J W Thom
- Yusuf Hamied Department of Chemistry, University of Cambridge, Cambridge, United Kingdom
| |
Collapse
|
4
|
Sattasathuchana T, Xu P, Bertoni C, Kim YL, Leang SS, Pham BQ, Gordon MS. The Effective Fragment Molecular Orbital Method: Achieving High Scalability and Accuracy for Large Systems. J Chem Theory Comput 2024; 20:2445-2461. [PMID: 38450638 DOI: 10.1021/acs.jctc.3c01309] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/08/2024]
Abstract
The effective fragment molecular orbital (EFMO) method has been developed to predict the total energy of a very large molecular system accurately (with respect to the underlying quantum mechanical method) and efficiently by taking advantage of the locality of strong chemical interactions and employing a two-level hierarchical parallelism. The accuracy of the EFMO method is partly attributed to the accurate and robust intermolecular interaction prediction between distant fragments, in particular, the many-body polarization and dispersion effects, which require the generation of static and dynamic polarizability tensors by solving the coupled perturbed Hartree-Fock (CPHF) and time-dependent HF (TDHF) equations, respectively. Solving the CPHF and TDHF equations is the main EFMO computational bottleneck due to the inefficient (serial) and I/O-intensive implementation of the CPHF and TDHF solvers. In this work, the efficiency and scalability of the EFMO method are significantly improved with a new CPU memory-based implementation for solving the CPHF and TDHF equations that are parallelized by either message passing interface (MPI) or hybrid MPI/OpenMP. The accuracy of the EFMO method is demonstrated for both covalently bonded systems and noncovalently bound molecular clusters by systematically examining the effects of basis sets and a key distance-related cutoff parameter, Rcut. Rcut determines whether a fragment pair (dimer) is treated by the chosen ab initio method or calculated using the effective fragment potential (EFP) method (separated dimers). Decreasing the value of Rcut increases the number of separated (EFP) dimers, thereby decreasing the computational effort. It is demonstrated that excellent accuracy (<1 kcal/mol error per fragment) can be achieved when using a sufficiently large basis set with diffuse functions coupled with a small Rcut value. With the new parallel implementation, the total EFMO wall time is substantially reduced, especially with a high number of MPI ranks. Given a sufficient workload, nearly ideal strong scaling is achieved for the CPHF and TDHF parts of the calculation. For the first time, EFMO calculations with the inclusion of long-range polarization and dispersion interactions on a hydrated mesoporous silica nanoparticle with explicit water solvent molecules (more than 15k atoms) are achieved on a massively parallel supercomputer using nearly 1000 physical nodes. In addition, EFMO calculations on the carbinolamine formation step of an amine-catalyzed aldol reaction at the nanoscale with explicit solvent effects are presented.
Collapse
Affiliation(s)
- Tosaporn Sattasathuchana
- Department of Chemistry, Iowa State University and Ames National Laboratory, Ames, Iowa 50011, United States
| | - Peng Xu
- Department of Chemistry, Iowa State University and Ames National Laboratory, Ames, Iowa 50011, United States
| | - Colleen Bertoni
- Argonne Leadership Computing Facility, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - Yu Lim Kim
- Chemical Sciences and Engineering Division, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - Sarom S Leang
- EP Analytics, Inc., 9909 Mira Mesa Blvd Ste. 230, San Diego, California 92131, United States
| | - Buu Q Pham
- Department of Chemistry, Iowa State University and Ames National Laboratory, Ames, Iowa 50011, United States
| | - Mark S Gordon
- Department of Chemistry, Iowa State University and Ames National Laboratory, Ames, Iowa 50011, United States
| |
Collapse
|
5
|
Stocks R, Palethorpe E, Barca GMJ. High-Performance Multi-GPU Analytic RI-MP2 Energy Gradients. J Chem Theory Comput 2024; 20:2505-2519. [PMID: 38456899 DOI: 10.1021/acs.jctc.3c01424] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/09/2024]
Abstract
This article presents a novel algorithm for the calculation of analytic energy gradients from second-order Møller-Plesset perturbation theory within the Resolution-of-the-Identity approximation (RI-MP2), which is designed to achieve high performance on clusters with multiple graphical processing units (GPUs). The algorithm uses GPUs for all major steps of the calculation, including integral generation, formation of all required intermediate tensors, solution of the Z-vector equation and gradient accumulation. The implementation in the EXtreme Scale Electronic Structure System (EXESS) software package includes a tailored, highly efficient, multistream scheduling system to hide CPU-GPU data transfer latencies and allows nodes with 8 A100 GPUs to operate at over 80% of theoretical peak floating-point performance. Comparative performance analysis shows a significant reduction in computational time relative to traditional multicore CPU-based methods, with our approach achieving up to a 95-fold speedup over the single-node performance of established software such as Q-Chem and ORCA. Additionally, we demonstrate that pairing our implementation with the molecular fragmentation framework in EXESS can drastically lower the computational scaling of RI-MP2 gradient calculations from quintic to subquadratic, enabling further substantial savings in runtime while retaining high numerical accuracy in the resulting gradients.
Collapse
Affiliation(s)
- Ryan Stocks
- School of Computing, Australian National University, Canberra, ACT 2601, Australia
| | - Elise Palethorpe
- School of Computing, Australian National University, Canberra, ACT 2601, Australia
| | - Giuseppe M J Barca
- School of Computing, Australian National University, Canberra, ACT 2601, Australia
| |
Collapse
|
6
|
Panchagnula K, Graf D, Albertani FEA, Thom AJW. Translational eigenstates of He@C60 from four-dimensional ab initio potential energy surfaces interpolated using Gaussian process regression. J Chem Phys 2024; 160:104303. [PMID: 38465682 DOI: 10.1063/5.0197903] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Accepted: 02/22/2024] [Indexed: 03/12/2024] Open
Abstract
We investigate the endofullerene system 3He@C60 with a four-dimensional potential energy surface (PES) to include the three He translational degrees of freedom and C60 cage radius. We compare second order Møller-Plesset perturbation theory (MP2), spin component scaled-MP2, scaled opposite spin-MP2, random phase approximation (RPA)@Perdew, Burke, and Ernzerhof (PBE), and corrected Hartree-Fock-RPA to calibrate and gain confidence in the choice of electronic structure method. Due to the high cost of these calculations, the PES is interpolated using Gaussian Process Regression (GPR), owing to its effectiveness with sparse training data. The PES is split into a two-dimensional radial surface, to which corrections are applied to achieve an overall four-dimensional surface. The nuclear Hamiltonian is diagonalized to generate the in-cage translational/vibrational eigenstates. The degeneracy of the three-dimensional harmonic oscillator energies with principal quantum number n is lifted due to the anharmonicity in the radial potential. The (2l + 1)-fold degeneracy of the angular momentum states is also weakly lifted, due to the angular dependence in the potential. We calculate the fundamental frequency to range between 96 and 110 cm-1 depending on the electronic structure method used. Error bars of the eigenstate energies were calculated from the GPR and are on the order of ∼±1.5 cm-1. Wavefunctions are also compared by considering their overlap and Hellinger distance to the one-dimensional empirical potential. As with the energies, the two ab initio methods MP2 and RPA@PBE show the best agreement. While MP2 has better agreement than RPA@PBE, due to its higher computational efficiency and comparable performance, we recommend RPA as an alternative electronic structure method of choice to MP2 for these systems.
Collapse
Affiliation(s)
- K Panchagnula
- Yusuf Hamied Department of Chemistry, University of Cambridge, Cambridge, United Kingdom
| | - D Graf
- Yusuf Hamied Department of Chemistry, University of Cambridge, Cambridge, United Kingdom
| | - F E A Albertani
- Yusuf Hamied Department of Chemistry, University of Cambridge, Cambridge, United Kingdom
| | - A J W Thom
- Yusuf Hamied Department of Chemistry, University of Cambridge, Cambridge, United Kingdom
| |
Collapse
|
7
|
Martin RL, Heifetz A, Bodkin MJ, Townsend-Nicholson A. High-Throughput Structure-Based Drug Design (HT-SBDD) Using Drug Docking, Fragment Molecular Orbital Calculations, and Molecular Dynamic Techniques. Methods Mol Biol 2024; 2716:293-306. [PMID: 37702945 DOI: 10.1007/978-1-0716-3449-3_13] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/14/2023]
Abstract
Structure-based drug design (SBDD) is rapidly evolving to be a fundamental tool for faster and more cost-effective methods of lead drug discovery. SBDD aims to offer a computational replacement to traditional high-throughput screening (HTS) methods of drug discovery. This "virtual screening" technique utilizes the structural data of a target protein in conjunction with large databases of potential drug candidates and then applies a range of different computational techniques to determine which potential candidates are likely to bind with high affinity and efficacy. It is proposed that high-throughput SBDD (HT-SBDD) will significantly enrich the success rate of HTS methods, which currently fluctuates around ~1%. In this chapter, we focus on the theory and utility of high-throughput drug docking, fragment molecular orbital calculations, and molecular dynamics techniques. We also offer a comparative review of the benefits and limitations of traditional methods against more recent SBDD advances. As HT-SBDD is computationally intensive, we will also cover the important role high-performance computing (HPC) clusters play in the future of computational drug discovery.
Collapse
Affiliation(s)
- Reuben L Martin
- Research Department of Structural & Molecular Biology, Division of Biosciences, University College London, London, UK.
- Evotec (UK) Ltd., Abingdon, Oxfordshire, UK.
| | | | | | - Andrea Townsend-Nicholson
- Research Department of Structural & Molecular Biology, Division of Biosciences, University College London, London, UK
| |
Collapse
|
8
|
Kim S, Conrad JA, Tow GM, Maginn EJ, Boatz JA, Gordon MS. Intermolecular interactions in clusters of ethylammonium nitrate and 1-amino-1,2,3-triazole. Phys Chem Chem Phys 2023; 25:30428-30457. [PMID: 37917371 DOI: 10.1039/d3cp02407e] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2023]
Abstract
The intermolecular interaction energies, including hydrogen bonds (H-bonds), of clusters of the ionic liquid ethylammonium nitrate (EAN) and 1-amino-1,2,3-triazole (1-AT) based deep eutectic propellants (DeEP) are examined. 1-AT is introduced as a neutral hydrogen bond donor (HBD) to EAN in order to form a eutectic mixture. The effective fragment potential (EFP) is used to examine the bonding interactions in the DeEP clusters. The resolution of the Identity (RI) approximated second order Møller-Plesset perturbation theory (RI-MP2) and coupled cluster theory (RI-CCSD(T)) are used to validate the EFP results. The EFP method predicts that there are significant polarization and charge transfer effects in the EAN:1-AT complexes, along with Coulombic, dispersion and exchange repulsion interactions. The EFP interaction energies are in good agreement with the RI-MP2 and RI-CCSD(T) results. The quasi-atomic orbital (QUAO) bonding and kinetic bond order (KBO) analyses are additionally used to develop a conceptual and semi-quantitative understanding of the H-bonding interactions as a function of the size of the system. The QUAO and KBO analyses suggest that the H-bonds in the examined clusters follow the characteristic hydrogen bonding three-center four electron interactions. The strongest H-bonding interactions between the (EAN)1:(1-AT)n and (EAN)2:(1-AT)n (n = 1-5) complexes are observed internally within EAN; that is, between the ethylammonium cation [EA]+ and the nitrate anion ([NO3]-). The weakest H-bonding interactions occur between [NO3]- and 1-AT. Consequently, the average strengths of the H-bonds within a given (EAN)x:(1-AT)n complex decrease as more 1-AT molecules are introduced into the EAN monomer and EAN dimer. The QUAO bonding analysis suggests that 1-AT in (EAN)x:(1-AT)n can act as both a HBD and a hydrogen bond acceptor simultaneously. It is observed that two 1-AT molecules can form H-bonds to each other. Although the KBOs that correspond to H-bonding interactions in [EA]+:1-AT, [NO3]-:1-AT and between two 1-AT molecules are weaker than the H-bonds in EAN, those weak H-bond networks with 1-AT could be important to form a stable DeEP.
Collapse
Affiliation(s)
- Shinae Kim
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, IA 50011, USA.
- Combustion Research Facility, Sandia National Laboratories, Livermore, California 94550, USA
| | - Justin A Conrad
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, IA 50011, USA.
| | - Garrett M Tow
- Department of Chemical & Biomolecular Engineering, University of Notre Dame, Notre Dame, Indiana 46556, USA
| | - Edward J Maginn
- Department of Chemical & Biomolecular Engineering, University of Notre Dame, Notre Dame, Indiana 46556, USA
| | - Jerry A Boatz
- Aerospace Systems Directorate, Air Force Research Laboratory, Edwards Air Force Base, California 93524, USA
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, IA 50011, USA.
| |
Collapse
|
9
|
Zahariev F, Xu P, Westheimer BM, Webb S, Galvez Vallejo J, Tiwari A, Sundriyal V, Sosonkina M, Shen J, Schoendorff G, Schlinsog M, Sattasathuchana T, Ruedenberg K, Roskop LB, Rendell AP, Poole D, Piecuch P, Pham BQ, Mironov V, Mato J, Leonard S, Leang SS, Ivanic J, Hayes J, Harville T, Gururangan K, Guidez E, Gerasimov IS, Friedl C, Ferreras KN, Elliott G, Datta D, Cruz DDA, Carrington L, Bertoni C, Barca GMJ, Alkan M, Gordon MS. The General Atomic and Molecular Electronic Structure System (GAMESS): Novel Methods on Novel Architectures. J Chem Theory Comput 2023; 19:7031-7055. [PMID: 37793073 DOI: 10.1021/acs.jctc.3c00379] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/06/2023]
Abstract
The primary focus of GAMESS over the last 5 years has been the development of new high-performance codes that are able to take effective and efficient advantage of the most advanced computer architectures, both CPU and accelerators. These efforts include employing density fitting and fragmentation methods to reduce the high scaling of well-correlated (e.g., coupled-cluster) methods as well as developing novel codes that can take optimal advantage of graphical processing units and other modern accelerators. Because accurate wave functions can be very complex, an important new functionality in GAMESS is the quasi-atomic orbital analysis, an unbiased approach to the understanding of covalent bonds embedded in the wave function. Best practices for the maintenance and distribution of GAMESS are also discussed.
Collapse
Affiliation(s)
- Federico Zahariev
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Peng Xu
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Bryce M Westheimer
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Simon Webb
- VeraChem LLC, 12850 Middlebrook Road, Suite 205, Germantown, Maryland 20874-5244, United States
| | - Jorge Galvez Vallejo
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
- Research School of Computer Science, Australian National University, Canberra, ACT 2601, Australia
| | - Ananta Tiwari
- EP Analytics, Inc., 9909 Mira Mesa Boulevard, Suite 230, San Diego, California 92131, United States
| | - Vaibhav Sundriyal
- Department of Computational Modeling and Simulation Engineering, Old Dominion University, Norfolk, Virginia 23529, United States
| | - Masha Sosonkina
- Department of Computational Modeling and Simulation Engineering, Old Dominion University, Norfolk, Virginia 23529, United States
| | - Jun Shen
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, United States
| | - George Schoendorff
- Propellants Branch, Rocket Propulsion Division, Aerospace Systems Directorate, Air Force Research Laboratory, AFRL/RQRP, Edwards Air Force Base, California 93524, United States
| | - Megan Schlinsog
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Tosaporn Sattasathuchana
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Klaus Ruedenberg
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Luke B Roskop
- Hewlett-Packard Enterprise, 2131 Lindau Lane #1000, Bloomington, Minnesota 55425, United States
| | | | - David Poole
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
- School of Chemistry & Biochemistry, Georgia Institute of Technology, Athens, Georgia 30332, United States
| | - Piotr Piecuch
- Department of Chemistry and Department of Physics and Astronomy, Michigan State University, East Lansing, Michigan 48824, United States
| | - Buu Q Pham
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Vladimir Mironov
- Department of Chemistry, Kyungpook National University, Daegu 41566, South Korea
| | - Joani Mato
- Physical Sciences Division, Pacific Northwest National Laboratory, 902 Battelle Boulevard, P.O. Box 999, MS K1-83, Richland, Washington 99352, United States
| | - Sam Leonard
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Sarom S Leang
- EP Analytics, Inc., 9909 Mira Mesa Boulevard, Suite 230, San Diego, California 92131, United States
| | - Joe Ivanic
- Advanced Biomedical Computational Science, Frederick National Laboratory for Cancer Research, Frederick, Maryland 21702, United States
| | - Jackson Hayes
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Taylor Harville
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Karthik Gururangan
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, United States
| | - Emilie Guidez
- Department of Chemistry, University of Colorado Denver, Denver, Colorado 80217, United States
| | - Igor S Gerasimov
- Department of Chemistry, Kyungpook National University, Daegu 41566, South Korea
| | - Christian Friedl
- Institut für Theoretische Physik, Johannes Kepler Universität Linz, Altenberger Str. 69, 4040 Linz, Austria
| | - Katherine N Ferreras
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - George Elliott
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Dipayan Datta
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Daniel Del Angel Cruz
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Laura Carrington
- EP Analytics, Inc., 9909 Mira Mesa Boulevard, Suite 230, San Diego, California 92131, United States
| | - Colleen Bertoni
- Argonne Leadership Computing Facility, Argonne National Laboratory, Lemont, Illinois 60439, United States
| | - Giuseppe M J Barca
- Research School of Computer Science, Australian National University, Canberra, ACT 2601, Australia
| | - Melisa Alkan
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| |
Collapse
|
10
|
Galvez Vallejo JL, Snowdon C, Stocks R, Kazemian F, Yan Yu FC, Seidl C, Seeger Z, Alkan M, Poole D, Westheimer BM, Basha M, De La Pierre M, Rendell A, Izgorodina EI, Gordon MS, Barca GMJ. Toward an extreme-scale electronic structure system. J Chem Phys 2023; 159:044112. [PMID: 37497819 DOI: 10.1063/5.0156399] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Accepted: 07/03/2023] [Indexed: 07/28/2023] Open
Abstract
Electronic structure calculations have the potential to predict key matter transformations for applications of strategic technological importance, from drug discovery to material science and catalysis. However, a predictive physicochemical characterization of these processes often requires accurate quantum chemical modeling of complex molecular systems with hundreds to thousands of atoms. Due to the computationally demanding nature of electronic structure calculations and the complexity of modern high-performance computing hardware, quantum chemistry software has historically failed to operate at such large molecular scales with accuracy and speed that are useful in practice. In this paper, novel algorithms and software are presented that enable extreme-scale quantum chemistry capabilities with particular emphasis on exascale calculations. This includes the development and application of the multi-Graphics Processing Unit (GPU) library LibCChem 2.0 as part of the General Atomic and Molecular Electronic Structure System package and of the standalone Extreme-scale Electronic Structure System (EXESS), designed from the ground up for scaling on thousands of GPUs to perform high-performance accurate quantum chemistry calculations at unprecedented speed and molecular scales. Among various results, we report that the EXESS implementation enables Hartree-Fock/cc-pVDZ plus RI-MP2/cc-pVDZ/cc-pVDZ-RIFIT calculations on an ionic liquid system with 623 016 electrons and 146 592 atoms in less than 45 min using 27 600 GPUs on the Summit supercomputer with a 94.6% parallel efficiency.
Collapse
Affiliation(s)
| | - Calum Snowdon
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Ryan Stocks
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Fazeleh Kazemian
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Fiona Chuo Yan Yu
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Christopher Seidl
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Zoe Seeger
- School of Chemistry, Monash University, Clayton 3800, VIC, Australia
| | - Melisa Alkan
- Department of Chemistry, Iowa State University, Ames, Iowa 50011-3111, USA
| | - David Poole
- School of Chemistry and Biochemistry, Georgia Institute of Technology, Atlanta, Georgia 30332, USA
| | - Bryce M Westheimer
- Department of Chemistry, Iowa State University, Ames, Iowa 50011-3111, USA
| | - Mehaboob Basha
- Pawsey Supercomputing Research Centre, Kensington, WA 6151, Australia
| | | | - Alistair Rendell
- College of Science and Engineering, Flinders University, Adelaide, SA 5042, Australia
| | | | | | - Giuseppe M J Barca
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| |
Collapse
|
11
|
Pham BQ, Carrington L, Tiwari A, Leang SS, Alkan M, Bertoni C, Datta D, Sattasathuchana T, Xu P, Gordon MS. Porting fragmentation methods to GPUs using an OpenMP API: Offloading the resolution-of-the-identity second-order Møller-Plesset perturbation method. J Chem Phys 2023; 158:2887208. [PMID: 37114705 DOI: 10.1063/5.0143424] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Accepted: 04/10/2023] [Indexed: 04/29/2023] Open
Abstract
Using an OpenMP Application Programming Interface, the resolution-of-the-identity second-order Møller-Plesset perturbation (RI-MP2) method has been off-loaded onto graphical processing units (GPUs), both as a standalone method in the GAMESS electronic structure program and as an electron correlation energy component in the effective fragment molecular orbital (EFMO) framework. First, a new scheme has been proposed to maximize data digestion on GPUs that subsequently linearizes data transfer from central processing units (CPUs) to GPUs. Second, the GAMESS Fortran code has been interfaced with GPU numerical libraries (e.g., NVIDIA cuBLAS and cuSOLVER) for efficient matrix operations (e.g., matrix multiplication, matrix decomposition, and matrix inversion). The standalone GPU RI-MP2 code shows an increasing speedup of up to 7.5× using one NVIDIA V100 GPU with one IBM 42-core P9 CPU for calculations on fullerenes of increasing size from 40 to 260 carbon atoms using the 6-31G(d)/cc-pVDZ-RI basis sets. A single Summit node with six V100s can compute the RI-MP2 correlation energy of a cluster of 175 water molecules using the correlation consistent basis sets cc-pVDZ/cc-pVDZ-RI containing 4375 atomic orbitals and 14 700 auxiliary basis functions in ∼0.85 h. In the EFMO framework, the GPU RI-MP2 component shows near linear scaling for a large number of V100s when computing the energy of an 1800-atom mesoporous silica nanoparticle in a bath of 4000 water molecules. The parallel efficiencies of the GPU RI-MP2 component with 2304 and 4608 V100s are 98.0% and 96.1%, respectively.
Collapse
Affiliation(s)
- Buu Q Pham
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | | | | | | | - Melisa Alkan
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Colleen Bertoni
- Leadership Computing Facility, Argonne National Laboratory, Lemont, Illinois 60439, USA
| | - Dipayan Datta
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | | | - Peng Xu
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| |
Collapse
|
12
|
Galvez Vallejo JL, Tow GM, Maginn EJ, Pham BQ, Datta D, Gordon MS. Quantum Chemical Modeling of Propellant Degradation. J Phys Chem A 2023; 127:1874-1882. [PMID: 36791340 DOI: 10.1021/acs.jpca.2c08722] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/17/2023]
Abstract
An ab initio quantum chemical approach for the modeling of propellant degradation is presented. Using state-of-the-art bonding analysis techniques and composite methods, a series of potential degradation reactions are devised for a sample hydroxyl-terminated-polybutadiene (HTPB) type solid fuel. By applying thermochemical procedures and isodesmic reactions, accurate thermochemical quantities are obtained using a modified G3 composite method based on the resolution of the identity. The calculated heats of formation for the different structures produced presents an ∼2 kcal/mol average error when compared against experimental values.
Collapse
Affiliation(s)
- Jorge L Galvez Vallejo
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Garrett M Tow
- Department of Chemical and Biomolecular Engineering, University of Notre Dame, Notre Dame, Indiana 46556, United States
| | - Edward J Maginn
- Department of Chemical and Biomolecular Engineering, University of Notre Dame, Notre Dame, Indiana 46556, United States
| | - Buu Q Pham
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Dipayan Datta
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50014, United States
| |
Collapse
|
13
|
Poole D, Galvez Vallejo JL, Gordon MS. A Task-Based Approach to Parallel Restricted Hartree-Fock Calculations. J Chem Theory Comput 2022; 18:2144-2161. [PMID: 35377639 DOI: 10.1021/acs.jctc.1c00820] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
In recent years, parallelism via multithreading has become extremely important to the optimization of high-performance electronic structure theory codes. Such multithreading is generally achieved via OpenMP constructs, using a fork-join threading model to enable thread-level data parallelism within the code. An alternative approach to multithreading is task-based parallelism, which displays multiple benefits relative to fork-join thread parallelism. A novel Restricted Hartree-Fock (RHF) algorithm, utilizing task-based parallelism to achieve optimal performance, was developed and implemented into the JuliaChem electronic structure theory software package. The new RHF algorithm utilizes a unique method of shell quartet batch creation, enabling construction and distribution of fine-grained shell quartet batches in a load-balanced manner using the Julia task construct. These shell quartet batches are then distributed statically across message-passing interface (MPI) ranks and dynamically across threads within an MPI rank, requiring no explicit inter-rank or interthread synchronization to do so. Compared to the hybrid MPI/OpenMP RHF algorithm present in the GAMESS software package, the task-based algorithm demonstrates speedups of up to ∼40% for systems in the S22(3) test set of molecules, with system sizes up to ∼1000 basis functions. The JuliaChem algorithm demonstrates the viability of both the task-based parallelism model and the Julia programming language for construction of performant electronic structure theory codes targeting systems of a size of chemical interest.
Collapse
Affiliation(s)
- David Poole
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, United States
| | - Jorge L Galvez Vallejo
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, United States
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, United States
| |
Collapse
|
14
|
Harville T, Gordon MS. Electronic Structure Theory Calculations Using Modern Architectures: KNL vs Haswell. J Chem Theory Comput 2021; 17:6910-6917. [PMID: 34699218 DOI: 10.1021/acs.jctc.1c00705] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
The time to solution and parallel efficiency of several commonly used electronic structure methods (Hartree-Fock, density functional theory, second order perturbation theory, resolution of the identity second order perturbation theory, coupled cluster) are evaluated on both the Intel Xeon Haswell and the Intel Xeon Phi Knights Landing (KNL) architectures. The Haswell completes the benchmark calculations with a faster time to solution than the KNL for all molecules and methods tested. While the Haswell exhibits an average speedup of at least 3.5 relative to the KNL for all nonthreaded computations, the KNL has a better parallel efficiency than the Haswell with increasing core counts. The architectures are further tested using a more computationally costly coupled cluster method on a transition state reaction. The Haswell appears to be the best choice to minimize the time to solution, though for very large systems and high levels of theory that require memory intensive processes the superior memory hierarchy and larger on node memory of the KNL can make it a better choice. These results are used to showcase aspects of novel architectures that will increase efficiency for quantum chemistry applications.
Collapse
Affiliation(s)
- Taylor Harville
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, United States
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, United States
| |
Collapse
|
15
|
Pham BQ, Datta D, Gordon MS. PDG: A Composite Method Based on the Resolution of the Identity. J Phys Chem A 2021; 125:9421-9429. [PMID: 34658243 DOI: 10.1021/acs.jpca.1c06186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
The Gaussian-3 (G3) composite approach for thermochemical properties is revisited in light of the enhanced computational efficiency and reduced memory costs by applying the resolution-of-the-identity (RI) approximation for two-electron repulsion integrals (ERIs) to the computationally demanding component methods in the G3 model: the energy and gradient computations via the second-order Møller-Plesset perturbation theory (MP2) and the energy computations using the coupled-cluster singles-doubles method augmented with noniterative triples corrections [CCSD(T)]. Efficient implementation of the RI-based methods is achieved by employing a hybrid distributed/shared memory model based on MPI and OpenMP. The new variant of the G3 composite approach based on the RI approximation is termed the RI-G3 scheme, or alternatively the PDG method. The accuracy of the new RI-G3/PDG scheme is compared to the "standard" G3 composite approach that employs the memory-expensive four-center ERIs in the MP2 and CCSD(T) calculations. Taking the computation of the heats of formation of the closed-shell molecules in the G3/99 test set as a test case, it is demonstrated that the RI approximation introduces negligible changes to the mean absolute errors relative to the standard G3 model (less than 0.1 kcal/mol), while the standard deviations remain unaltered. The efficiency and memory requirements for the RI-MP2 and RI-CCSD(T) methods are compared to the standard MP2 and CCSD(T) approaches, respectively. The hybrid MPI/OpenMP-based RI-MP2 energy plus gradient computation is found to attain a 7.5× speedup over the standard MP2 calculations. For the most demanding CCSD(T) calculations, the application of the RI approximation is found to nearly halve the memory demand, confer about a 4-5× speedup for the CCSD iterations, and reduce the computational time for the compute-intensive triples correction step by several hours.
Collapse
|
16
|
Datta D, Gordon MS. A Massively Parallel Implementation of the CCSD(T) Method Using the Resolution-of-the-Identity Approximation and a Hybrid Distributed/Shared Memory Parallelization Model. J Chem Theory Comput 2021; 17:4799-4822. [PMID: 34279094 DOI: 10.1021/acs.jctc.1c00389] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
A parallel algorithm is described for the coupled-cluster singles and doubles method augmented with a perturbative correction for triple excitations [CCSD(T)] using the resolution-of-the-identity (RI) approximation for two-electron repulsion integrals (ERIs). The algorithm bypasses the storage of four-center ERIs by adopting an integral-direct strategy. The CCSD amplitude equations are given in a compact quasi-linear form by factorizing them in terms of amplitude-dressed three-center intermediates. A hybrid MPI/OpenMP parallelization scheme is employed, which uses the OpenMP-based shared memory model for intranode parallelization and the MPI-based distributed memory model for internode parallelization. Parallel efficiency has been optimized for all terms in the CCSD amplitude equations. Two different algorithms have been implemented for the rate-limiting terms in the CCSD amplitude equations that entail O(NO2NV4) and O(NO3NV3)-scaling computational costs, where NO and NV denote the number of correlated occupied and virtual orbitals, respectively. One of the algorithms assembles the four-center ERIs requiring NV4 and NO2NV2-scaling memory costs in a distributed manner on a number of MPI ranks, while the other algorithm completely bypasses the assembling of quartic memory-scaling ERIs and thus largely reduces the memory demand. It is demonstrated that the former memory-expensive algorithm is faster on a few hundred cores, while the latter memory-economic algorithm shows a better strong scaling in the limit of a few thousand cores. The program is shown to exhibit a near-linear scaling, in particular for the compute-intensive triples correction step, on up to 8000 cores. The performance of the program is demonstrated via calculations involving molecules with 24-51 atoms and up to 1624 atomic basis functions. As the first application, the complete basis set (CBS) limit for the interaction energy of the π-stacked uracil dimer from the S66 data set has been investigated. This work reports the first calculation of the interaction energy at the CCSD(T)/aug-cc-pVQZ level without local orbital approximation. The CBS limit for the CCSD correlation contribution to the interaction energy was found to be -8.01 kcal/mol, which agrees very well with the value -7.99 kcal/mol reported by Schmitz, Hättig, and Tew [ Phys. Chem. Chem. Phys. 2014, 16, 22167-22178]. The CBS limit for the total interaction energy was estimated to be -9.64 kcal/mol.
Collapse
Affiliation(s)
- Dipayan Datta
- Department of Chemistry and Ames Laboratory, Iowa State University, 2416 Pammel Drive, Ames 50011-2416, Iowa United States of America
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory, Iowa State University, 2416 Pammel Drive, Ames 50011-2416, Iowa United States of America
| |
Collapse
|
17
|
Xu P, Sattasathuchana T, Guidez E, Webb SP, Montgomery K, Yasini H, Pedreira IFM, Gordon MS. Computation of host-guest binding free energies with a new quantum mechanics based mining minima algorithm. J Chem Phys 2021; 154:104122. [PMID: 33722015 PMCID: PMC7955858 DOI: 10.1063/5.0040759] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2020] [Accepted: 02/11/2021] [Indexed: 11/14/2022] Open
Abstract
A new method called QM-VM2 is presented that efficiently combines statistical mechanics with quantum mechanical (QM) energy potentials in order to calculate noncovalent binding free energies of host-guest systems. QM-VM2 efficiently couples the use of semi-empirical QM (SEQM) energies and geometry optimizations with an underlying molecular mechanics (MM) based conformational search, to find low SEQM energy minima, and allows for processing of these minima at higher levels of ab initio QM theory. A progressive geometry optimization scheme is introduced as a means to increase conformational sampling efficiency. The newly implemented QM-VM2 is used to compute the binding free energies of the host molecule cucurbit[7]uril and a set of 15 guest molecules. The results are presented along with comparisons to experimentally determined binding affinities. For the full set of 15 host-guest complexes, which have a range of formal charges from +1 to +3, SEQM-VM2 based binding free energies show poor correlation with experiment, whereas for the ten +1 complexes only, a significant correlation (R2 = 0.8) is achieved. SEQM-VM2 generation of conformers followed by single-point ab initio QM calculations at the dispersion corrected restricted Hartree-Fock-D3(BJ) and TPSS-D3(BJ) levels of theory, as post-processing corrections, yields a reasonable correlation with experiment for the full set of host-guest complexes (R2 = 0.6 and R2 = 0.7, respectively) and an excellent correlation for the +1 formal charge set (R2 = 1.0 and R2 = 0.9, respectively), as long as a sufficiently large basis set (triple-zeta quality) is employed. The importance of the inclusion of configurational entropy, even at the MM level, for the achievement of good correlation with experiment was demonstrated by comparing the calculated ΔE values with experiment and finding a considerably poorer correlation with experiment than for the calculated free energy ΔE - TΔS. For the complete set of host-guest systems with the range of formal charges, it was observed that the deviation of the predicted binding free energy from experiment correlates somewhat with the net charge of the systems. This observation leads to a simple empirical interpolation scheme to improve the linear regression of the full set.
Collapse
Affiliation(s)
- Peng Xu
- Department of Chemistry, Iowa State University, Ames, Iowa 50014, USA
| | | | - Emilie Guidez
- Department of Chemistry, University of Colorado Denver, Denver, Colorado 80204, USA
| | - Simon P. Webb
- VeraChem LLC, 12850 Middlebrook Rd. Ste 205, Germantown, Maryland 20874-5244, USA
| | | | - Hussna Yasini
- Department of Chemistry, University of Colorado Denver, Denver, Colorado 80204, USA
| | - Iara F. M. Pedreira
- Department of Chemistry, University of Colorado Denver, Denver, Colorado 80204, USA
| | - Mark S. Gordon
- Department of Chemistry, Iowa State University, Ames, Iowa 50014, USA
| |
Collapse
|
18
|
Glasbrenner M, Graf D, Ochsenfeld C. Efficient Reduced-Scaling Second-Order Møller-Plesset Perturbation Theory with Cholesky-Decomposed Densities and an Attenuated Coulomb Metric. J Chem Theory Comput 2020; 16:6856-6868. [PMID: 33074664 DOI: 10.1021/acs.jctc.0c00600] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
We present a novel, highly efficient method for the computation of second-order Møller-Plesset perturbation theory (MP2) correlation energies, which uses the resolution of the identity (RI) approximation and local molecular orbitals obtained from a Cholesky decomposition of pseudodensity matrices (CDD), as in the RI-CDD-MP2 method developed previously in our group [Maurer, S. A.; Clin, L.; Ochsenfeld, C. J. Chem. Phys. 2014, 140, 224112]. In addition, we introduce an attenuated Coulomb metric and subsequently redesign the RI-CDD-MP2 method in order to exploit the resulting sparsity in the three-center integrals. Coulomb and exchange energy contributions are computed separately using specialized algorithms. A simple, yet effective integral screening protocol based on Schwarz estimates is used for the MP2 exchange energy. The Coulomb energy computation and the preceding transformations of the three-center integrals are accelerated using a modified version of the natural blocking approach [Jung, Y.; Head-Gordon, M. Phys. Chem. Chem. Phys. 2006, 8, 2831-2840]. Effective subquadratic scaling for a wide range of molecule sizes is demonstrated in test calculations in conjunction with a low prefactor. The method is shown to enable cost-efficient MP2 calculations on large molecular systems with several thousand basis functions.
Collapse
Affiliation(s)
- Michael Glasbrenner
- Chair of Theoretical Chemistry, Department of Chemistry, University of Munich (LMU), Butenandtstrasse 7, 81377 Munich, Germany
| | - Daniel Graf
- Chair of Theoretical Chemistry, Department of Chemistry, University of Munich (LMU), Butenandtstrasse 7, 81377 Munich, Germany
| | - Christian Ochsenfeld
- Chair of Theoretical Chemistry, Department of Chemistry, University of Munich (LMU), Butenandtstrasse 7, 81377 Munich, Germany
| |
Collapse
|
19
|
Poole D, Galvez Vallejo JL, Gordon MS. A New Kid on the Block: Application of Julia to Hartree–Fock Calculations. J Chem Theory Comput 2020; 16:5006-5013. [DOI: 10.1021/acs.jctc.0c00337] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/03/2023]
Affiliation(s)
- David Poole
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, United States
| | - Jorge L. Galvez Vallejo
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, United States
| | - Mark S. Gordon
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, United States
| |
Collapse
|
20
|
Barca GMJ, Bertoni C, Carrington L, Datta D, De Silva N, Deustua JE, Fedorov DG, Gour JR, Gunina AO, Guidez E, Harville T, Irle S, Ivanic J, Kowalski K, Leang SS, Li H, Li W, Lutz JJ, Magoulas I, Mato J, Mironov V, Nakata H, Pham BQ, Piecuch P, Poole D, Pruitt SR, Rendell AP, Roskop LB, Ruedenberg K, Sattasathuchana T, Schmidt MW, Shen J, Slipchenko L, Sosonkina M, Sundriyal V, Tiwari A, Galvez Vallejo JL, Westheimer B, Włoch M, Xu P, Zahariev F, Gordon MS. Recent developments in the general atomic and molecular electronic structure system. J Chem Phys 2020; 152:154102. [PMID: 32321259 DOI: 10.1063/5.0005188] [Citation(s) in RCA: 575] [Impact Index Per Article: 115.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
A discussion of many of the recently implemented features of GAMESS (General Atomic and Molecular Electronic Structure System) and LibCChem (the C++ CPU/GPU library associated with GAMESS) is presented. These features include fragmentation methods such as the fragment molecular orbital, effective fragment potential and effective fragment molecular orbital methods, hybrid MPI/OpenMP approaches to Hartree-Fock, and resolution of the identity second order perturbation theory. Many new coupled cluster theory methods have been implemented in GAMESS, as have multiple levels of density functional/tight binding theory. The role of accelerators, especially graphical processing units, is discussed in the context of the new features of LibCChem, as it is the associated problem of power consumption as the power of computers increases dramatically. The process by which a complex program suite such as GAMESS is maintained and developed is considered. Future developments are briefly summarized.
Collapse
Affiliation(s)
- Giuseppe M J Barca
- Research School of Computer Science, Australian National University, Canberra, ACT 2601, Australia
| | - Colleen Bertoni
- Argonne Leadership Computing Facility, Argonne National Laboratory, Lemont, Illinois 60439, USA
| | - Laura Carrington
- EP Analytics, 12121 Scripps Summit Dr. Ste. 130, San Diego, California 92131, USA
| | - Dipayan Datta
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Nuwan De Silva
- Department of Physical and Biological Sciences, Western New England University, Springfield, Massachusetts 01119, USA
| | - J Emiliano Deustua
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, USA
| | - Dmitri G Fedorov
- Research Center for Computational Design of Advanced Functional Materials (CD-FMat), National Institute of Advanced Industrial Science and Technology (AIST), Umezono 1-1-1, Tsukuba 305-8568, Japan
| | - Jeffrey R Gour
- Microsoft, 15590 NE 31st St., Redmond, Washington 98052, USA
| | - Anastasia O Gunina
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Emilie Guidez
- Department of Chemistry, University of Colorado Denver, Denver, Colorado 80217, USA
| | - Taylor Harville
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Stephan Irle
- Computational Science and Engineering Division, Oak Ridge National Laboratory, Oak Ridge, Tennessee 37830, USA
| | - Joe Ivanic
- Advanced Biomedical Computational Science, Frederick National Laboratory for Cancer Research, Frederick, Maryland 21702, USA
| | - Karol Kowalski
- Physical Sciences Division, Battelle, Pacific Northwest National Laboratory, K8-91, P.O. Box 999, Richland, Washington 99352, USA
| | - Sarom S Leang
- EP Analytics, 12121 Scripps Summit Dr. Ste. 130, San Diego, California 92131, USA
| | - Hui Li
- Department of Chemistry, University of Nebraska, Lincoln, Nebraska 68588, USA
| | - Wei Li
- School of Chemistry and Chemical Engineering, Key Laboratory of Mesoscopic Chemistry of Ministry of Education, Institute of Theoretical and Computational Chemistry, Nanjing University, Nanjing 210023, People's Republic of China
| | - Jesse J Lutz
- Center for Computing Research, Sandia National Laboratories, Albuquerque, New Mexico 87185, USA
| | - Ilias Magoulas
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, USA
| | - Joani Mato
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Vladimir Mironov
- Department of Chemistry, Lomonosov Moscow State University, Leninskie Gory 1/3, Moscow 119991, Russian Federation
| | - Hiroya Nakata
- Kyocera Corporation, Research Institute for Advanced Materials and Devices, 3-5-3 Hikaridai Seika-cho, Souraku-gun, Kyoto 619-0237, Japan
| | - Buu Q Pham
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Piotr Piecuch
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, USA
| | - David Poole
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Spencer R Pruitt
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Alistair P Rendell
- Research School of Computer Science, Australian National University, Canberra, ACT 2601, Australia
| | - Luke B Roskop
- Cray Inc., a Hewlett Packard Enterprise Company, 2131 Lindau Ln #1000, Bloomington, Minnesota 55425, USA
| | - Klaus Ruedenberg
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | | | - Michael W Schmidt
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Jun Shen
- Department of Chemistry, Michigan State University, East Lansing, Michigan 48824, USA
| | - Lyudmila Slipchenko
- Department of Chemistry, Purdue University, West Lafayette, Indiana 47907, USA
| | - Masha Sosonkina
- Department of Computational Modeling and Simulation Engineering, Old Dominion University, Norfolk, Virginia 23529, USA
| | - Vaibhav Sundriyal
- Department of Computational Modeling and Simulation Engineering, Old Dominion University, Norfolk, Virginia 23529, USA
| | - Ananta Tiwari
- EP Analytics, 12121 Scripps Summit Dr. Ste. 130, San Diego, California 92131, USA
| | - Jorge L Galvez Vallejo
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Bryce Westheimer
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Marta Włoch
- 530 Charlesina Dr., Rochester, Michigan 48306, USA
| | - Peng Xu
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Federico Zahariev
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory, Iowa State University, Ames, Iowa 50011, USA
| |
Collapse
|
21
|
Pham BQ, Gordon MS. Development of the FMO/RI-MP2 Fully Analytic Gradient Using a Hybrid-Distributed/Shared Memory Programming Model. J Chem Theory Comput 2020; 16:1039-1054. [PMID: 31899632 DOI: 10.1021/acs.jctc.9b01082] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
Abstract
The fully analytic gradient of the second-order Møller-Plesset perturbation theory (MP2) with the resolution-of-the-identity (RI) approximation in the fragment molecular orbital (FMO) framework is derived and implemented using a hybrid multilevel parallel programming model, a combination of the general distributed data interface (GDDI) and the OpenMP API. The FMO/MP2 analytic gradient contains three parts, i.e., the internal fragment component, the electrostatic potential (ESP) component, and the response terms. The RI approximation is applied to the internal fragment MP2 gradient term, whose MP2 densities and monomer MP2 Lagrangians are shared with the ESP and the response terms. The FMO/RI-MP2 analytic gradient implementation is validated against the numerical gradient (with errors ∼10-6-10-5 Hartree/Bohr) and the energy conservation in molecular dynamics (MD) simulations using NVE ensembles. The RI approximation introduces an error of ∼10-5 Hartree/Bohr with a speedup of 4.0-8.0× compared with the currently available GDDI FMO/MP2 gradient. The node linear scaling of the fragmentation framework due to multilevel parallelism is well-preserved and is demonstrated in single-point gradient calculations of large water clusters (e.g., 1120 and 2165 molecules) using 300-800 KNL compute nodes with a parallel efficiency of more than 90%.
Collapse
Affiliation(s)
- Buu Q Pham
- Department of Chemistry and Ames Laboratory , Iowa State University , Ames , Iowa 50011 , United States
| | - Mark S Gordon
- Department of Chemistry and Ames Laboratory , Iowa State University , Ames , Iowa 50011 , United States
| |
Collapse
|
22
|
Abstract
Since the introduction of the fragment molecular orbital method 20 years ago, fragment-based approaches have occupied a small but growing niche in quantum chemistry. These methods decompose a large molecular system into subsystems small enough to be amenable to electronic structure calculations, following which the subsystem information is reassembled in order to approximate an otherwise intractable supersystem calculation. Fragmentation sidesteps the steep rise (with respect to system size) in the cost of ab initio calculations, replacing it with a distributed cost across numerous computer processors. Such methods are attractive, in part, because they are easily parallelizable and therefore readily amenable to exascale computing. As such, there has been hope that distributed computing might offer the proverbial "free lunch" in quantum chemistry, with the entrée being high-level calculations on very large systems. While fragment-based quantum chemistry can count many success stories, there also exists a seedy underbelly of rarely acknowledged problems. As these methods begin to mature, it is time to have a serious conversation about what they can and cannot be expected to accomplish in the near future. Both successes and challenges are highlighted in this Perspective.
Collapse
Affiliation(s)
- John M Herbert
- Department of Chemistry and Biochemistry, The Ohio State University, Columbus, Ohio 43210, USA
| |
Collapse
|