1
|
Woo J, Youn Kim W, Choi S. Efficient Shift-and-Invert Preconditioning for Multi-GPU Accelerated Density Functional Calculations. J Chem Theory Comput 2024; 20:7443-7452. [PMID: 39190438 PMCID: PMC11391578 DOI: 10.1021/acs.jctc.4c00721] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/28/2024]
Abstract
To accelerate the iterative diagonalization of electronic structure calculations, we propose a new inexact shift-and-invert (ISI) preconditioning method. The key idea is to improve shift values in the ISI preconditioning to be closer to the exact eigenvalues, leading to a significant boost in the convergence speed of the iterative diagonalization. Furthermore, we adopted a preconditioned conjugate gradient solver to rapidly evaluate an inversion process. Finally, we accelerated overall processes, including the proposed modification, with state-of-the-art graphical processing units (GPUs) and assessed its parallel efficiency with real-space density functional calculations of 1D, 2D, and 3D periodic systems. Our method attains both fast diagonalization convergence and high multi-GPU parallel efficiency. This is evident from the fact that single-point density functional calculations for hundreds of atom systems can be done in approximately 10 s using 8 GPUs. The proposed method can be generally applied to any electronic structure calculation methods involving large-scale diagonalizations.
Collapse
Affiliation(s)
- Jeheon Woo
- Department of Chemistry, KAIST, 291 Daehak-ro, Daejeon, Yuseong-gu 34141, Republic of Korea
| | - Woo Youn Kim
- Department of Chemistry, KAIST, 291 Daehak-ro, Daejeon, Yuseong-gu 34141, Republic of Korea
| | - Sunghwan Choi
- Department of Chemistry, Inha University, 100 Inha-ro, Incheon, Michuhol-gu 22212, Republic of Korea
| |
Collapse
|
2
|
Sethio D, Azzopardi E, Fdez. Galván I, Lindh R. A Story of Three Levels of Sophistication in SCF/KS-DFT Orbital Optimization Procedures. J Phys Chem A 2024; 128:2472-2486. [PMID: 38483190 PMCID: PMC10983011 DOI: 10.1021/acs.jpca.3c07647] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2023] [Revised: 02/23/2024] [Accepted: 02/23/2024] [Indexed: 04/04/2024]
Abstract
In this work, three versions of self-consistent field/Kohn-Sham density functional theory (SCF/KS-DFT) orbital optimization are described and benchmarked. The methods are a modified version of the geometry version of the direct inversion in the iterative subspace approach (which we call r-GDIIS), the modified restricted step rational function optimization method (RS-RFO), and the novel subspace gradient-enhanced Kriging method combined with restricted variance optimization (S-GEK/RVO). The modifications introduced are aimed at improving the robustness and computational scaling of the procedures. In particular, the subspace approach in S-GEK/RVO allows the application to SCF/KS-DFT optimization of a machine learning technique that has proven to be successful in geometry optimizations. The performance of the three methods is benchmarked for a large number of small- to medium-sized organic molecules, at equilibrium structures and close to a transition state, and a second set of molecules containing closed- and open-shell transition metals. The results indicate the importance of the resetting technique in boosting the performance of the r-GDIIS procedure. Moreover, it is demonstrated that already at the inception of the subspace version of GEK to optimize SCF wave functions, it displays superior and robust convergence properties as compared to those of the standard state-of-the-art SCF/KS-DFT optimization methods.
Collapse
Affiliation(s)
- Daniel Sethio
- Department
of Chemistry—BMC, Uppsala University, P.O. Box 576, SE-75123 Uppsala, Sweden
- Department
of Chemistry—Ångström, Uppsala University, P.O. Box 538, SE-75121 Uppsala, Sweden
| | - Emily Azzopardi
- Department
of Chemistry—BMC, Uppsala University, P.O. Box 576, SE-75123 Uppsala, Sweden
| | - Ignacio Fdez. Galván
- Department
of Chemistry—BMC, Uppsala University, P.O. Box 576, SE-75123 Uppsala, Sweden
| | - Roland Lindh
- Department
of Chemistry—BMC, Uppsala University, P.O. Box 576, SE-75123 Uppsala, Sweden
- Uppsala
Center for Computational Chemistry (UC3), Uppsala University, P.O. Box 576, SE-751 23 Uppsala, Sweden
| |
Collapse
|
3
|
Slattery SA, Surjuse KA, Peterson CC, Penchoff DA, Valeev EF. Economical quasi-Newton unitary optimization of electronic orbitals. Phys Chem Chem Phys 2024; 26:6557-6573. [PMID: 38329140 DOI: 10.1039/d3cp05557d] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/09/2024]
Abstract
We present an efficient quasi-Newton orbital solver optimized to reduce the number of gradient evaluations and other computational steps of comparable cost. The solver optimizes orthogonal orbitals by sequences of unitary rotations generated by the (preconditioned) limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm equipped with trust-region step restriction. The low-rank structure of the L-BFGS inverse Hessian is exploited when solving the trust-region problem. The efficiency of the proposed "Quasi-Newton Unitary Optimization with Trust-Region" (QUOTR) solver is compared to that of the standard Roothaan-Hall approach accelerated by the Direct Inversion of Iterative Subspace (DIIS), and other exact and approximate Newton solvers for mean-field (Hartree-Fock and Kohn-Sham) problems.
Collapse
Affiliation(s)
| | | | - Charles C Peterson
- Office of Advanced Research Computing, University of California, Los Angeles, CA 90095, USA
| | - Deborah A Penchoff
- UT Innovative Computing Laboratory, University of Tennessee, Knoxville, TN 37996, USA
| | - Edward F Valeev
- Department of Chemistry, Virginia Tech, Blacksburg, VA 24061, USA.
| |
Collapse
|
4
|
Galvez Vallejo JL, Snowdon C, Stocks R, Kazemian F, Yan Yu FC, Seidl C, Seeger Z, Alkan M, Poole D, Westheimer BM, Basha M, De La Pierre M, Rendell A, Izgorodina EI, Gordon MS, Barca GMJ. Toward an extreme-scale electronic structure system. J Chem Phys 2023; 159:044112. [PMID: 37497819 DOI: 10.1063/5.0156399] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Accepted: 07/03/2023] [Indexed: 07/28/2023] Open
Abstract
Electronic structure calculations have the potential to predict key matter transformations for applications of strategic technological importance, from drug discovery to material science and catalysis. However, a predictive physicochemical characterization of these processes often requires accurate quantum chemical modeling of complex molecular systems with hundreds to thousands of atoms. Due to the computationally demanding nature of electronic structure calculations and the complexity of modern high-performance computing hardware, quantum chemistry software has historically failed to operate at such large molecular scales with accuracy and speed that are useful in practice. In this paper, novel algorithms and software are presented that enable extreme-scale quantum chemistry capabilities with particular emphasis on exascale calculations. This includes the development and application of the multi-Graphics Processing Unit (GPU) library LibCChem 2.0 as part of the General Atomic and Molecular Electronic Structure System package and of the standalone Extreme-scale Electronic Structure System (EXESS), designed from the ground up for scaling on thousands of GPUs to perform high-performance accurate quantum chemistry calculations at unprecedented speed and molecular scales. Among various results, we report that the EXESS implementation enables Hartree-Fock/cc-pVDZ plus RI-MP2/cc-pVDZ/cc-pVDZ-RIFIT calculations on an ionic liquid system with 623 016 electrons and 146 592 atoms in less than 45 min using 27 600 GPUs on the Summit supercomputer with a 94.6% parallel efficiency.
Collapse
Affiliation(s)
| | - Calum Snowdon
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Ryan Stocks
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Fazeleh Kazemian
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Fiona Chuo Yan Yu
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Christopher Seidl
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| | - Zoe Seeger
- School of Chemistry, Monash University, Clayton 3800, VIC, Australia
| | - Melisa Alkan
- Department of Chemistry, Iowa State University, Ames, Iowa 50011-3111, USA
| | - David Poole
- School of Chemistry and Biochemistry, Georgia Institute of Technology, Atlanta, Georgia 30332, USA
| | - Bryce M Westheimer
- Department of Chemistry, Iowa State University, Ames, Iowa 50011-3111, USA
| | - Mehaboob Basha
- Pawsey Supercomputing Research Centre, Kensington, WA 6151, Australia
| | | | - Alistair Rendell
- College of Science and Engineering, Flinders University, Adelaide, SA 5042, Australia
| | | | | | - Giuseppe M J Barca
- School of Computing, Australian National University, Canberra 2601, ACT, Australia
| |
Collapse
|
5
|
Manathunga M, Aktulga HM, Götz AW, Merz KM. Quantum Mechanics/Molecular Mechanics Simulations on NVIDIA and AMD Graphics Processing Units. J Chem Inf Model 2023; 63:711-717. [PMID: 36720086 DOI: 10.1021/acs.jcim.2c01505] [Citation(s) in RCA: 17] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
Abstract
We have ported and optimized the graphics processing unit (GPU)-accelerated QUICK and AMBER-based ab initio quantum mechanics/molecular mechanics (QM/MM) implementation on AMD GPUs. This encompasses the entire Fock matrix build and force calculation in QUICK including one-electron integrals, two-electron repulsion integrals, exchange-correlation quadrature, and linear algebra operations. General performance improvements to the QUICK GPU code are also presented. Benchmarks carried out on NVIDIA V100 and AMD MI100 cards display similar performance on both hardware for standalone HF/DFT calculations with QUICK and QM/MM molecular dynamics simulations with QUICK/AMBER. Furthermore, with respect to the QUICK/AMBER release version 21, significant speedups are observed for QM/MM molecular dynamics simulations. This significantly increases the range of scientific problems that can be addressed with open-source QM/MM software on state-of-the-art computer hardware.
Collapse
Affiliation(s)
- Madushanka Manathunga
- Department of Chemistry and Department of Biochemistry and Molecular Biology, Michigan State University, East Lansing, Michigan48824-1322, United States
| | - Hasan Metin Aktulga
- Department of Computer Science and Engineering, Michigan State University, East Lansing, Michigan48824-1322, United States
| | - Andreas W Götz
- San Diego Supercomputer Center, University of California San Diego, La Jolla, California92093-0505, United States
| | - Kenneth M Merz
- Department of Chemistry and Department of Biochemistry and Molecular Biology, Michigan State University, East Lansing, Michigan48824-1322, United States
| |
Collapse
|