On the Efficient Evaluation of the Exchange Correlation Potential on Graphics Processing Unit Clusters

Accelerating Relativistic Exact-Two-Component Density Functional Theory Calculations with Graphical Processing Units

Numerical integration of the exchange-correlation potential is an inherently parallel problem that can be significantly accelerated by graphical processing units (GPUs). In this Letter, we present the first implementation of GPU-accelerated exchange-correlation potential in the GauXC library for relativistic, 2-component density functional theory. By benchmarking against copper, silver, and gold coinage metal clusters, we demonstrate the speed and efficiency of our implementation, achieving significant speedup compared to CPU-based calculations. One GPU card provides computational power equivalent to roughly 400 CPU cores in the context of this work. The speedup further increases for larger systems, highlighting the potential of our approach for future, more computationally demanding simulations. Our implementation supports arbitrary angular momentum basis functions, enabling the simulation of systems with heavy elements and providing substantial speedup to relativistic electronic structure calculations. This advancement paves the way for more efficient and extensive computational studies in the field of density functional theory.

A modular, composite framework for the utilization of reduced-scaling Coulomb and exchange construction algorithms: Design and implementation

Multiple algorithms exist for calculating Coulomb (J) or exchange (K) contributions to Fock-like matrices, and it is beneficial to develop a framework that allows the seamless integration and combination of different J and K construction algorithms. In Psi4, we have implemented the "CompositeJK" formalism for this purpose. CompositeJK allows for the combination of any J and K construction algorithms for any quantum chemistry method formulated in terms of J-like or K-like matrices (including, but not limited to, Hartree-Fock and density functional theory) in a highly modular and intuitive fashion, which is simple to utilize for both developers and users. Using the CompositeJK framework, Psi4 was interfaced to the sn-LinK implementation in the GauXC library, adding the first instance of noncommercial graphics processing unit (GPU) support for the construction of Fock matrix elements to Psi4. On systems with hundreds of atoms, the interface to the CPU sn-LinK implementation displays a higher performance than all the alternative JK construction methods available in Psi4, with up to x2.8 speedups compared to existing Psi4JK implementations. The GPU sn-LinK implementation, harnessing the power of GPUs, improves the observed performance gains to up to x7.0.

3-center and 4-center 2-particle Gaussian AO integrals on modern accelerated processors

We report an implementation of the McMurchie-Davidson (MD) algorithm for 3-center and 4-center 2-particle integrals over Gaussian atomic orbitals (AOs) with low and high angular momenta l and varying degrees of contraction for graphical processing units (GPUs). This work builds upon our recent implementation of a matrix form of the MD algorithm that is efficient for GPU evaluation of 4-center 2-particle integrals over Gaussian AOs of high angular momenta (l ≥ 4) [A. Asadchev and E. F. Valeev, J. Phys. Chem. A 127, 10889-10895 (2023)]. The use of unconventional data layouts and three variants of the MD algorithm allow for the evaluation of integrals with double precision and sustained performance between 25% and 70% of the theoretical hardware peak. Performance assessment includes integrals over AOs with l ≤ 6 (a higher l is supported). Preliminary implementation of the Hartree-Fock exchange operator is presented and assessed for computations with up to a quadruple-zeta basis and more than 20 000 AOs. The corresponding C++ code is part of the experimental open-source LibintX library available at https://github.com/ValeevGroup/libintx.

CMaize: Simplifying inter-package modularity from the build up

There is a growing desire for inter-package modularity within the chemistry software community to reuse encapsulated code units across a variety of software packages. Most comprehensive efforts at achieving inter-package modularity will quickly run afoul of a very practical problem, being able to cohesively build the modules. Writing and maintaining build systems has long been an issue for many scientific software packages that rely on compiled languages such as C/C++. The push for inter-package modularity compounds this issue by additionally requiring binary artifacts from disparate developers to interoperate at a binary level. Thankfully, the de facto build tool for C/C++, CMake, is more than capable of supporting the myriad of edge cases that complicate writing robust build systems. Unfortunately, writing and maintaining a robust CMake build system can be a laborious endeavor because CMake provides few abstractions to aid the developer. The need to significantly simplify the process of writing robust CMake-based build systems, especially in inter-package builds, motivated us to write CMaize. In addition to describing the architecture and design of CMaize, the article also demonstrates how CMaize is used in production-level software.

A Perspective on Sustainable Computational Chemistry Software Development and Integration

The power of quantum chemistry to predict the ground and excited state properties of complex chemical systems has driven the development of computational quantum chemistry software, integrating advances in theory, applied mathematics, and computer science. The emergence of new computational paradigms associated with exascale technologies also poses significant challenges that require a flexible forward strategy to take full advantage of existing and forthcoming computational resources. In this context, the sustainability and interoperability of computational chemistry software development are among the most pressing issues. In this perspective, we discuss software infrastructure needs and investments with an eye to fully utilize exascale resources and provide unique computational tools for next-generation science problems and scientific discoveries.

Distributed memory, GPU accelerated Fock construction for hybrid, Gaussian basis density functional theory

With the growing reliance of modern supercomputers on accelerator-based architecture such a graphics processing units (GPUs), the development and optimization of electronic structure methods to exploit these massively parallel resources has become a recent priority. While significant strides have been made in the development GPU accelerated, distributed memory algorithms for many modern electronic structure methods, the primary focus of GPU development for Gaussian basis atomic orbital methods has been for shared memory systems with only a handful of examples pursing massive parallelism. In the present work, we present a set of distributed memory algorithms for the evaluation of the Coulomb and exact exchange matrices for hybrid Kohn-Sham DFT with Gaussian basis sets via direct density-fitted (DF-J-Engine) and seminumerical (sn-K) methods, respectively. The absolute performance and strong scalability of the developed methods are demonstrated on systems ranging from a few hundred to over one thousand atoms using up to 128 NVIDIA A100 GPUs on the Perlmutter supercomputer.

PluginPlay: Enabling exascale scientific software one module at a time

For many computational chemistry packages, being able to efficiently and effectively scale across an exascale cluster is a heroic feat. Collective experience from the Department of Energy's Exascale Computing Project suggests that achieving exascale performance requires far more planning, design, and optimization than scaling to petascale. In many cases, entire rewrites of software are necessary to address fundamental algorithmic bottlenecks. This in turn requires a tremendous amount of resources and development time, resources that cannot reasonably be afforded by every computational science project. It thus becomes imperative that computational science transition to a more sustainable paradigm. Key to such a paradigm is modular software. While the importance of modular software is widely recognized, what is perhaps not so widely appreciated is the effort still required to leverage modular software in a sustainable manner. The present manuscript introduces PluginPlay, https://github.com/NWChemEx-Project/PluginPlay, an inversion-of-control framework designed to facilitate developing, maintaining, and sustaining modular scientific software packages. This manuscript focuses on the design aspects of PluginPlay and how they specifically influence the performance of the resulting package. Although, PluginPlay serves as the framework for the NWChemEx package, PluginPlay is not tied to NWChemEx or even computational chemistry. We thus anticipate PluginPlay to prove to be a generally useful tool for a number of computational science packages looking to transition to the exascale.

Quantum Mechanics/Molecular Mechanics Simulations on NVIDIA and AMD Graphics Processing Units

We have ported and optimized the graphics processing unit (GPU)-accelerated QUICK and AMBER-based ab initio quantum mechanics/molecular mechanics (QM/MM) implementation on AMD GPUs. This encompasses the entire Fock matrix build and force calculation in QUICK including one-electron integrals, two-electron repulsion integrals, exchange-correlation quadrature, and linear algebra operations. General performance improvements to the QUICK GPU code are also presented. Benchmarks carried out on NVIDIA V100 and AMD MI100 cards display similar performance on both hardware for standalone HF/DFT calculations with QUICK and QM/MM molecular dynamics simulations with QUICK/AMBER. Furthermore, with respect to the QUICK/AMBER release version 21, significant speedups are observed for QM/MM molecular dynamics simulations. This significantly increases the range of scientific problems that can be addressed with open-source QM/MM software on state-of-the-art computer hardware.

Fermi.jl: A Modern Design for Quantum Chemistry

Approximating molecular wave functions involves heavy numerical effort; therefore, codes for such tasks are written completely or partially in efficient languages such as C, C++, and Fortran. While these tools are dominant throughout quantum chemistry packages, the efficient development of new methods is often hindered by the complexity associated with code development. In order to ameliorate this scenario, some software packages take a dual approach where a simpler, higher-level language, such as Python, substitutes the traditional ones wherever performance is not critical. Julia is a novel, dynamically typed, programming language that aims to solve this two-language problem. It gained attention because of its modern and intuitive design, while still being highly optimized to compete with "low-level" languages. Recently, some chemistry-related projects have emerged exploring the capabilities of Julia. Herein, we introduce the quantum chemistry package Fermi.jl, which contains the first implementations of post-Hartree-Fock methods written in Julia. Its design makes use of many Julia core features, including multiple dispatch, metaprogramming, and interactive usage. Fermi.jl is a modular package, where new methods and implementations can be easily added to the existing code. Furthermore, it is designed to maximize code reusability by relying on general functions with specialized methods for particular cases. The feasibility of the project is explored through evaluating the performance of popular ab initio methods. It is our hope that this project motivates the usage of Julia within the community and brings new contributions into Fermi.jl.

Harnessing the Power of Multi-GPU Acceleration into the Quantum Interaction Computational Kernel Program

We report a new multi-GPU capable ab initio Hartree-Fock/density functional theory implementation integrated into the open source QUantum Interaction Computational Kernel (QUICK) program. Details on the load balancing algorithms for electron repulsion integrals and exchange correlation quadrature across multiple GPUs are described. Benchmarking studies carried out on up to four GPU nodes, each containing four NVIDIA V100-SXM2 type GPUs demonstrate that our implementation is capable of achieving excellent load balancing and high parallel efficiency. For representative medium to large size protein/organic molecular systems, the observed parallel efficiencies remained above 82% for the Kohn-Sham matrix formation and above 90% for nuclear gradient calculations. The accelerations on NVIDIA A100, P100, and K80 platforms also have realized parallel efficiencies higher than 68% in all tested cases, paving the way for large-scale ab initio electronic structure calculations with QUICK.