1
|
Andoh Y, Ichikawa SI, Sakashita T, Fujimoto K, Yoshii N, Nagai T, Tang Z, Okazaki S. An exa-scale high-performance molecular dynamics simulation program: MODYLAS. J Chem Phys 2023; 158:2890480. [PMID: 37184018 DOI: 10.1063/5.0144361] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2023] [Accepted: 04/25/2023] [Indexed: 05/16/2023] Open
Abstract
A new version of the highly parallelized general-purpose molecular dynamics (MD) simulation program MODYLAS with high performance on the Fugaku computer was developed. A benchmark test using Fugaku indicated highly efficient communication, single instruction, multiple data (SIMD) processing, and on-cache arithmetic operations. The system's performance deteriorated only slightly, even under high parallelization. In particular, a newly developed minimum transferred data method, requiring a significantly lower amount of data transfer compared to conventional communications, showed significantly high performance. The coordinates and forces of 101 810 176 atoms and the multipole coefficients of the subcells could be distributed to the 32 768 nodes (1 572 864 cores) in 2.3 ms during one MD step calculation. The SIMD effective instruction rates for floating-point arithmetic operations in direct force and fast multipole method (FMM) calculations measured on Fugaku were 78.7% and 31.5%, respectively. The development of a data reuse algorithm enhanced the on-cache processing; the cache miss rate for direct force and FMM calculations was only 2.74% and 1.43%, respectively, on the L1 cache and 0.08% and 0.60%, respectively, on the L2 cache. The modified MODYLAS could complete one MD single time-step calculation within 8.5 ms for the aforementioned large system. Additionally, the program contains numerous functions for material research that enable free energy calculations, along with the generation of various ensembles and molecular constraints.
Collapse
Affiliation(s)
- Yoshimichi Andoh
- National Institute for Materials Science (NIMS), 1-2-1, Sengen, Tsukuba, Ibaraki 305-0047, Japan
| | - Shin-Ichi Ichikawa
- Computational Science Division, Technical Computing Business Unit, Fujitsu Limited, Chiba, Japan
| | - Tatsuya Sakashita
- Center for Quantum Information and Quantum Biology, Osaka University, 1-2, Machikaneyama, Toyonaka, Osaka 560-0043, Japan
| | - Kazushi Fujimoto
- Department of Materials Chemistry, Nagoya University, Chikusa-ku, Nagoya, Aichi 464-8603, Japan
| | - Noriyuki Yoshii
- Center for Computational Science, Graduate School of Engineering, Nagoya University, Nagoya 464-8603, Japan
| | - Tetsuro Nagai
- Department of Chemistry, Faculty of Science, Fukuoka University, 8-19-1, Nanakuma, Jonan-ku, Fukuoka 814-0180, Japan
| | - Zhiye Tang
- Department of Theoretical and Computational Molecular Science, Institute for Molecular Science, National Institutes of Natural Sciences, 38, Nishigonaka, Myodaiji, Okazaki, Aichi 444-8585, Japan
| | - Susumu Okazaki
- Department of Advanced Materials Science, The University of Tokyo, 5-1-5, Kashiwa-no-ha, Kashiwa, Chiba 277-0871, Japan
| |
Collapse
|
2
|
Nagai T, Okazaki S. Global diffusion of hydrogen molecules in the heterogeneous structure of polymer electrolytes for fuel cells: Dynamic Monte Carlo combined with molecular dynamics calculations. J Chem Phys 2022; 157:054502. [DOI: 10.1063/5.0096574] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
Using our recently developed dynamic Monte Carlo (MC) method [Nagai et al., J. Chem Phys. 156, 154506 (2022)], we investigated global diffusion of hydrogen molecules over structural heterogeneities of polymer electrolyte membranes in fuel cells. The three-dimensional position-dependent free energies and the diffusion constants of the hydrogen molecules, required by the present dynamic MC calculations, were taken from our previous study [Nagai et al., J. Chem. Phys. 156, 044507 (2022)] and newly evaluated in this work, respectively. The calculations enabled evaluating the hydrogen dynamics over long-time scales, including global diffusion constants. Based on the calculated global diffusion constants and free energies, the permeability of hydrogen molecules was estimated via the solubility-diffusion model. The estimated values were in good agreement with reported experimental data, thus validating the present methodology. The analysis of the Monte Carlo trajectories indicated that the main permeation paths are located in the polymer and interfacial phases, although the water phase may make a non-negligible contribution to mass transport.
Collapse
Affiliation(s)
- Tetsuro Nagai
- Graduate School of Frontier Sciences, The University of Tokyo Graduate School of Frontier Sciences, Japan
| | - Susumu Okazaki
- Department of Advanced Materials Science, University of Tokyo Graduate School of Frontier Sciences Department of Advanced Materials Science, Japan
| |
Collapse
|
3
|
Nagai T, Fujimoto K, Okazaki S. Three-dimensional free-energy landscape of hydrogen and oxygen molecules in polymer electrolyte membranes: Insight into diffusion paths. J Chem Phys 2022; 156:044507. [DOI: 10.1063/5.0075969] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Affiliation(s)
- Tetsuro Nagai
- Department of Advanced Materials Science, Graduate School of Frontier Sciences, The University of Tokyo, Kashiwa, Japan
| | - Kazushi Fujimoto
- Department of Materials Chemistry, Graduate School of Engineering, Nagoya University, Nagoya, Japan
| | - Susumu Okazaki
- Department of Advanced Materials Science, Graduate School of Frontier Sciences, The University of Tokyo, Kashiwa, Japan
| |
Collapse
|
4
|
Andoh Y, Ichikawa SI, Sakashita T, Yoshii N, Okazaki S. Algorithm to minimize MPI communications in the parallelized fast multipole method combined with molecular dynamics calculations. J Comput Chem 2021; 42:1073-1087. [PMID: 33780021 DOI: 10.1002/jcc.26524] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2020] [Revised: 03/09/2021] [Accepted: 03/11/2021] [Indexed: 11/07/2022]
Abstract
In the era of exascale supercomputers, large-scale, and long-time molecular dynamics (MD) calculations are expected to make breakthroughs in various fields of science and technology. Here, we propose a new algorithm to improve the parallelization performance of message passing interface (MPI)-communication in the MPI-parallelized fast multipole method (FMM) combined with MD calculations under three-dimensional periodic boundary conditions. Our approach enables a drastic reduction in the amount of communication data, including the atomic coordinates and multipole coefficients, both of which are required to calculate the electrostatic interaction by using the FMM. In communications of multipole coefficients, the reduction rate of communication data in the new algorithm relative to the amount of data in the conventional one increases as both the number of FMM levels and the number of MPI processes increase. The aforementioned rate increase could exceed 50% as the number of MPI processes becomes larger for very large systems. The proposed algorithm, named the minimum-transferred data (MTD) method, should enable large-scale and long-time MD calculations to be calculated efficiently, under the condition of massive MPI-parallelization on exascale supercomputers.
Collapse
Affiliation(s)
- Yoshimichi Andoh
- Center for Computational Science, Graduate School of Engineering, Nagoya University, Nagoya, Japan
| | - Shin-Ichi Ichikawa
- Computational Science Division, Technical Computing Business Unit, Fujitsu Limited, Chiba, Japan
| | - Tatsuya Sakashita
- Department of Information and Communication Technology, College of Engineering, Tamagawa University, Machida, Tokyo, Japan
| | - Noriyuki Yoshii
- Center for Computational Science, Graduate School of Engineering, Nagoya University, Nagoya, Japan
- Department of Materials Chemistry, Nagoya University, Nagoya, Japan
| | - Susumu Okazaki
- Department of Materials Chemistry, Nagoya University, Nagoya, Japan
- Department of Advanced Materials Science, The University of Tokyo, Kashiwa, Chiba, Japan
| |
Collapse
|
5
|
Kohnke B, Kutzner C, Grubmüller H. A GPU-Accelerated Fast Multipole Method for GROMACS: Performance and Accuracy. J Chem Theory Comput 2020; 16:6938-6949. [PMID: 33084336 PMCID: PMC7660746 DOI: 10.1021/acs.jctc.0c00744] [Citation(s) in RCA: 49] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
An important and computationally demanding part of molecular dynamics simulations is the calculation of long-range electrostatic interactions. Today, the prevalent method to compute these interactions is particle mesh Ewald (PME). The PME implementation in the GROMACS molecular dynamics package is extremely fast on individual GPU nodes. However, for large scale multinode parallel simulations, PME becomes the main scaling bottleneck as it requires all-to-all communication between the nodes; as a consequence, the number of exchanged messages scales quadratically with the number of involved nodes in that communication step. To enable efficient and scalable biomolecular simulations on future exascale supercomputers, clearly a method with a better scaling property is required. The fast multipole method (FMM) is such a method. As a first step on the path to exascale, we have implemented a performance-optimized, highly efficient GPU FMM and integrated it into GROMACS as an alternative to PME. For a fair performance comparison between FMM and PME, we first assessed the accuracies of the methods for various sets of input parameters. With parameters yielding similar accuracies for both methods, we determined the performance of GROMACS with FMM and compared it to PME for exemplary benchmark systems. We found that FMM with a multipole order of 8 yields electrostatic forces that are as accurate as PME with standard parameters. Further, for typical mixed-precision simulation settings, FMM does not lead to an increased energy drift with multipole orders of 8 or larger. Whereas an ≈50 000 atom simulation system with our FMM reaches only about a third of the performance with PME, for systems with large dimensions and inhomogeneous particle distribution, e.g., aerosol systems with water droplets floating in a vacuum, FMM substantially outperforms PME already on a single node.
Collapse
Affiliation(s)
- Bartosz Kohnke
- Theoretical and Computational Biophysics, Max Planck Institute for Biophysical Chemistry, Am Fassberg 11, 37077 Göttingen, Germany
| | - Carsten Kutzner
- Theoretical and Computational Biophysics, Max Planck Institute for Biophysical Chemistry, Am Fassberg 11, 37077 Göttingen, Germany
| | - Helmut Grubmüller
- Theoretical and Computational Biophysics, Max Planck Institute for Biophysical Chemistry, Am Fassberg 11, 37077 Göttingen, Germany
| |
Collapse
|