The performance of CIPSI on the ground state electronic energy of benzene

Compactification of determinant expansions via transcorrelation

Although selected configuration interaction (SCI) algorithms can tackle much larger Hilbert spaces than the conventional full CI method, the scaling of their computational cost with respect to the system size remains inherently exponential. In addition, inaccuracies in describing the correlation hole at small interelectronic distances lead to the slow convergence of the electronic energy relative to the size of the one-electron basis set. To alleviate these effects, we show that the non-Hermitian, transcorrelated (TC) version of SCI significantly compactifies the determinant space, allowing us to reach a given accuracy with a much smaller number of determinants. Furthermore, we note a significant acceleration in the convergence of the TC-SCI energy as the basis set size increases. The extent of this compression and the energy convergence rate are closely linked to the accuracy of the correlation factor used for the similarity transformation of the Coulombic Hamiltonian. Our systematic investigation of small molecular systems in increasingly large basis sets illustrates the magnitude of these effects.

Selected Configuration Interaction for Resonances

Electronic resonances are metastable states that can decay by electron loss. They are ubiquitous across various fields of science, such as chemistry, physics, and biology. However, current theoretical and computational models for resonances cannot yet rival the level of accuracy achieved by bound-state methodologies. Here, we generalize selected configuration interaction (SCI) to treat resonances by using the complex absorbing potential (CAP) technique. By modifying the selection procedure and the extrapolation protocol of standard SCI, the resulting CAP-SCI method yields resonance positions and widths of full configuration interaction quality. Initial results for the shape resonances of N2- and CO- reveal the important effect of high-order correlation, which shifts the values obtained with CAP-augmented equation-of-motion coupled-cluster with singles and doubles by more than 0.1 eV. The present CAP-SCI approach represents a cornerstone in the development of highly accurate methodologies for resonances.

Reference Energies for Double Excitations: Improvement and Extension

In the realm of photochemistry, the significance of double excitations (also known as doubly excited states), where two electrons are concurrently elevated to higher energy levels, lies in their involvement in key electronic transitions essential in light-induced chemical reactions as well as their challenging nature from the computational theoretical chemistry point of view. Based on state-of-the-art electronic structure methods (such as high-order coupled-cluster, selected configuration interaction, and multiconfigurational methods), we improve and expand our prior set of accurate reference excitation energies for electronic states exhibiting a substantial amount of double excitations [Loos et al. J. Chem. Theory Comput. 2019, 15, 1939]. This extended collection encompasses 47 electronic transitions across 26 molecular systems that we separate into two distinct subsets: (i) 28 "genuine" doubly excited states where the transitions almost exclusively involve doubly excited configurations and (ii) 19 "partial" doubly excited states which exhibit a more balanced character between singly and doubly excited configurations. For each subset, we assess the performance of high-order coupled-cluster (CC3, CCSDT, CC4, and CCSDTQ) and multiconfigurational methods (CASPT2, CASPT3, PC-NEVPT2, and SC-NEVPT2). Using as a probe the percentage of single excitations involved in a given transition (%T1) computed at the CC3 level, we also propose a simple correction that reduces the errors of CC3 by a factor of 3, for both sets of excitations. We hope that this more complete and diverse compilation of double excitations will help future developments of electronic excited-state methodologies.

State-Specific Coupled-Cluster Methods for Excited States

We reexamine ΔCCSD, a state-specific coupled-cluster (CC) with single and double excitations (CCSD) approach that targets excited states through the utilization of non-Aufbau determinants. This methodology is particularly efficient when dealing with doubly excited states, a domain in which the standard equation-of-motion CCSD (EOM-CCSD) formalism falls short. Our goal here to evaluate the effectiveness of ΔCCSD when applied to other types of excited states, comparing its consistency and accuracy with EOM-CCSD. To this end, we report a benchmark on excitation energies computed with the ΔCCSD and EOM-CCSD methods for a set of molecular excited-state energies that encompasses not only doubly excited states but also doublet-doublet transitions and (singlet and triplet) singly excited states of closed-shell systems. In the latter case, we rely on a minimalist version of multireference CC known as the two-determinant CCSD method to compute the excited states. Our data set, consisting of 276 excited states stemming from the quest database [Véril et al., WIREs Comput. Mol. Sci. 2021, 11, e1517], provides a significant base to draw general conclusions concerning the accuracy of ΔCCSD. Except for the doubly excited states, we found that ΔCCSD underperforms EOM-CCSD. For doublet-doublet transitions, the difference between the mean absolute errors (MAEs) of the two methodologies (of 0.10 and 0.07 eV) is less pronounced than that obtained for singly excited states of closed-shell systems (MAEs of 0.15 and 0.08 eV). This discrepancy is largely attributed to a greater number of excited states in the latter set exhibiting multiconfigurational characters, which are more challenging for ΔCCSD. We also found typically small improvements by employing state-specific optimized orbitals.

Rationale for the extrapolation procedure in selected configuration interaction

Selected configuration interaction (SCI) methods have emerged as state-of-the-art methodologies for achieving high accuracy and generating benchmark reference data for ground and excited states in small molecular systems. However, their precision relies heavily on extrapolation procedures to produce a final estimate of the exact result. Using the structure of the exact electronic energy landscape, we provide a rationale for the common linear extrapolation of the variational energy as a function of the second-order perturbative correction. In particular, we demonstrate that the energy gap and the coupling between the so-called internal and external spaces are the key factors determining the rate at which the linear regime is reached. Starting from the first principles, we also derive a new non-linear extrapolation formula that improves the post-processing of data generated from SCI methods and can be applied to both ground- and excited-state energies.

Dynamic and Nondynamic Electron Correlation Energy Decomposition Based on the Node of the Hartree-Fock Slater Determinant

Distinguishing between dynamic and nondynamic electron correlation energy is a fundamental concept in quantum chemistry. It can be challenging to make a clear distinction between the two types of correlation energy or to determine their actual contributions in specific cases using wave function theory. This is because both single-reference and multireference methods cover both types of correlation energy to some extent. Fixed-node diffusion quantum Monte Carlo (FNDMC) accurately covers dynamic correlations, but it is limited in overall accuracy by the node of the trial wave function. We introduce a methodology for partitioning an exact electron correlation energy into its dynamic and nondynamic components. This is accomplished by restricting a ground-state solution from sharing its node with a spin-restricted Hartree-Fock Slater determinant. The FNDMC method is used as a tool to conveniently project out a lowest-energy state obeying such a boundary condition. The proposed approach provides an unambiguous and useful procedure for separating electron correlation energy, as demonstrated on multiple systems, including the He atom, bond breaking of H2, the parametric H2-H2 system, the Be-Ne atomic series with low- and high-spin states for C, N, and O atoms, and small molecules such as BH, HF, and CO at both equilibrium and elongated configurations, respectively.

Reference Vertical Excitation Energies for Transition Metal Compounds

To enrich and enhance the diversity of the quest database of highly accurate excitation energies [Véril, M.; et al. Wiley Interdiscip. Rev.: Comput. Mol. Sci. 2021, 11, e1517], we report vertical transition energies in transition metal compounds. Eleven diatomic molecules with a singlet or doublet ground state containing a fourth-row transition metal (CuCl, CuF, CuH, ScF, ScH, ScO, ScS, TiN, ZnH, ZnO, and ZnS) are considered, and the corresponding excitation energies are computed using high-level coupled-cluster (CC) methods, namely, CC3, CCSDT, CC4, and CCSDTQ, as well as multiconfigurational methods such as CASPT2 and NEVPT2. In many cases, to provide more comprehensive benchmark data, we also provide full configuration interaction estimates computed with the configuration interaction using a perturbative selection made iteratively (CIPSI) method. Based on these calculations, theoretical best estimates of the transition energies are established in both the aug-cc-pVDZ and aug-cc-pVTZ basis sets. This allows us to accurately assess the performance of the CC and multiconfigurational methods for this specific set of challenging transitions. Furthermore, comparisons with experimental data and previous theoretical results are also reported.

Ground- and Excited-State Dipole Moments and Oscillator Strengths of Full Configuration Interaction Quality

We report ground- and excited-state dipole moments and oscillator strengths (computed in different "gauges" or representations) of full configuration interaction (FCI) quality using the selected configuration interaction method known as Configuration Interaction using a Perturbative Selection made Iteratively (CIPSI). Thanks to a set encompassing 35 ground- and excited-state properties computed in 11 small molecules, the present near-FCI estimates allow us to assess the accuracy of high-order coupled-cluster (CC) calculations including up to quadruple excitations. In particular, we show that incrementing the excitation degree of the CC expansion (from CC with singles and doubles (CCSD) to CC with singles, doubles, and triples (CCSDT) or from CCSDT to CC with singles, doubles, triples, and quadruples (CCSDTQ)) reduces the average error with respect to the near-FCI reference values by approximately 1 order of magnitude.

Full Configuration Interaction Excited-State Energies in Large Active Spaces from Subspace Iteration with Repeated Random Sparsification

We present a stable and systematically improvable quantum Monte Carlo (QMC) approach to calculating excited-state energies, which we implement using our fast randomized iteration method for the full configuration interaction problem (FCI-FRI). Unlike previous excited-state quantum Monte Carlo methods, our approach, which is based on an asymmetric variant of subspace iteration, avoids the use of dot products of random vectors and instead relies upon trial vectors to maintain orthogonality and estimate eigenvalues. By leveraging recent advances, we apply our method to calculate ground- and excited-state energies of challenging molecular systems in large active spaces, including the carbon dimer with 8 electrons in 108 orbitals (8e,108o), an oxo-Mn(salen) transition metal complex (28e,28o), ozone (18e,87o), and butadiene (22e,82o). In the majority of these test cases, our approach yields total excited-state energies that agree with those from state-of-the-art methods─including heat-bath CI, the density matrix renormalization group approach, and FCIQMC─to within sub-milliHartree accuracy. In all cases, estimated excitation energies agree to within about 0.1 eV.

Downfolded Configuration Interaction for Chemically Accurate Electron Correlation

A model subspace configuration interaction method is developed to obtain chemically accurate electron correlations by diagonalizing a very compact effective Hamiltonian of a realistic molecule. The construction of the effective Hamiltonian is deterministic and implemented by iteratively building a sufficiently small model subspace comprising local clusters of a small number of Slater determinants. Through the low-rank reciprocal of interaction Hamiltonian, important determinants can be incrementally identified to couple with selected local pairwise clusters and then downfolded into the model subspace. This method avoids direct ordering and selection of the configurations in the entire space. We demonstrate the efficiency and accuracy of this theory for obtaining the near-FCI ground- and excited-state potential energies by benchmarking the C₂ molecule and illustrate its application potential in computing accurate excitation energies of organometallic [Cu(NHC)₂(pyridine)₂]^x+ complexes and other organic molecules of various excitation character.

Reference Energies for Cyclobutadiene: Automerization and Excited States

Cyclobutadiene is a well-known playground for theoretical chemists and is particularly suitable to test ground- and excited-state methods. Indeed, due to its high spatial symmetry, especially at the D_4h square geometry but also in the D_2h rectangular arrangement, the ground and excited states of cyclobutadiene exhibit multiconfigurational characters and single-reference methods, such as standard adiabatic time-dependent density-functional theory (TD-DFT) or standard equation-of-motion coupled cluster (EOM-CC), are notoriously known to struggle in such situations. In this work, using a large panel of methods and basis sets, we provide an extensive computational study of the automerization barrier (defined as the difference between the square and rectangular ground-state energies) and the vertical excitation energies at D_2h and D_4h equilibrium structures. In particular, selected configuration interaction (SCI), multireference perturbation theory (CASSCF, CASPT2, and NEVPT2), and coupled-cluster (CCSD, CC3, CCSDT, CC4, and CCSDTQ) calculations are performed. The spin-flip formalism, which is known to provide a qualitatively correct description of these diradical states, is also tested within TD-DFT (combined with numerous exchange-correlation functionals) and the algebraic diagrammatic construction [ADC(2)-s, ADC(2)-x, and ADC(3)]. A theoretical best estimate is defined for the automerization barrier and for each vertical transition energy.

Removing the Deadwood from DFT/MRCI Wave Functions: The p-DFT/MRCI Method

The combined density functional theory and multireference configuration interaction (DFT/MRCI) method is a powerful tool for the calculation of excited electronic states of large molecules. There exists, however, a large amount of superfluous configurations in a typical DFT/MRCI wave function. We show that this deadwood may be effectively removed using a simple configuration pruning algorithm based on second-order Epstein-Nesbet perturbation theory. The resulting method, which we denote p-DFT/MRCI, is shown to result in orders of magnitude saving in computational timings, while retaining the accuracy of the original DFT/MRCI method.

High-level coupled-cluster energetics by merging moment expansions with selected configuration interaction

Inspired by our earlier semi-stochastic work aimed at converging high-level coupled-cluster (CC) energetics [J. E. Deustua, J. Shen, and P. Piecuch, Phys. Rev. Lett. 119, 223003 (2017) and J. E. Deustua, J. Shen, and P. Piecuch, J. Chem. Phys. 154, 124103 (2021)], we propose a novel form of the CC(P; Q) theory in which the stochastic Quantum Monte Carlo propagations, used to identify dominant higher-than-doubly excited determinants, are replaced by the selected configuration interaction (CI) approach using the perturbative selection made iteratively (CIPSI) algorithm. The advantages of the resulting CIPSI-driven CC(P; Q) methodology are illustrated by a few molecular examples, including the dissociation of F₂ and the automerization of cyclobutadiene, where we recover the electronic energies corresponding to the CC calculations with a full treatment of singles, doubles, and triples based on the information extracted from compact CI wave functions originating from relatively inexpensive Hamiltonian diagonalizations.

Fixed- and Partial-Node Approximations in Slater Determinant Space for Molecules

We present a study of fixed- and partial-node approximations in Slater determinant basis sets, using full configuration interaction quantum Monte Carlo (FCIQMC) to perform sampling. Walker annihilation in the FCIQMC method allows partial-node simulations to be performed, relaxing the nodal constraint to converge to the FCI solution. This is applied to ab initio molecular systems, using symmetry-projected Jastrow mean-field wave functions for complete active space (CAS) problems. Convergence and the sign problem within the partial-node approximation are studied, which is shown to eventually be limited in its use due to the large walker populations required. However, the fixed-node approximation results in an accurate and practical method. We apply these approaches to various molecular systems and active spaces, including ferrocene and acenes. This also provides a test of symmetry-projected Jastrow mean-field wave functions in variational Monte Carlo for a new set of problems. For trans-polyacetylene molecules and acenes, we find that the time to perform a constant number of fixed-node FCIQMC iterations scales as O(N1.44) and O(N1.75), respectively, resulting in an efficient method for CAS-based problems that can be applied accurately to large active spaces.

Accurate full configuration interaction correlation energy estimates for five- and six-membered rings

Following our recent work on the benzene molecule [P.-F. Loos, Y. Damour, and A. Scemama, J. Chem. Phys. 153, 176101 (2020)], motivated by the blind challenge of Eriksen et al. [J. Phys. Chem. Lett. 11, 8922 (2020)] on the same system, we report accurate full configuration interaction (FCI) frozen-core correlation energy estimates for 12 five- and six-membered ring molecules (cyclopentadiene, furan, imidazole, pyrrole, thiophene, benzene, pyrazine, pyridazine, pyridine, pyrimidine, s-tetrazine, and s-triazine) in the standard correlation-consistent double-ζ Dunning basis set (cc-pVDZ). Our FCI correlation energy estimates, with an estimated error smaller than 1 millihartree, are based on energetically optimized-orbital selected configuration interaction calculations performed with the configuration interaction using a perturbative selection made iteratively algorithm. Having at our disposal these accurate reference energies, the respective performance and convergence properties of several popular and widely used families of single-reference quantum chemistry methods are investigated. In particular, we study the convergence properties of (i) the Møller-Plesset perturbation series up to fifth-order (MP2, MP3, MP4, and MP5), (ii) the iterative approximate coupled-cluster series CC2, CC3, and CC4, and (iii) the coupled-cluster series CCSD, CCSDT, and CCSDTQ. The performance of the ground-state gold standard CCSD(T) as well as the completely renormalized CC model, CR-CC(2,3), is also investigated. We show that MP4 provides an interesting accuracy/cost ratio, while MP5 systematically worsens the correlation energy estimates. In addition, CC3 outperforms CCSD(T) and CR-CC(2,3), as well as its more expensive parent CCSDT. A similar trend is observed for the methods including quadruple excitations, where the CC4 model is shown to be slightly more accurate than CCSDTQ, both methods providing correlation energies within 2 millihartree of the FCI limit.

Reinforcement Learning Configuration Interaction

Selected configuration interaction (sCI) methods exploit the sparsity of the full configuration interaction (FCI) wave function, yielding significant computational savings and wave function compression without sacrificing the accuracy. Despite recent advances in sCI methods, the selection of important determinants remains an open problem. We explore the possibility of utilizing reinforcement learning approaches to solve the sCI problem. By mapping the configuration interaction problem onto a sequential decision-making process, the agent learns on-the-fly which determinants to include and which to ignore, yielding a compressed wave function at near-FCI accuracy. This method, which we call reinforcement-learned configuration interaction, adds another weapon to the sCI arsenal and highlights how reinforcement learning approaches can potentially help solve challenging problems in electronic structure theory.

Is Externally Corrected Coupled Cluster Always Better Than the Underlying Truncated Configuration Interaction?

The short answer to the question in the title is "no". We identify classes of truncated configuration interaction (CI) wave functions for which the externally corrected coupled-cluster (ec-CC) approach using the three-body (T₃) and four-body (T₄) components of the cluster operator extracted from CI does not improve the results of the underlying CI calculations. Implications of our analysis, illustrated by numerical examples, for the ec-CC computations using truncated and selected CI methods are discussed. We also introduce a novel ec-CC approach using the T₃ and T₄ amplitudes obtained with the selected CI scheme abbreviated as CIPSI, correcting the resulting energies for the missing T₃ correlations not captured by CIPSI with the help of moment expansions similar to those employed in the completely renormalized CC methods.

How accurate are EOM-CC4 vertical excitation energies?

We report the first investigation of the performance of EOM-CC4-an approximate equation-of-motion coupled-cluster model, which includes iterative quadruple excitations-for vertical excitation energies in molecular systems. By considering a set of 28 excited states in 10 small molecules for which we have computed CC with singles, doubles, triples, quadruples, and pentuples and full configuration interaction reference energies, we show that, in the case of excited states with a dominant contribution from the single excitations, CC4 yields excitation energies with sub-kJ mol^-1 accuracy (i.e., error below 0.01 eV), in very close agreement with its more expensive CC with singles, doubles, triples, and quadruples parent. Therefore, if one aims at high accuracy, CC4 stands as a highly competitive approximate method to model molecular excited states, with a significant improvement over both CC3 and CC with singles, doubles, and triples. Our results also evidence that, although the same qualitative conclusions hold, one cannot reach the same level of accuracy for transitions with a dominant contribution from the double excitations.

Near-Exact CASSCF-Level Geometry Optimization with a Large Active Space using Adaptive Sampling Configuration Interaction Self-Consistent Field Corrected with Second-Order Perturbation Theory (ASCI-SCF-PT2)

An accurate description of electron correlation is one of the most challenging problems in quantum chemistry. The exact electron correlation can be obtained by means of full configuration interaction (FCI). A simple strategy for approximating FCI at a reduced computational cost is selected CI (SCI), which diagonalizes the Hamiltonian within only the chosen configuration space. Recovery of the contributions of the remaining configurations is possible with second-order perturbation theory. Here, we apply adaptive sampling configuration interaction (ASCI) combined with molecular orbital optimizations (ASCI-SCF) corrected with second-order perturbation theory (ASCI-SCF-PT2) for geometry optimization by implementing the analytical nuclear gradient algorithm for ASCI-PT2 with the Z-vector (Lagrangian) formalism. We demonstrate that for phenalenyl radicals and anthracene, optimized geometries and the number of unpaired electrons can be obtained at nearly the CASSCF accuracy by incorporating PT2 corrections and extrapolating them. We demonstrate the current algorithm's utility for optimizing the equilibrium geometries and electronic structures of six-ring-fused polycyclic aromatic hydrocarbons and 4-periacene.

Revealing the nature of electron correlation in transition metal complexes with symmetry breaking and chemical intuition

In this work, we provide a nuanced view of electron correlation in the context of transition metal complexes, reconciling computational characterization via spin and spatial symmetry breaking in single-reference methods with qualitative concepts from ligand-field and molecular orbital theories. These insights provide the tools to reliably diagnose the multi-reference character, and our analysis reveals that while strong (i.e., static) correlation can be found in linear molecules (e.g., diatomics) and weakly bound and antiferromagnetically coupled (monometal-noninnocent ligand or multi-metal) complexes, it is rarely found in the ground-states of mono-transition-metal complexes. This leads to a picture of static correlation that is no more complex for transition metals than it is, e.g., for organic biradicaloids. In contrast, the ability of organometallic species to form more complex interactions, involving both ligand-to-metal σ-donation and metal-to-ligand π-backdonation, places a larger burden on a theory's treatment of dynamic correlation. We hypothesize that chemical bonds in which inter-electron pair correlation is non-negligible cannot be adequately described by theories using MP2 correlation energies and indeed find large errors vs experiment for carbonyl-dissociation energies from double-hybrid density functionals. A theory's description of dynamic correlation (and to a less important extent, delocalization error), which affects relative spin-state energetics and thus spin symmetry breaking, is found to govern the efficacy of its use to diagnose static correlation.

The Shape of Full Configuration Interaction to Come

We present a Perspective on what the future holds for full configuration interaction (FCI) theory, with an emphasis on conceptual rather than technical details. Upon revisiting the early history of FCI, a number of its key contemporary approximations are compared on as equal a footing as possible, using a recent blind challenge on the benzene molecule as a testbed [Eriksen et al., J. Phys. Chem. Lett., 2020 11, 8922]. In the process, we review the scope of applications for which FCI continues to prove indispensable, and the required traits in terms of robustness, efficacy, and reliability its modern approximations must satisfy are discussed. We close by conveying a number of general observations on the merits offered by the state-of-the-art alongside some of the challenges still faced to this day. While the field has altogether seen immense progress over the years-the past decade, in particular-it remains clear that our community as a whole has a substantial way to go in enhancing the overall applicability of near-exact electronic structure theory for systems of general composition and increasing size.

Towards Near-Exact Solutions of Molecular Electronic Structure: Full Coupled-Cluster Reduction with a Second-Order Perturbative Correction

We introduce a new augmented adaptation of the recently developed full coupled-cluster reduction (FCCR) with a second-order perturbative correction, abbreviated as FCCR(2). FCCR is a selected coupled-cluster expansion aimed at optimally reducing the excitation manifold and commutator expansions for high-rank excitations for obtaining accurate solutions of the electronic Schödinger equation in a size-extensive manner. The present FCCR(2) enables estimating the residual correlation of FCCR by the second-order perturbative correction E⁽²⁾ from the complementary space of the FCCR projection manifold. The linear relationship between E⁽²⁾ and the energy of FCCR(2) allows accurate estimates of near-exact energies for a wide variety of molecules with strong electron correlation. The potential of the method is demonstrated using challenging cases, the ground-state electronic energy of the benzene molecule in equilibrium and stretched geometries, and the isomerization energy of the transition metal complex [Cu(NH₃)]₂O₂²⁺.