A Generalized Variational Principle with Applications to Excited State Mean Field Theory

State-Specific Coupled-Cluster Methods for Excited States

We reexamine ΔCCSD, a state-specific coupled-cluster (CC) with single and double excitations (CCSD) approach that targets excited states through the utilization of non-Aufbau determinants. This methodology is particularly efficient when dealing with doubly excited states, a domain in which the standard equation-of-motion CCSD (EOM-CCSD) formalism falls short. Our goal here to evaluate the effectiveness of ΔCCSD when applied to other types of excited states, comparing its consistency and accuracy with EOM-CCSD. To this end, we report a benchmark on excitation energies computed with the ΔCCSD and EOM-CCSD methods for a set of molecular excited-state energies that encompasses not only doubly excited states but also doublet-doublet transitions and (singlet and triplet) singly excited states of closed-shell systems. In the latter case, we rely on a minimalist version of multireference CC known as the two-determinant CCSD method to compute the excited states. Our data set, consisting of 276 excited states stemming from the quest database [Véril et al., WIREs Comput. Mol. Sci. 2021, 11, e1517], provides a significant base to draw general conclusions concerning the accuracy of ΔCCSD. Except for the doubly excited states, we found that ΔCCSD underperforms EOM-CCSD. For doublet-doublet transitions, the difference between the mean absolute errors (MAEs) of the two methodologies (of 0.10 and 0.07 eV) is less pronounced than that obtained for singly excited states of closed-shell systems (MAEs of 0.15 and 0.08 eV). This discrepancy is largely attributed to a greater number of excited states in the latter set exhibiting multiconfigurational characters, which are more challenging for ΔCCSD. We also found typically small improvements by employing state-specific optimized orbitals.

Aufbau Suppressed Coupled Cluster Theory for Electronically Excited States

We introduce an approach to improve single-reference coupled cluster theory in settings where the Aufbau determinant is absent from or plays only a small role in the true wave function. Using a de-excitation operator that can be efficiently hidden within a similarity transform, we create a coupled cluster wave function in which de-excitations work to suppress the Aufbau determinant and produce wave functions dominated by other determinants. Thanks to an invertible and fully exponential form, the approach is systematically improvable, size consistent, size extensive, and, interestingly, size intensive in a granular way that should make the adoption of some ground state techniques, such as local correlation, relatively straightforward. In this initial study, we apply the general formalism to create a state-specific method for orbital-relaxed, singly excited states. We find that this approach matches the accuracy of similar-cost equation-of-motion methods in valence excitations while offering improved accuracy for charge transfer states. We also find the approach to be more accurate than excited-state-specific perturbation theory in both types of states.

Concurrence distribution in excited states of the one-dimensional spin-1/2 transverse-field XY model: Two different regions

We investigate the variation of concurrence in a spin-1/2 transverse field XY chain system in an excited state. Initially, we precisely solve the eigenvalue problem of the system Hamiltonian using the fermionization technique. Subsequently, we calculate the concurrence between nearest-neighbor pairs of spins in all excited states with higher energy than the ground state. Below the factorized field, denoted as h_{f}=sqrt[J^{2}-(Jδ)^{2}], we find no pairwise entanglement between nearest neighbors in excited states. At the factorized field, corresponding to a factorized state, we observe weak concurrence in very low energy states. Beyond h_{f}, the concurrence strengthens, entangling all excited states. The density of entangled states peaks at the center of the excited spectrum. Additionally, the distribution of concurrence reveals that the midpoint of the nonzero concurrence range harbors the most entangled excited states.

Seniority and Hierarchy Configuration Interaction for Radicals and Excited States

Hierarchy configuration interaction (hCI) has recently been introduced as an alternative configuration interaction (CI) route combining excitation degree and seniority number and has been shown to efficiently recover both dynamic and static correlations for closed-shell molecular systems [ J. Phys. Chem. Lett. 2022, 13, 4342]. Here we generalize hCI for an arbitrary reference determinant, allowing calculations for radicals and excited states in a state-specific way. We gauge this route against excitation-based CI (eCI) and seniority-based CI (sCI) by evaluating how different ground-state properties of radicals converge to the full CI limit. We find that hCI outperforms or matches eCI, whereas sCI is far less accurate, in line with previous observations for closed-shell molecules. Employing second-order Epstein-Nesbet (EN2) perturbation theory as a correction significantly accelerates the convergence of hCI and eCI. We further explore various hCI and sCI models to calculate the excitation energies of closed- and open-shell systems. Our results underline that the choice of both the reference determinant and the set of orbitals drives the fine balance between correlation of ground and excited states. State-specific hCI2 and higher-order models perform similarly to their eCI counterparts, whereas lower orders of hCI deliver poor results unless supplemented by the EN2 correction, which substantially improves their accuracy. In turn, sCI1 produces decent excitation energies for radicals, encouraging the development of related seniority-based coupled-cluster methods.

Exact Excited-State Functionals of the Asymmetric Hubbard Dimer

The exact functionals associated with the (singlet) ground state and the two singlet excited states of the asymmetric Hubbard dimer at half-filling are calculated using both Levy's constrained search and Lieb's convex formulation. While the ground-state functional is, as is commonly known, a convex function with respect to the density, the functional associated with the doubly excited state is found to be concave. Also, because the density-potential mapping associated with the first excited state is noninvertible, its "functional" is a partial, multivalued function composed of one concave and one convex branch that correspond to two separate domains of the external potential. Remarkably, it is found that, although the one-to-one mapping between density and external potential may not apply (as in the case of the first excited state), each state-specific energy and corresponding universal functional are "functions" whose derivatives are each other's inverse, just as in the ground state formalism.

Excited-State-Specific Pseudoprojected Coupled-Cluster Theory

We present an excited-state-specific coupled-cluster approach in which both the molecular orbitals and cluster amplitudes are optimized for an individual excited state. The theory is formulated via a pseudoprojection of the traditional coupled-cluster wavefunction that allows correlation effects to be introduced atop an excited-state mean field starting point. The approach shares much in common with ground-state CCSD, including size extensivity and an N6 cost scaling. Preliminary numerical tests show that, when augmented with N5 cost perturbative corrections for key terms, the method can improve over excited-state-specific second-order perturbation theory in valence, charge transfer, and Rydberg states.

Excited States, Symmetry Breaking, and Unphysical Solutions in State-Specific CASSCF Theory

State-specific electronic structure theory provides a route toward balanced excited-state wave functions by exploiting higher-energy stationary points of the electronic energy. Multiconfigurational wave function approximations can describe both closed- and open-shell excited states and avoid the issues associated with state-averaged approaches. We investigate the existence of higher-energy solutions in complete active space self-consistent field (CASSCF) theory and characterize their topological properties. We demonstrate that state-specific approximations can provide accurate higher-energy excited states in H₂ (6-31G) with more compact active spaces than would be required in a state-averaged formalism. We then elucidate the unphysical stationary points, demonstrating that they arise from redundant orbitals when the active space is too large or symmetry breaking when the active space is too small. Furthermore, we investigate the singlet-triplet crossing in CH₂ (6-31G) and the avoided crossing in LiF (6-31G), revealing the severity of root flipping and demonstrating that state-specific solutions can behave quasi-diabatically or adiabatically. These results elucidate the complexity of the CASSCF energy landscape, highlighting the advantages and challenges of practical state-specific calculations.

State-Specific Configuration Interaction for Excited States

We introduce and benchmark a systematically improvable route for excited-state calculations, labeled state-specific configuration interaction (ΔCI), which is a particular realization of multiconfigurational self-consistent field and multireference configuration interaction. Starting with a reference built from optimized configuration state functions, separate CI calculations are performed for each targeted state (hence, state-specific orbitals and determinants). Accounting for single and double excitations produces the ΔCISD model, which can be improved with second-order Epstein-Nesbet perturbation theory (ΔCISD+EN2) or a posteriori Davidson corrections (ΔCISD+Q). These models were gauged against a vast and diverse set of 294 reference excitation energies. We have found that ΔCI is significantly more accurate than standard ground-state-based CI, whereas close performances were found between ΔCISD and EOM-CC2 and between ΔCISD+EN2 and EOM-CCSD. For larger systems, ΔCISD+Q delivers more accurate results than EOM-CC2 and EOM-CCSD. The ΔCI route can handle challenging multireference problems, singly and doubly excited states, from closed- and open-shell species, with overall comparable accuracy and thus represents a promising alternative to more established methodologies. In its current form, however, it is reliable only for relatively low-lying excited states.

Target State Optimized Density Functional Theory for Electronic Excited and Diabatic States

A flexible self-consistent field method, called target state optimization (TSO), is presented for exploring electronic excited configurations and localized diabatic states. The key idea is to partition molecular orbitals into different subspaces according to the excitation or localization pattern for a target state. Because of the orbital-subspace constraint, orbitals belonging to different subspaces do not mix. Furthermore, the determinant wave function for such excited or diabatic configurations can be variationally optimized as a ground state procedure, unlike conventional ΔSCF methods, without the possibility of collapsing back to the ground state or other lower-energy configurations. The TSO method can be applied both in Hartree-Fock theory and in Kohn-Sham density functional theory (DFT). The density projection procedure and the working equations for implementing the TSO method are described along with several illustrative applications. For valence excited states of organic compounds, it was found that the computed excitation energies from TSO-DFT and time-dependent density functional theory (TD-DFT) are of similar quality with average errors of 0.5 and 0.4 eV, respectively. For core excitation, doubly excited states and charge-transfer states, the performance of TSO-DFT is clearly superior to that from conventional TD-DFT calculations. It is shown that variationally optimized charge-localized diabatic states can be defined using TSO-DFT in energy decomposition analysis to gain both qualitative and quantitative insights on intermolecular interactions. Alternatively, the variational diabatic states may be used in molecular dynamics simulation of charge transfer processes. The TSO method can also be used to define basis states in multistate density functional theory for excited states through nonorthogonal state interaction calculations. The software implementing TSO-DFT can be accessed from the authors.

Applying Generalized Variational Principles to Excited-State-Specific Complete Active Space Self-consistent Field Theory

We employ a generalized variational principle to improve the stability, reliability, and precision of fully excited-state-specific complete active space self-consistent field theory. Compared to previous approaches that similarly seek to tailor this ansatz's orbitals and configuration interaction expansion for an individual excited state, we find the present approach to be more resistant to root flipping and better at achieving tight convergence to an energy stationary point. Unlike state-averaging, this approach allows orbital shapes to be optimal for individual excited states, which is especially important for charge-transfer states and some doubly excited states. We demonstrate the convergence and state-targeting abilities of this method in LiH, ozone, and MgO, showing in the latter that it is capable of finding three excited-state energy stationary points that no previous method has been able to locate.

Self-Adaptive Real-Time Time-Dependent Density Functional Theory for X-ray Absorptions

Real-time time-dependent density functional theory (RT-TDDFT) can in principle access the whole absorption spectrum of a many-electron system exposed to a narrow pulse. However, this requires an accurate and efficient propagator for the numerical integration of the time-dependent Kohn-Sham equation. While a low-order time propagator is already sufficient for the low-lying valence absorption spectra, it is no longer the case for the X-ray absorption spectra (XAS) of systems composed even only of light elements, for which the use of a high-order propagator is indispensable. It is then crucial to choose a largest possible time step and a shortest possible simulation time, so as to minimize the computational cost. To this end, we propose here a robust AutoPST approach to determine automatically (Auto) the propagator (P), step (S), and time (T) for relativistic RT-TDDFT simulations of XAS.

Computing x-ray absorption spectra from linear-response particles atop optimized holes

State specific orbital optimized density functional theory (OO-DFT) methods, such as restricted open-shell Kohn-Sham (ROKS), can attain semiquantitative accuracy for predicting x-ray absorption spectra of closed-shell molecules. OO-DFT methods, however, require that each state be individually optimized. In this Communication, we present an approach to generate an approximate core-excited state density for use with the ROKS energy ansatz, which is capable of giving reasonable accuracy without requiring state-specific optimization. This is achieved by fully optimizing the core-hole through the core-ionized state, followed by the use of electron-addition configuration interaction singles to obtain the particle level. This hybrid approach can be viewed as a DFT generalization of the static-exchange (STEX) method and can attain ∼0.6 eV rms error for the K-edges of C-F through the use of local functionals, such as PBE and OLYP. This ROKS(STEX) approach can also be used to identify important transitions for full OO ROKS treatment and can thus help reduce the computational cost of obtaining OO-DFT quality spectra. ROKS(STEX), therefore, appears to be a useful technique for the efficient prediction of x-ray absorption spectra.

Relativistic Orbital-Optimized Density Functional Theory for Accurate Core-Level Spectroscopy

Core-level spectra of 1s electrons of elements heavier than Ne show significant relativistic effects. We combine advances in orbital-optimized density functional theory (OO-DFT) with the spin-free exact two-component (X2C) model for scalar relativistic effects to study K-edge spectra of third period elements. OO-DFT/X2C is found to be quite accurate at predicting energies, yielding a ∼0.5 eV root-mean-square error versus experiment with the modern SCAN (and related) functionals. This marks a significant improvement over the >50 eV deviations that are typical for the popular time-dependent DFT (TDDFT) approach. Consequently, experimental spectra are quite well reproduced by OO-DFT/X2C, sans empirical shifts for alignment. OO-DFT/X2C combines high accuracy with ground state DFT cost and is thus a promising route for computing core-level spectra of third period elements. We also explored K and L edges of 3d transition metals to identify limitations of the OO-DFT/X2C approach in modeling the spectra of heavier atoms.

Energy Landscape of State-Specific Electronic Structure Theory

State-specific approximations can provide a more accurate representation of challenging electronic excitations by enabling relaxation of the electron density. While state-specific wave functions are known to be local minima or saddle points of the approximate energy, the global structure of the exact electronic energy remains largely unexplored. In this contribution, a geometric perspective on the exact electronic energy landscape is introduced. On the exact energy landscape, ground and excited states form stationary points constrained to the surface of a hypersphere, and the corresponding Hessian index increases at each excitation level. The connectivity between exact stationary points is investigated, and the square-magnitude of the exact energy gradient is shown to be directly proportional to the Hamiltonian variance. The minimal basis Hartree-Fock and excited-state mean-field representations of singlet H₂ (STO-3G) are then used to explore how the exact energy landscape controls the existence and properties of state-specific approximations. In particular, approximate excited states correspond to constrained stationary points on the exact energy landscape, and their Hessian index also increases for higher energies. Finally, the properties of the exact energy are used to derive the structure of the variance optimization landscape and elucidate the challenges faced by variance optimization algorithms, including the presence of unphysical saddle points or maxima of the variance.

Self-Consistent Field Methods for Excited States in Strong Magnetic Fields: a Comparison between Energy- and Variance-Based Approaches

Self-consistent field methods for excited states offer an attractive low-cost route to study not only excitation energies but also properties of excited states. Here, we present the generalization of two self-consistent field methods, the maximum overlap method (MOM) and the σ-SCF method, to calculate excited states in strong magnetic fields and investigate their stability and accuracy in this context. These methods use different strategies to overcome the well-known variational collapse of energy-based optimizations to the lowest solution of a given symmetry. The MOM tackles this problem in the definition of the orbital occupations to constrain the self-consistent field procedure to converge on excited states, while the σ-SCF method is based on the minimization of the variance instead of the energy. To overcome the high computational cost of the variance minimization, we present a new implementation of the σ-SCF method with the resolution of identity approximation, allowing the use of large basis sets, which is an important requirement for calculations in strong magnetic fields. The accuracy of these methods is assessed by comparison with the benchmark literature data for He, H₂, and CH⁺. The results reveal severe limitations of the variance-based scheme, which become more acute in large basis sets. In particular, many states are not accessible using variance optimization. Detailed analysis shows that this is a general feature of variance optimization approaches due to the masking of local minima in the optimization. In contrast, the MOM shows promising performance for computing excited states under these conditions, yielding results consistent with available benchmark data for a diverse range of electronic states.

Accuracy of the distinguishable cluster approximation for triple excitations for open-shell molecules and excited states

The distinguishable cluster approximation for triple excitations has been applied to calculate thermochemical properties and excited states involving closed-shell and open-shell species, such as small molecules, 3d transition metal atoms, ozone, and an iron-porphyrin model. Excitation energies have been computed using the ΔCC approach by directly optimizing the excited states. A fixed-reference technique has been introduced to target selected spin-states for open-shell molecular systems. The distinguishable cluster approximation consistently improves coupled cluster with singles doubles and triples results for absolute and relative energies. For excited states dominated by a single configuration state function, the fixed-reference approach combined with high-level coupled-cluster methods has a comparable accuracy to the corresponding equation-of-motion coupled-cluster methods with a negligible amount of spin contamination.

Dynamic-then-Static Approach for Core Excitations of Open-Shell Molecules

Delta self-consistent-field methods are widely used in studies of electronically excited states. However, the nonaufbau determinants are generally spin-contaminated. Here, we describe a general approach for spin-coupling interactions of open-shell molecules, making use of multistate density functional theory (MSDFT). In particular, the effective exchange integrals that determine spin coupling are obtained by enforcing the multiplet degeneracy of the S+1 state in the M_S = S manifold. Consequently, they are consistent with the energy of the high-spin state that is adequately treated by Kohn-Sham density functional theory (DFT) and, thereby, free of double counting of correlation. The method was applied to core excitations of open-shell molecules and compared with those by spin-adapted time-dependent DFT. An excellent agreement with experiment was found employing the BLYP functional and aug-cc-pCVQZ basis set. Overall, MSDFT provides an effective combination of the strengths of DFT and wave function theory to achieve efficiency and accuracy.

Orbital Optimized Density Functional Theory for Electronic Excited States

Density functional theory (DFT) based modeling of electronic excited states is of importance for investigation of the photophysical/photochemical properties and spectroscopic characterization of large systems. The widely used linear response time-dependent DFT (TDDFT) approach is, however, not effective at modeling many types of excited states, including (but not limited to) charge-transfer states, doubly excited states, and core-level excitations. In this perspective, we discuss state-specific orbital optimized (OO) DFT approaches as an alterative to TDDFT for electronic excited states. We motivate the use of OO-DFT methods and discuss reasons behind their relatively restricted historical usage (vs TDDFT). We subsequently highlight modern developments that address these factors and allow efficient and reliable OO-DFT computations. Several successful applications of OO-DFT for challenging electronic excitations are also presented, indicating their practical efficacy. OO-DFT approaches are thus increasingly becoming a useful route for computing excited states of large chemical systems. We conclude by discussing the limitations and challenges still facing OO-DFT methods, as well as some potential avenues for addressing them.

Accurate Electronic Excitation Energies in Full-Valence Active Space via Bootstrap Embedding

Fragment embedding has been widely used to circumvent the high computational scaling of using accurate electron correlation methods to describe the electronic ground states of molecules and materials. However, similar applications that utilize fragment embedding to treat electronic excited states are comparably less reported in the literature. The challenge here is twofold. First, most fragment embedding methods are most effective when the property of interest is local, but the change of the wave function upon excitation is nonlocal in general. Second, even for local excitations, an accurate estimate of, for example, the excitation energy can still be challenging owing to the need for a balanced treatment of both the ground and the excited states. In this work, we show that bootstrap embedding (BE), a fragment embedding method developed recently by our group, is promising toward describing general electronic excitations. Numerical simulations show that the excitation energies in full-valence active space (FVAS) can be well-estimated by BE to an error of ∼0.05 eV using relatively small fragments, for both local excitations and the excitations of some large dye molecules that exhibit strong charge-transfer characters. We hence anticipate BE to be a promising solution to accurately describing the excited states of large chemical systems.

Limitations of Hartree-Fock with quantum resources

The Hartree-Fock problem provides the conceptual and mathematical underpinning of a large portion of quantum chemistry. As efforts in quantum technology aim to enhance computational chemistry algorithms, the Hartree-Fock method, central to many other numerical approaches, is a natural target for quantum enhanced algorithms. While quantum computers and quantum simulation offer many prospects for the future of modern chemistry, the non-deterministic polynomial-complete Hartree-Fock problem is not a likely candidate. We highlight this fact from a number of perspectives including computational complexity, practical examples, and the full characterization of energy landscapes for simple systems.

A feasible approach for automatically differentiable unitary coupled-cluster on quantum computers

We develop computationally affordable and encoding independent gradient evaluation procedures for unitary coupled-cluster type operators, applicable on quantum computers. We show that, within our framework, the gradient of an expectation value with respect to a parameterized n-fold fermionic excitation can be evaluated by four expectation values of similar form and size, whereas most standard approaches, based on the direct application of the parameter-shift-rule, come with an associated cost of expectation values. For real wavefunctions, this cost can be further reduced to two expectation values. Our strategies are implemented within the open-source package Tequila and allow blackboard style construction of differentiable objective functions. We illustrate initial applications through extended adaptive approaches for electronic ground and excited states.

Generalization of Block-Localized Wave Function for Constrained Optimization of Excited Determinants

The block-localized wave function method is useful to provide insights on chemical bonding and intermolecular interactions through energy decomposition analysis. The method relies on block localization of molecular orbitals (MOs) by constraining the orbitals to basis functions within given blocks. Here, a generalized block-localized orbital (GBLO) method is described to allow both physically localized and delocalized MOs to be constrained in orbital-block definitions. Consequently, GBLO optimization can be conveniently tailored by imposing specific constraints. The GBLO method is illustrated by three examples: (1) constrained polarization response orbitals through dipole and quadrupole perturbation in a water dimer complex, (2) the ground and first excited-state potential energy curves of ethene about its C-C bond rotation, and (3) excitation energies of double electron excited states. Multistate density functional theory is used to determine the energies of the adiabatic ground and excited states using a minimal active space (MAS) comprising specifically charge-constrained and excited determinant configurations that are variationally optimized by the GBLO method. We find that the GBLO expansion that includes delocalized MOs in configurational blocks significantly reduces computational errors in comparison with physical block localization, and the computed ground- and excited-state energies are in good accordance with experiments and results obtained from multireference configuration interaction calculations.

A hybrid approach to excited-state-specific variational Monte Carlo and doubly excited states

We extend our hybrid linear-method/accelerated-descent variational Monte Carlo optimization approach to excited states and investigate its efficacy in double excitations. In addition to showing a superior statistical efficiency when compared to the linear method, our tests on small molecules show good energetic agreement with benchmark methods. We also demonstrate the ability to treat double excitations in systems that are too large for a full treatment by using selected configuration interaction methods via an application to 4-aminobenzonitrile. Finally, we investigate the stability of state-specific variance optimization against collapse to other states' variance minima and find that symmetry, Ansatz quality, and sample size all have roles to play in achieving stability.

Core excitations with excited state mean field and perturbation theory

We test the efficacy of excited state mean field theory and its excited-state-specific perturbation theory on the prediction of K-edge positions and x-ray peak separations. We find that the mean field theory is surprisingly accurate, even though it contains no accounting of differential electron correlation effects. In the perturbation theory, we test multiple core-valence separation schemes and find that, with the mean field theory already so accurate, electron-counting biases in one popular separation scheme become a dominant error when predicting K-edges. Happily, these appear to be relatively easy to correct for, leading to a perturbation theory for K-edge positions that is lower scaling and more accurate than coupled cluster theory and competitive in accuracy with recent high-accuracy results from restricted open-shell Kohn-Sham theory. For peak separations, our preliminary data show excited state mean field theory to be exceptionally accurate, but more extensive testing will be needed to see how it and its perturbation theory compare to coupled cluster peak separations more broadly.

Excited State Orbital Optimization via Minimizing the Square of the Gradient: General Approach and Application to Singly and Doubly Excited States via Density Functional Theory

We present a general approach to converge excited state solutions to any quantum chemistry orbital optimization process, without the risk of variational collapse. The resulting square gradient minimization (SGM) approach only requires analytic energy/Lagrangian orbital gradients and merely costs 3 times as much as ground state orbital optimization (per iteration), when implemented via a finite difference approach. SGM is applied to both single determinant ΔSCF and spin-purified restricted open-shell Kohn-Sham (ROKS) approaches to study the accuracy of orbital optimized DFT excited states. It is found that SGM can converge challenging states where the maximum overlap method (MOM) or analogues either collapse to the ground state or fail to converge. We also report that ΔSCF/ROKS predict highly accurate excitation energies for doubly excited states (which are inaccessible via TDDFT). Singly excited states obtained via ROKS are also found to be quite accurate, especially for Rydberg states that frustrate (semi)local TDDFT. Our results suggest that orbital optimized excited state DFT methods can be used to push past the limitations of TDDFT to doubly excited, charge-transfer, or Rydberg states, making them a useful tool for the practical quantum chemist's toolbox for studying excited states in large systems.

Variational calculations of excited states via direct optimization of the orbitals in DFT

A direct optimization method for obtaining excited electronic states using density functionals is presented.