Hartree and Exchange in Ensemble Density Functional Theory: Avoiding the Nonuniqueness Disaster

Extended N-centered ensemble density functional theory of double electronic excitations

A recent work (arXiv:2401.04685) has merged N-centered ensembles of neutral and charged electronic ground states with ensembles of neutral ground and excited states, thus providing a general and in-principle exact (so-called extended N-centered) ensemble density functional theory of neutral and charged electronic excitations. This formalism made it possible to revisit the concept of density-functional derivative discontinuity, in the particular case of single excitations from the highest occupied Kohn-Sham (KS) molecular orbital, without invoking the usual "asymptotic behavior of the density" argument. In this work, we address a broader class of excitations, with a particular focus on double excitations. An exact implementation of the theory is presented for the two-electron Hubbard dimer model. A thorough comparison of the true physical ground- and excited-state electronic structures with that of the fictitious ensemble density-functional KS system is also presented. Depending on the choice of the density-functional ensemble as well as the asymmetry of the dimer and the correlation strength, an inversion of states can be observed. In some other cases, the strong mixture of KS states within the true physical system makes the assignment "single excitation" or "double excitation" irrelevant.

Electronic Vector Potential from the Exact Factorization of a Complex Wavefunction

We generalize the definitions of local scalar potentials namedυ kin ${\upsilon _{{\rm{kin}}} }$ andυ N - 1 ${\upsilon _{N - 1} }$ , which are relevant to properly describe phenomena such as molecular dissociation with density-functional theory, to the case in which the electronic wavefunction corresponds to a complex current-carrying state. In such a case, an extra term in the form of a vector potential appears which cannot be gauged away. Both scalar and vector potentials are introduced via the exact factorization formalism which allows us to express the given Schrödinger equation as two coupled equations, one for the marginal and one for the conditional amplitude. The electronic vector potential is directly related to the paramagnetic current density carried by the total wavefunction and to the diamagnetic current density in the equation for the marginal amplitude. An explicit example of this vector potential in a triplet state of two non-interacting electrons is showcased together with its associated circulation, giving rise to a non-vanishing geometric phase. Some connections with the exact factorization for the full molecular wavefunction beyond the Born-Oppenheimer approximation are also discussed.

What can lattice DFT teach us about real-space DFT?

In this paper we establish a connection between density functional theory (DFT) for lattice models and common real-space DFT. We consider the lattice DFT description of a two-level model subject to generic interactions in Mermin's DFT formulation in the grand canonical ensemble at finite temperature. The case of only density-density and Hund's rule interaction studied in earlier work is shown to be equivalent to an exact-exchange description of DFT in the real-space picture. In addition, we also include the so-called pair-hopping interaction which can be treated analytically and, crucially, leads to non-integer occupations of the Kohn-Sham (KS) levels even in the limit of zero temperature. Treating the hydrogen molecule in a minimal basis is shown to be equivalent to our two-level lattice DFT model. By means of the fractional occupations of the KS orbitals (which, in this case, are identical to the many-body ones) we reproduce the results of full configuration interaction, even in the dissociation limit and without breaking the spin symmetry. Beyond the minimal basis, we embed our HOMO-LUMO model into a standard DFT calculation and, again, obtain results in overall good agreement with exact ones without the need of breaking the spin symmetry.

Exact Excited-State Functionals of the Asymmetric Hubbard Dimer

The exact functionals associated with the (singlet) ground state and the two singlet excited states of the asymmetric Hubbard dimer at half-filling are calculated using both Levy's constrained search and Lieb's convex formulation. While the ground-state functional is, as is commonly known, a convex function with respect to the density, the functional associated with the doubly excited state is found to be concave. Also, because the density-potential mapping associated with the first excited state is noninvertible, its "functional" is a partial, multivalued function composed of one concave and one convex branch that correspond to two separate domains of the external potential. Remarkably, it is found that, although the one-to-one mapping between density and external potential may not apply (as in the case of the first excited state), each state-specific energy and corresponding universal functional are "functions" whose derivatives are each other's inverse, just as in the ground state formalism.

Better performance of Hartree-Fock over DFT: a quantum mechanical investigation on pyridinium benzimidazolate types of zwitterions in the light of localization/delocalization issues

CONTEXT

With the advent of fast computing facilities, combined with rapid emerges of many new and intricate quantum mechanical functionals, computations with pure Hartree-Fock (HF) theory are now-a-days regarded as trivial or obsolete, or even considered as not reliable by many researchers. Consequently, current trends in computational chemistry show extensive use of post-HF theories for smaller molecular systems and various DFT methods for organic and inorganic chemistry related problems (larger molecules/systems). In this contribution, I have tried to show that sometimes, HF might be more suitable over DFT methodologies in addressing structure-property correlations. Molecules studied here were previously synthesized by Boyd in 1966 and important experimental data were produced by Alcalde and co-workers in 1987. Comparison of computed and experimental results clearly shows that HF method was more effective in reproducing the experimental data compared to especially the DFT methodologies. Reliability of HF method was further assured from the very similar results shown by the CCSD, CASSCF, CISD and QCISD methods. Current study also indicates that the localization issue associated with HF proved to be advantageous over delocalization issue of DFT based methodologies, in correctly describing the structure-property correlation for zwitterion systems.

METHODS

All computations were performed with Gaussian 09. A wide-range of quantum mechanical methodologies, HF, B3LYP, CAM-B3LYP, BMK, B3PW91, TPSSh, LC-ωPBE, M06-2X, M06-HF, ωB97xD, MP2, CASSCF, CCSD, QCISD, CISD and semi-empirical methods like, Huckel, CNDO, AM1, PM3MM and PM6, were used for investigations.

Electronic Excited States in Extreme Limits via Ensemble Density Functionals

Density functional theory (DFT) has greatly expanded our ability to affordably compute and understand electronic ground states, by replacing intractable ab initio calculations by models based on paradigmatic physics from high- and low-density limits. But, a comparable treatment of excited states lags behind. Here, we solve this outstanding problem by employing a generalization of density functional theory to ensemble states (EDFT). We thus address important paradigmatic cases of all electronic systems in strongly (low-density) and weakly (high-density) correlated regimes. We show that the high-density limit connects to recent, exactly solvable EDFT results. The low-density limit reveals an unnoticed and most unexpected result-density functionals for strictly correlated ground states can be reused directly for excited states. Nontrivial dependence on excitation structure only shows up at third leading order. Overall, our results provide foundations for effective models of excited states that interpolate between exact low- and high-density limits, which we illustrate on the cases of singlet-singlet excitations in H_{2} and a ring of quantum wells.

DFT exchange: sharing perspectives on the workhorse of quantum chemistry and materials science

In this paper, the history, present status, and future of density-functional theory (DFT) is informally reviewed and discussed by 70 workers in the field, including molecular scientists, materials scientists, method developers and practitioners. The format of the paper is that of a roundtable discussion, in which the participants express and exchange views on DFT in the form of 302 individual contributions, formulated as responses to a preset list of 26 questions. Supported by a bibliography of 777 entries, the paper represents a broad snapshot of DFT, anno 2022.

Multistate Density Functional Theory of Excited States

We report a rigorous formulation of density functional theory for excited states, providing a theoretical foundation for a multistate density functional theory. We prove the existence of a Hamiltonian matrix functional H[D] of the multistate matrix density D(r) in the subspace spanned by the lowest N eigenstates. Here, D(r) is an N-dimensional matrix of state densities and transition densities. Then, a variational principle of the multistate subspace energy is established, whose minimization yields both the energies and densities of the individual N eigenstates. Furthermore, we prove that the N-dimensional matrix density D(r) can be sufficiently represented by N² nonorthogonal Slater determinants, based on which an interacting active space is introduced for practical calculations. This work establishes that the ground and excited states can be treated on an equal footing in density functional theory.

Single Excitation Energies Obtained from the Ensemble "HOMO-LUMO Gap": Exact Results and Approximations

In calculations based on density functional theory, the "HOMO-LUMO gap" (difference between the highest occupied and lowest unoccupied molecular orbital energies) is often used as a low-cost, ad hoc approximation for the lowest excitation energy. Here we show that a simple correction based on rigorous ensemble density functional theory makes the HOMO-LUMO gap exact in principle and significantly more accurate in practice. The introduced perturbative ensemble density functional theory approach predicts different and useful values for singlet-singlet and singlet-triplet excitations, using semilocal and hybrid approximations. Excitation energies are similar in quality to time-dependent density functional theory, especially at high fractions of exact exchange. The approach therefore offers an easy-to-implement and low-cost route to robust prediction of molecular excitation energies.

Density functionals with spin-density accuracy for open shells

Electrons in zero external magnetic field can be studied with density functional theory (DFT) or with spin-DFT (SDFT). The latter is normally used for open shell systems because its approximations appear to model better the exchange and correlation (xc) functional, but also because so far the application of DFT implied a closed-shell-like approximation. Correcting this error for open shells allows the approximate DFT xc functionals to become as accurate as those in SDFT. In the limit of zero magnetic field, the Kohn-Sham equations of SDFT emerge as the generalised KS equations of DFT.

Foundation of One-Particle Reduced Density Matrix Functional Theory for Excited States

In Phys. Rev. Lett. 2021, 127, 023001 a reduced density matrix functional theory (RDMFT) was proposed for calculating energies of selected eigenstates of interacting many-Fermion systems. Here, we develop a solid foundation for this so-called w-RDMFT and present the details of various derivations. First, we explain how a generalization of the Ritz variational principle to ensemble states with fixed weights w in combination with the constrained search would lead to a universal functional of the one-particle reduced density matrix. To turn this into a viable functional theory, however, we also need to implement an exact convex relaxation. This general procedure includes Valone's pioneering work on ground state RDMFT as the special case w = (1,0, ···). Then, we work out in a comprehensive manner a methodology for deriving a compact description of the functional's domain. This leads to a hierarchy of generalized exclusion principle constraints which we illustrate in great detail. By anticipating their future pivotal role in functional theories and to keep our work self-contained, several required concepts from convex analysis are introduced and discussed.

Ensemble Density Functional Theory of Neutral and Charged Excitations : Exact Formulations, Standard Approximations, and Open Questions

Recent progress in the field of (time-independent) ensemble density-functional theory (DFT) for excited states are reviewed. Both Gross-Oliveira-Kohn (GOK) and N-centered ensemble formalisms, which are mathematically very similar and allow for an in-principle-exact description of neutral and charged electronic excitations, respectively, are discussed. Key exact results, for example, the equivalence between the infamous derivative discontinuity problem and the description of weight dependencies in the ensemble exchange-correlation density functional, are highlighted. The variational evaluation of orbital-dependent ensemble Hartree-exchange (Hx) energies is discussed in detail. We show in passing that state-averaging individual exact Hx energies can lead to severe (although solvable) v-representability issues. Finally, we explore the possibility of using the concept of density-driven correlation, which has been introduced recently and does not exist in regular ground-state DFT, for improving state-of-the-art correlation density-functional approximations for ensembles. The present review reflects the efforts of a growing community to turn ensemble DFT into a rigorous and reliable low-cost computational method for excited states. We hope that, in the near future, this contribution will stimulate new formal and practical developments in the field.

Minimal-active-space multistate density functional theory for excitation energy involving local and charge transfer states

Multistate density functional theory (MSDFT) employing a minimum active space (MAS) is presented to determine charge transfer (CT) and local excited states of bimolecular complexes. MSDFT is a hybrid wave function theory (WFT) and density functional theory, in which dynamic correlation is first incorporated in individual determinant configurations using a Kohn-Sham exchange-correlation functional. Then, nonorthogonal configuration-state interaction is performed to treat static correlation. Because molecular orbitals are optimized separately for each determinant by including Kohn-Sham dynamic correlation, a minimal number of configurations in the active space, essential to representing low-lying excited and CT states of interest, is sufficient to yield the adiabatic states. We found that the present MAS-MSDFT method provides a good description of covalent and CT excited states in comparison with experiments and high-level computational results. Because of the simplicity and interpretive capability through diabatic configuration weights, the method may be useful in dynamic simulations of CT and nonadiabatic processes.

Variations of the Hartree-Fock fractional-spin error for one electron

Fractional-spin errors are inherent in all current approximate density functionals, including Hartree-Fock theory, and their origin has been related to strong static correlation effects. The conventional way to encode fractional-spin calculations is to construct an ensemble density that scales between the high-spin and low-spin densities. In this article, we explore the variation of the Hartree-Fock fractional-spin (or ghost-interaction) error in one-electron systems using restricted and unrestricted ensemble densities and the exact generalized Hartree-Fock representation. By considering the hydrogen atom and H⁺ ₂ cation, we analyze how the unrestricted and generalized Hartree-Fock schemes minimize this error by localizing the electrons or rotating the spin coordinates. We also reveal a clear similarity between the Coulomb hole of He-like ions and the density depletion near the nucleus induced by the fractional-spin error in the unpolarized hydrogen atom. Finally, we analyze the effect of the fractional-spin error on the Møller-Plesset adiabatic connection, excited states, and functional- and density-driven errors.

Description of Sudden Polarization in the Excited Electronic States with an Ensemble Density Functional Theory Method

Sudden polarization (SP) is one of the manifestations of electron transfer in the electronically excited states of molecules. Proposed initially to explain the unusual reactivity of photoexcited olefins, SP often occurs in the excited states of molecules possessing strongly correlated diradical ground state. Theoretical description of SP involves mixing between the singly excited and the doubly excited zwitterionic states, which makes it inaccessible with the use of the popular linear-response time-dependent density functional theory methods. In this work, an extended variant of the state-interaction state-averaged spin-restricted ensemble-referenced Kohn-Sham (SI-SA-REKS, or SSR) method is applied to study SP in a number of organic diradical systems. To this end, the analytical derivative formalism is derived and implemented for the SSR(3,2) method (see the main text for explanation of the acronym), which enables the automatic geometry optimization and obtains the relaxed density matrices as well as the electron binding energies and respective Dyson's orbitals. Application of the new method to SP in the lowest singlet excited state of ethylene agrees with the results obtained previously with the use of multireference methods of wavefunction theory. A number of interesting manifestations of SP are observed, such as the charge transfer in photoexcited tetramethyleneethene (TME) diradical mediated by the vibrational motion and conductivity switching in the excited state of a donor-acceptor dyad placed in an external electric field.

Ensemble Reduced Density Matrix Functional Theory for Excited States and Hierarchical Generalization of Pauli's Exclusion Principle

We propose and work out a reduced density matrix functional theory (RDMFT) for calculating energies of eigenstates of interacting many-electron systems beyond the ground state. Various obstacles which historically have doomed such an approach to be unfeasible are overcome. First, we resort to a generalization of the Ritz variational principle to ensemble states with fixed weights. This in combination with the constrained search formalism allows us to establish a universal functional of the one-particle reduced density matrix. Second, we employ tools from convex analysis to circumvent the too involved N-representability constraints. Remarkably, this identifies Valone's pioneering work on RDMFT as a special case of convex relaxation and reveals that crucial information about the excitation structure is contained in the functional's domain. Third, to determine the crucial latter object, a methodology is developed which eventually leads to a generalized exclusion principle. The corresponding linear constraints are calculated for systems of arbitrary size.

Ensemble generalized Kohn-Sham theory: The good, the bad, and the ugly

Two important extensions of Kohn-Sham (KS) theory are generalized: KS theory and ensemble KS theory. The former allows for non-multiplicative potential operators and greatly facilitates practical calculations with advanced, orbital-dependent functionals. The latter allows for quantum ensembles and enables the treatment of open systems and excited states. Here, we combine the two extensions, both formally and practically, first via an exact yet complicated formalism and then via a computationally tractable variant that involves a controlled approximation of ensemble "ghost interactions" by means of an iterative algorithm. The resulting formalism is illustrated using selected examples. This opens the door to the application of generalized KS theory in more challenging quantum scenarios and to the improvement of ensemble theories for the purpose of practical and accurate calculations.

Ensemble Density Functional Theory: Insight from the Fluctuation-Dissipation Theorem

Density functional theory can be generalized to mixtures of ground and excited states, for the purpose of determining energies of excitations using low-cost density functional approximations. Adapting approximations originally developed for ground states to work in the new setting would fast-forward progress enormously. But, previous attempts have stumbled on daunting fundamental issues. Here we show that these issues can be prevented from the outset, by using a fluctuation dissipation theorem (FDT) to dictate key functionals. We thereby show that existing exchange energy approximations are readily adapted to excited states, when combined with a rigorous exact Hartree term that is different in form from its ground state counterpart, and counterparts based on ensemble Ansatzë. Applying the FDT to correlation energies also provides insights into ground statelike and ensemble-only correlations. We thus provide a comprehensive and versatile framework for ensemble density functional approximations.

Approximately Self-Consistent Ensemble Density Functional Theory: Toward Inclusion of All Correlations

Recent theory developments in ensemble density functional theory (EDFT) promise to bring decades of work for ground states to the practical resolution of excited states, provided newly discovered "density-driven correlations" can be dealt with and adequate effective potentials can be found. This Letter introduces simple theories for both and shows that EDFT using these theories outperforms ΔSCF DFT and time-dependent DFT for low-lying gaps in most of the small atoms and molecules tested, even when all use the same density functional approximations. It thus establishes EDFT as a promising tool for low-cost studies of low-lying excited states and provides a clear route to practical EDFT implementation of arbitrary functional approximations.

Individual Correlations in Ensemble Density Functional Theory: State- and Density-Driven Decompositions without Additional Kohn-Sham Systems

Gould and Pittalis [Phys. Rev. Lett. 123, 016401 (2019)PRLTAO0031-900710.1103/PhysRevLett.123.016401 recently revealed a density-driven (DD) correlation energy that is specific to many-electron ensembles and must be accounted for by approximations. We derive in this Letter a general and simpler expression in terms of the ensemble weights, the ensemble Kohn-Sham (KS) orbitals, and their linear response to variations in the ensemble weights. As no additional state-driven KS systems are needed, its evaluation is greatly simplified. We confirm the importance of DD effects and introduce a direct and promising route to approximations.

What do we learn from the classical turning surface of the Kohn-Sham potential as electron number is varied continuously?

The classical Kohn-Sham turning radius R_t of an atom can be defined as the radius where the Kohn-Sham potential is equal to the negative ionization potential of the atom, i.e., where v_s(R_t) = ϵ_h. It was recently shown [E. Ospadov et al., Proc. Natl. Acad. Sci. U. S. A. 115, E11578-E11585 (2018)] to yield chemically relevant bonding distances, in line with known empirical values. In this work, we show that extension of the concept to non-integer electron number yields additional information about atomic systems and can be used to detect the difficulty of adding or subtracting electrons. Notably, it reflects the ease of bonding in open p-shells and its greater difficulty in open s-shells. The latter manifests in significant discontinuities in the turning radius as the electron number changes the principal quantum number of the outermost electronic shell (e.g., going from Na to Na²⁺). We then show that a non-integer picture is required to correctly interpret bonding and dissociation in H₂ ⁺. Results are consistent when properties are calculated exactly or via an appropriate approximation. They can be interpreted in the context of conceptual density functional theory.

The one-electron self-interaction error in 74 density functional approximations: a case study on hydrogenic mono- and dinuclear systems

The self-interaction error (SIE), i.e. unphysical interactions of electrons with themselves, has plagued developers and users of Density Functional Approximations (DFAs) since the inception of Density Functional Theory (DFT). Formally, it can be separated into the one-electron and many-electron SIE; herein we present one of the most comprehensive studies of the first. While we focus mostly on the total SIE, we also make use of two different decompositions. The first is a separation into functional and density-driven errors as championed by Sim, Burke and co-workers [J. Phys. Chem. Lett., 2018, 9, 6385-6392]; the second separates the error into exchange, correlation, and one-electron components, with the latter being a density error that has not been discussed in this form before. After investigating the familiar hydrogen atom and dihydrogen cation, we establish a relationship between the SIE and the nuclear charge with the help of a series of heavier hydrogenic analogues. For the mononuclear systems and the diatomics at the dissociation limit, this relationship is linear in nature with prominent exceptions, mostly belonging to the Minnesota and range-separated (double-)hybrid DFAs. For the first time, we also show how the magnitude of the SIE depends on the underlying atomic-orbital basis set and how DFAs that rely on a popular van-der-Waals DFT type London-dispersion term exhibit "self-dispersion". We find that range separation is not a panacea for solving the one-electron SIE. DFAs that have been developed to be one-electron SIE free for one system, such as the hydrogen atom, show larger errors for heavier hydrogenic systems. Often, one-electron SIE-free DFAs rely on fortuitous error cancellation between their exchange and correlation components. An analysis of the most robust methods for general applications to date reveals that they suffer moderately from the one-electron SIE, while DFAs that are nearly SIE-free do not perform well in applications. Implicit in the continued existence of the one-electron SIE is that well-performing DFAs continue to suffer insufficiencies at their fundamental levels that are being compensated for by the SIE. Our analysis includes more than 250 000 datapoints, resulting in multiple insights that may drive future developments of new DFAs or SIE corrections.

Weight dependence of local exchange–correlation functionals in ensemble density-functional theory: double excitations in two-electron systems

We discuss the construction of first-rung weight-dependent exchange–correlation density-functional approximations for He and H₂ specifically designed for the computation of double excitations within Gross–Oliveira–Kohn-DFT.

Density-Driven Correlations in Ensemble Density Functional Theory: Insights from Simple Excitations in Atoms

Ensemble density functional theory extends the usual Kohn-Sham machinery to quantum state ensembles involving ground- and excited states. Recent work by the authors [Phys. Rev. Lett. 119, 243001 (2017); 123, 016401 (2019)] has shown that both the Hartree-exchange and correlation energies can attain unusual features in ensembles. Density-driven (DD) correlations – which account for the fact that pure-state densities in Kohn-Sham ensembles do not necessarily reproduce those of interacting pure states – are one such feature. Here we study atoms (specifically S–P and S–S transitions) and show that the magnitude and behaviour of DD correlations can vary greatly with the variation of the orbital angular momentum of the involved states. Such estimations are obtained through an approximation for DD correlations built from relevant exact conditions, Kohn-Sham inversion, and plausible assumptions for weakly correlated systems.

Density-Driven Correlations in Many-Electron Ensembles: Theory and Application for Excited States

Density functional theory can be extended to excited states by means of a unified variational approach for passive state ensembles. This extension overcomes the restriction of the typical density functional approach to ground states, and offers useful formal and demonstrated practical benefits. The correlation energy functional in the generalized case acquires higher complexity than its ground state counterpart, however. Little is known about its internal structure nor how to effectively approximate it in general. Here we show that such a functional can be broken down into natural components, including what we call "state-" and "density-driven" correlations, with the former amenable to conventional approximations, and the latter being a unique feature of ensembles. Such a decomposition provides us with a pathway to general approximations that are able to routinely handle low-lying excited states. The importance of density-driven correlations is demonstrated, and an approximation for them is introduced and shown to be useful.

Asymptotic behavior of the Hartree-exchange and correlation potentials in ensemble density functional theory

We show that the Hartree-exchange and correlation potentials of ensemble systems can have unexpected features, including non-vanishing asymptotic constants and non-trivial screening of electrons. These features are demonstrated here on Li, C, and F.