Density-Driven Correlations in Many-Electron Ensembles: Theory and Application for Excited States

Extended N-centered ensemble density functional theory of double electronic excitations

A recent work (arXiv:2401.04685) has merged N-centered ensembles of neutral and charged electronic ground states with ensembles of neutral ground and excited states, thus providing a general and in-principle exact (so-called extended N-centered) ensemble density functional theory of neutral and charged electronic excitations. This formalism made it possible to revisit the concept of density-functional derivative discontinuity, in the particular case of single excitations from the highest occupied Kohn-Sham (KS) molecular orbital, without invoking the usual "asymptotic behavior of the density" argument. In this work, we address a broader class of excitations, with a particular focus on double excitations. An exact implementation of the theory is presented for the two-electron Hubbard dimer model. A thorough comparison of the true physical ground- and excited-state electronic structures with that of the fictitious ensemble density-functional KS system is also presented. Depending on the choice of the density-functional ensemble as well as the asymmetry of the dimer and the correlation strength, an inversion of states can be observed. In some other cases, the strong mixture of KS states within the true physical system makes the assignment "single excitation" or "double excitation" irrelevant.

Electronic Vector Potential from the Exact Factorization of a Complex Wavefunction

We generalize the definitions of local scalar potentials namedυ kin ${\upsilon _{{\rm{kin}}} }$ andυ N - 1 ${\upsilon _{N - 1} }$ , which are relevant to properly describe phenomena such as molecular dissociation with density-functional theory, to the case in which the electronic wavefunction corresponds to a complex current-carrying state. In such a case, an extra term in the form of a vector potential appears which cannot be gauged away. Both scalar and vector potentials are introduced via the exact factorization formalism which allows us to express the given Schrödinger equation as two coupled equations, one for the marginal and one for the conditional amplitude. The electronic vector potential is directly related to the paramagnetic current density carried by the total wavefunction and to the diamagnetic current density in the equation for the marginal amplitude. An explicit example of this vector potential in a triplet state of two non-interacting electrons is showcased together with its associated circulation, giving rise to a non-vanishing geometric phase. Some connections with the exact factorization for the full molecular wavefunction beyond the Born-Oppenheimer approximation are also discussed.

Excited states in warm and hot dense matter

Accurate modeling of warm and hot dense matter is challenging in part due to the multitude of excited states that must be considered. Here, we present a variational framework that models these excited states. In this framework an excited state is defined by a set of effective one-electron occupation factors, and the corresponding energy is defined by the effective one-body energy with an exchange and correlation term. The variational framework is applied to an atom-in-plasma model (a generalization of the so-called average atom model). Comparisons with a density functional theory based average atom model generally reveal good agreement in the calculated pressure, but our model also gives access to the excitation energies and charge state distributions.

What can lattice DFT teach us about real-space DFT?

In this paper we establish a connection between density functional theory (DFT) for lattice models and common real-space DFT. We consider the lattice DFT description of a two-level model subject to generic interactions in Mermin's DFT formulation in the grand canonical ensemble at finite temperature. The case of only density-density and Hund's rule interaction studied in earlier work is shown to be equivalent to an exact-exchange description of DFT in the real-space picture. In addition, we also include the so-called pair-hopping interaction which can be treated analytically and, crucially, leads to non-integer occupations of the Kohn-Sham (KS) levels even in the limit of zero temperature. Treating the hydrogen molecule in a minimal basis is shown to be equivalent to our two-level lattice DFT model. By means of the fractional occupations of the KS orbitals (which, in this case, are identical to the many-body ones) we reproduce the results of full configuration interaction, even in the dissociation limit and without breaking the spin symmetry. Beyond the minimal basis, we embed our HOMO-LUMO model into a standard DFT calculation and, again, obtain results in overall good agreement with exact ones without the need of breaking the spin symmetry.

Exact Excited-State Functionals of the Asymmetric Hubbard Dimer

The exact functionals associated with the (singlet) ground state and the two singlet excited states of the asymmetric Hubbard dimer at half-filling are calculated using both Levy's constrained search and Lieb's convex formulation. While the ground-state functional is, as is commonly known, a convex function with respect to the density, the functional associated with the doubly excited state is found to be concave. Also, because the density-potential mapping associated with the first excited state is noninvertible, its "functional" is a partial, multivalued function composed of one concave and one convex branch that correspond to two separate domains of the external potential. Remarkably, it is found that, although the one-to-one mapping between density and external potential may not apply (as in the case of the first excited state), each state-specific energy and corresponding universal functional are "functions" whose derivatives are each other's inverse, just as in the ground state formalism.

Electronic Excited States in Extreme Limits via Ensemble Density Functionals

Density functional theory (DFT) has greatly expanded our ability to affordably compute and understand electronic ground states, by replacing intractable ab initio calculations by models based on paradigmatic physics from high- and low-density limits. But, a comparable treatment of excited states lags behind. Here, we solve this outstanding problem by employing a generalization of density functional theory to ensemble states (EDFT). We thus address important paradigmatic cases of all electronic systems in strongly (low-density) and weakly (high-density) correlated regimes. We show that the high-density limit connects to recent, exactly solvable EDFT results. The low-density limit reveals an unnoticed and most unexpected result-density functionals for strictly correlated ground states can be reused directly for excited states. Nontrivial dependence on excitation structure only shows up at third leading order. Overall, our results provide foundations for effective models of excited states that interpolate between exact low- and high-density limits, which we illustrate on the cases of singlet-singlet excitations in H_{2} and a ring of quantum wells.

Toward routine Kohn-Sham inversion using the "Lieb-response" approach

Kohn-Sham (KS) inversion, in which the effective KS mean-field potential is found for a given density, provides insights into the nature of exact density functional theory (DFT) that can be exploited for the development of density functional approximations. Unfortunately, despite significant and sustained progress in both theory and software libraries, KS inversion remains rather difficult in practice, especially in finite basis sets. The present work presents a KS inversion method, dubbed the "Lieb-response" approach, that naturally works with existing Fock-matrix DFT infrastructure in finite basis sets, is numerically efficient, and directly provides meaningful matrix and energy quantities for pure-state and ensemble systems. Some additional work yields potential. It thus enables the routine inversion of even difficult KS systems, as illustrated in a variety of problems within this work, and provides outputs that can be used for embedding schemes or machine learning of density functional approximations. The effect of finite basis sets on KS inversion is also analyzed and investigated.

DFT exchange: sharing perspectives on the workhorse of quantum chemistry and materials science

In this paper, the history, present status, and future of density-functional theory (DFT) is informally reviewed and discussed by 70 workers in the field, including molecular scientists, materials scientists, method developers and practitioners. The format of the paper is that of a roundtable discussion, in which the participants express and exchange views on DFT in the form of 302 individual contributions, formulated as responses to a preset list of 26 questions. Supported by a bibliography of 777 entries, the paper represents a broad snapshot of DFT, anno 2022.

Multistate Density Functional Theory of Excited States

We report a rigorous formulation of density functional theory for excited states, providing a theoretical foundation for a multistate density functional theory. We prove the existence of a Hamiltonian matrix functional H[D] of the multistate matrix density D(r) in the subspace spanned by the lowest N eigenstates. Here, D(r) is an N-dimensional matrix of state densities and transition densities. Then, a variational principle of the multistate subspace energy is established, whose minimization yields both the energies and densities of the individual N eigenstates. Furthermore, we prove that the N-dimensional matrix density D(r) can be sufficiently represented by N² nonorthogonal Slater determinants, based on which an interacting active space is introduced for practical calculations. This work establishes that the ground and excited states can be treated on an equal footing in density functional theory.

Is the non-additive kinetic potential always equal to the difference of effective potentials from inverting the Kohn-Sham equation?

The relation used frequently in the literature {according to which the non-additive kinetic potential which is a functional depending on a pair of electron densities is equal (up to a constant) to} the difference of two potentials obtained from inverting two Kohn-Sham equations, is examined. <p>The relation is based on a silent assumption that the two densities can be obtained from two independent Kohn-Sham equations - i.e. are $v_s$-representable.</p> <p>It is shown that this assumption does not hold for pairs of densities: $\rho_{tot}$ - being the Kohn-Sham density in some system and $\rho_B$ obtained from such partitioning of $\rho_{tot}$ that the difference $\rho_{tot}-\rho_B$ vanish on a Lebesgue measurable volume element. The inversion procedure is still applicable for $\rho_{tot}-\rho_B$ but cannot be interpreted as the inversion of Kohn-Sham equation but rather inversion of Kohn-Sham equation with an additional constraint. The obtained effective potential comprises a "contaminant" that might even not be unique. It is shown, that the construction of the non-additive kinetic potential based on the examined relation is not applicable for such pairs.

Improving Results by Improving Densities: Density-Corrected Density Functional Theory

Density functional theory (DFT) calculations have become widespread in both chemistry and materials, because they usually provide useful accuracy at much lower computational cost than wavefunction-based methods. All practical DFT calculations require an approximation to the unknown exchange-correlation energy, which is then used self-consistently in the Kohn-Sham scheme to produce an approximate energy from an approximate density. Density-corrected DFT is simply the study of the relative contributions to the total energy error. In the vast majority of DFT calculations, the error due to the approximate density is negligible. But with certain classes of functionals applied to certain classes of problems, the density error is sufficiently large as to contribute to the energy noticeably, and its removal leads to much better results. These problems include reaction barriers, torsional barriers involving π-conjugation, halogen bonds, radicals and anions, most stretched bonds, etc. In all such cases, use of a more accurate density significantly improves performance, and often the simple expedient of using the Hartree-Fock density is enough. This Perspective explains what DC-DFT is, where it is likely to improve results, and how DC-DFT can produce more accurate functionals. We also outline challenges and prospects for the field.

Single Excitation Energies Obtained from the Ensemble "HOMO-LUMO Gap": Exact Results and Approximations

In calculations based on density functional theory, the "HOMO-LUMO gap" (difference between the highest occupied and lowest unoccupied molecular orbital energies) is often used as a low-cost, ad hoc approximation for the lowest excitation energy. Here we show that a simple correction based on rigorous ensemble density functional theory makes the HOMO-LUMO gap exact in principle and significantly more accurate in practice. The introduced perturbative ensemble density functional theory approach predicts different and useful values for singlet-singlet and singlet-triplet excitations, using semilocal and hybrid approximations. Excitation energies are similar in quality to time-dependent density functional theory, especially at high fractions of exact exchange. The approach therefore offers an easy-to-implement and low-cost route to robust prediction of molecular excitation energies.

Foundation of One-Particle Reduced Density Matrix Functional Theory for Excited States

In Phys. Rev. Lett. 2021, 127, 023001 a reduced density matrix functional theory (RDMFT) was proposed for calculating energies of selected eigenstates of interacting many-Fermion systems. Here, we develop a solid foundation for this so-called w-RDMFT and present the details of various derivations. First, we explain how a generalization of the Ritz variational principle to ensemble states with fixed weights w in combination with the constrained search would lead to a universal functional of the one-particle reduced density matrix. To turn this into a viable functional theory, however, we also need to implement an exact convex relaxation. This general procedure includes Valone's pioneering work on ground state RDMFT as the special case w = (1,0, ···). Then, we work out in a comprehensive manner a methodology for deriving a compact description of the functional's domain. This leads to a hierarchy of generalized exclusion principle constraints which we illustrate in great detail. By anticipating their future pivotal role in functional theories and to keep our work self-contained, several required concepts from convex analysis are introduced and discussed.

Ensemble Density Functional Theory of Neutral and Charged Excitations : Exact Formulations, Standard Approximations, and Open Questions

Recent progress in the field of (time-independent) ensemble density-functional theory (DFT) for excited states are reviewed. Both Gross-Oliveira-Kohn (GOK) and N-centered ensemble formalisms, which are mathematically very similar and allow for an in-principle-exact description of neutral and charged electronic excitations, respectively, are discussed. Key exact results, for example, the equivalence between the infamous derivative discontinuity problem and the description of weight dependencies in the ensemble exchange-correlation density functional, are highlighted. The variational evaluation of orbital-dependent ensemble Hartree-exchange (Hx) energies is discussed in detail. We show in passing that state-averaging individual exact Hx energies can lead to severe (although solvable) v-representability issues. Finally, we explore the possibility of using the concept of density-driven correlation, which has been introduced recently and does not exist in regular ground-state DFT, for improving state-of-the-art correlation density-functional approximations for ensembles. The present review reflects the efforts of a growing community to turn ensemble DFT into a rigorous and reliable low-cost computational method for excited states. We hope that, in the near future, this contribution will stimulate new formal and practical developments in the field.

Minimal-active-space multistate density functional theory for excitation energy involving local and charge transfer states

Multistate density functional theory (MSDFT) employing a minimum active space (MAS) is presented to determine charge transfer (CT) and local excited states of bimolecular complexes. MSDFT is a hybrid wave function theory (WFT) and density functional theory, in which dynamic correlation is first incorporated in individual determinant configurations using a Kohn-Sham exchange-correlation functional. Then, nonorthogonal configuration-state interaction is performed to treat static correlation. Because molecular orbitals are optimized separately for each determinant by including Kohn-Sham dynamic correlation, a minimal number of configurations in the active space, essential to representing low-lying excited and CT states of interest, is sufficient to yield the adiabatic states. We found that the present MAS-MSDFT method provides a good description of covalent and CT excited states in comparison with experiments and high-level computational results. Because of the simplicity and interpretive capability through diabatic configuration weights, the method may be useful in dynamic simulations of CT and nonadiabatic processes.

Ensemble Reduced Density Matrix Functional Theory for Excited States and Hierarchical Generalization of Pauli's Exclusion Principle

We propose and work out a reduced density matrix functional theory (RDMFT) for calculating energies of eigenstates of interacting many-electron systems beyond the ground state. Various obstacles which historically have doomed such an approach to be unfeasible are overcome. First, we resort to a generalization of the Ritz variational principle to ensemble states with fixed weights. This in combination with the constrained search formalism allows us to establish a universal functional of the one-particle reduced density matrix. Second, we employ tools from convex analysis to circumvent the too involved N-representability constraints. Remarkably, this identifies Valone's pioneering work on RDMFT as a special case of convex relaxation and reveals that crucial information about the excitation structure is contained in the functional's domain. Third, to determine the crucial latter object, a methodology is developed which eventually leads to a generalized exclusion principle. The corresponding linear constraints are calculated for systems of arbitrary size.

Ensemble generalized Kohn-Sham theory: The good, the bad, and the ugly

Two important extensions of Kohn-Sham (KS) theory are generalized: KS theory and ensemble KS theory. The former allows for non-multiplicative potential operators and greatly facilitates practical calculations with advanced, orbital-dependent functionals. The latter allows for quantum ensembles and enables the treatment of open systems and excited states. Here, we combine the two extensions, both formally and practically, first via an exact yet complicated formalism and then via a computationally tractable variant that involves a controlled approximation of ensemble "ghost interactions" by means of an iterative algorithm. The resulting formalism is illustrated using selected examples. This opens the door to the application of generalized KS theory in more challenging quantum scenarios and to the improvement of ensemble theories for the purpose of practical and accurate calculations.

Ensemble Density Functional Theory: Insight from the Fluctuation-Dissipation Theorem

Density functional theory can be generalized to mixtures of ground and excited states, for the purpose of determining energies of excitations using low-cost density functional approximations. Adapting approximations originally developed for ground states to work in the new setting would fast-forward progress enormously. But, previous attempts have stumbled on daunting fundamental issues. Here we show that these issues can be prevented from the outset, by using a fluctuation dissipation theorem (FDT) to dictate key functionals. We thereby show that existing exchange energy approximations are readily adapted to excited states, when combined with a rigorous exact Hartree term that is different in form from its ground state counterpart, and counterparts based on ensemble Ansatzë. Applying the FDT to correlation energies also provides insights into ground statelike and ensemble-only correlations. We thus provide a comprehensive and versatile framework for ensemble density functional approximations.

Approximately Self-Consistent Ensemble Density Functional Theory: Toward Inclusion of All Correlations

Recent theory developments in ensemble density functional theory (EDFT) promise to bring decades of work for ground states to the practical resolution of excited states, provided newly discovered "density-driven correlations" can be dealt with and adequate effective potentials can be found. This Letter introduces simple theories for both and shows that EDFT using these theories outperforms ΔSCF DFT and time-dependent DFT for low-lying gaps in most of the small atoms and molecules tested, even when all use the same density functional approximations. It thus establishes EDFT as a promising tool for low-cost studies of low-lying excited states and provides a clear route to practical EDFT implementation of arbitrary functional approximations.

Individual Correlations in Ensemble Density Functional Theory: State- and Density-Driven Decompositions without Additional Kohn-Sham Systems

Gould and Pittalis [Phys. Rev. Lett. 123, 016401 (2019)PRLTAO0031-900710.1103/PhysRevLett.123.016401 recently revealed a density-driven (DD) correlation energy that is specific to many-electron ensembles and must be accounted for by approximations. We derive in this Letter a general and simpler expression in terms of the ensemble weights, the ensemble Kohn-Sham (KS) orbitals, and their linear response to variations in the ensemble weights. As no additional state-driven KS systems are needed, its evaluation is greatly simplified. We confirm the importance of DD effects and introduce a direct and promising route to approximations.

Weight dependence of local exchange–correlation functionals in ensemble density-functional theory: double excitations in two-electron systems

We discuss the construction of first-rung weight-dependent exchange–correlation density-functional approximations for He and H₂ specifically designed for the computation of double excitations within Gross–Oliveira–Kohn-DFT.

Density-Driven Correlations in Ensemble Density Functional Theory: Insights from Simple Excitations in Atoms

Ensemble density functional theory extends the usual Kohn-Sham machinery to quantum state ensembles involving ground- and excited states. Recent work by the authors [Phys. Rev. Lett. 119, 243001 (2017); 123, 016401 (2019)] has shown that both the Hartree-exchange and correlation energies can attain unusual features in ensembles. Density-driven (DD) correlations – which account for the fact that pure-state densities in Kohn-Sham ensembles do not necessarily reproduce those of interacting pure states – are one such feature. Here we study atoms (specifically S–P and S–S transitions) and show that the magnitude and behaviour of DD correlations can vary greatly with the variation of the orbital angular momentum of the involved states. Such estimations are obtained through an approximation for DD correlations built from relevant exact conditions, Kohn-Sham inversion, and plausible assumptions for weakly correlated systems.