Function-valued adaptive dynamics and the calculus of variations

Evolutionary game theory and the adaptive dynamics approach: adaptation where individuals interact

Evolutionary game theory and the adaptive dynamics approach have made invaluable contributions to understanding how gradual evolution leads to adaptation when individuals interact. Here, we review some of the basic tools that have come out of these contributions to model the evolution of quantitative traits in complex populations. We collect together mathematical expressions that describe directional and disruptive selection in class- and group-structured populations in terms of individual fitness, with the aims of bridging different models and interpreting selection. In particular, our review of disruptive selection suggests there are two main paths that can lead to diversity: (i) when individual fitness increases more than linearly with trait expression; (ii) when trait expression simultaneously increases the probability that an individual is in a certain context (e.g. a given age, sex, habitat, size or social environment) and fitness in that context. We provide various examples of these and more broadly argue that population structure lays the ground for the emergence of polymorphism with unique characteristics. Beyond this, we hope that the descriptions of selection we present here help see the tight links among fundamental branches of evolutionary biology, from life history to social evolution through evolutionary ecology, and thus favour further their integration. This article is part of the theme issue 'Half a century of evolutionary games: a synthesis of theory, application and future directions'.

Hamilton's rule, gradual evolution, and the optimal (feedback) control of phenotypically plastic traits

Most traits expressed by organisms, such as gene expression profiles, developmental trajectories, behavioural sequences and reaction norms are function-valued traits (colloquially "phenotypically plastic traits"), since they vary across an individual's age and in response to various internal and/or external factors (state variables). Furthermore, most organisms live in populations subject to limited genetic mixing and are thus likely to interact with their relatives. We here formalise selection on genetically determined function-valued traits of individuals interacting in a group-structured population, by deriving the marginal version of Hamilton's rule for function-valued traits. This rule simultaneously gives a condition for the invasion of an initially rare mutant function-valued trait and its ultimate fixation in the population (invasion thus implies substitution). Hamilton's rule thus underlies the gradual evolution of function-valued traits and gives rise to necessary first-order conditions for their uninvadability (evolutionary stability). We develop a novel analysis using optimal control theory and differential game theory, to simultaneously characterise and compare the first-order conditions of (i) open-loop traits - functions of time (or age) only, and (ii) closed-loop (state-feedback) traits - functions of both time and state variables. We show that closed-loop traits can be represented as the simpler open-loop traits when individuals do not interact or when they interact with clonal relatives. Our analysis delineates the role of state-dependence and interdependence between individuals for trait evolution, which has implications to both life-history theory and social evolution.

Exploring Evolutionary Fitness in Biological Systems Using Machine Learning Methods

Here, we propose a computational approach to explore evolutionary fitness in complex biological systems based on empirical data using artificial neural networks. The essence of our approach is the following. We first introduce a ranking order of inherited elements (behavioral strategies or/and life history traits) in considered self-reproducing systems: we use available empirical information on selective advantages of such elements. Next, we introduce evolutionary fitness, which is formally described as a certain function reflecting the introduced ranking order. Then, we approximate fitness in the space of key parameters using a Taylor expansion. To estimate the coefficients in the Taylor expansion, we utilize artificial neural networks: we construct a surface to separate the domains of superior and interior ranking of pair inherited elements in the space of parameters. Finally, we use the obtained approximation of the fitness surface to find the evolutionarily stable (optimal) strategy which maximizes fitness. As an ecologically important study case, we apply our approach to explore the evolutionarily stable diel vertical migration of zooplankton in marine and freshwater ecosystems. Using machine learning we reconstruct the fitness function of herbivorous zooplankton from empirical data and predict the daily trajectory of a dominant species in the northeastern Black Sea.

The Evolution of Immigration Strategies Facilitates Niche Expansion by Divergent Adaptation in a Structured Metapopulation Model

Local adaptation and habitat choice are two key factors that control the distribution and diversification of species. Here we model habitat choice mechanistically as the outcome of dispersal with nonrandom immigration. We consider a structured metapopulation with a continuous distribution of patch types and determine the evolutionarily stable immigration strategy as the function linking patch type to the probability of settling in the patch on encounter. We uncover a novel mechanism whereby coexisting strains that only slightly differ in their local adaptation trait can evolve substantially different immigration strategies. In turn, different habitat use selects for divergent adaptations in the two strains. We propose that the joint evolution of immigration and local adaptation can facilitate diversification and discuss our results in the light of niche conservatism versus niche expansion.

Revealing Evolutionarily Optimal Strategies in Self-Reproducing Systems via a New Computational Approach

Modelling the evolution of complex life history traits and behavioural patterns observed in the natural world is a challenging task. Here, we develop a novel computational method to obtain evolutionarily optimal life history traits/behavioural patterns in population models with a strong inheritance. The new method is based on the reconstruction of evolutionary fitness using underlying equations for population dynamics and it can be applied to self-reproducing systems (including complicated age-structured models), where fitness does not depend on initial conditions, however, it can be extended to some frequency-dependent cases. The technique provides us with a tool to efficiently explore both scalar-valued and function-valued traits with any required accuracy. Moreover, the method can be implemented even in the case where we ignore the underlying model equations and only have population dynamics time series. As a meaningful ecological case study, we explore optimal strategies of diel vertical migration (DVM) of herbivorous zooplankton in the vertical water column which is a widespread phenomenon in both oceans and lakes, generally considered to be the largest synchronised movement of biomass on Earth. We reveal optimal trajectories of daily vertical motion of zooplankton grazers in the water column depending on the presence of food and predators. Unlike previous studies, we explore both scenarios of DVM with static and dynamic predators. We find that the optimal pattern of DVM drastically changes in the presence of dynamic predation. Namely, with an increase in the amount of food available for zooplankton grazers, the amplitude of DVM progressively increases, whereas for static predators DVM would abruptly cease.

Towards the Construction of a Mathematically Rigorous Framework for the Modelling of Evolutionary Fitness

Modelling of natural selection in self-replicating systems has been heavily influenced by the concept of fitness which was inspired by Darwin’s original idea of the survival of the fittest. However, so far the concept of fitness in evolutionary modelling is still somewhat vague, intuitive and often subjective. Unfortunately, as a result of this, using different definitions of fitness can lead to conflicting evolutionary outcomes. Here we formalise the definition of evolutionary fitness to describe the selection of strategies in deterministic self-replicating systems for generic modelling settings which involve an arbitrary function space of inherited strategies. Our mathematically rigorous definition of fitness is closely related to the underlying population dynamic equations which govern the selection processes. More precisely, fitness is defined based on the concept of the ranking of competing strategies which compares the long-term dynamics of measures of sets of inherited units in the space of strategies. We also formulate the variational principle of modelling selection which states that in a self-replicating system with inheritance, selection will eventually maximise evolutionary fitness. We demonstrate how expressions for evolutionary fitness can be derived for a class of models with age structuring including systems with delay, which has previously been considered as a challenge.

Towards developing a general framework for modelling vertical migration in zooplankton

Diel vertical migration (DVM) of zooplankton is a widespread phenomenon in both oceans and lakes, and is generally considered to be the largest synchronized movement of biomass on Earth. Most existing mathematical models of DVM are based on the assumption that animals maximize a certain criterion such as the expected reproductive value, the venturous revenue, the ratio of energy gain/mortality or some predator avoidance function when choosing their instantaneous depth. The major shortcoming of this general point of view is that the predicted DVM may be strongly affected by a subjective choice of a particular optimization criterion. Here we argue that the optimal strategy of DVM can be unambiguously obtained as an outcome of selection in the underlying equations of genotype/traits frequency dynamics. Using this general paradigm, we explore the optimal strategy for the migration across different depths by zooplankton grazers throughout the day. To illustrate our ideas we consider four generic DVM models, each making different assumptions on the population dynamics of zooplankton, and demonstrate that in each model we need to maximize a particular functional to find the optimal strategy. Surprisingly, patterns of DVM obtained for different models greatly differ in terms of their parameters dependence. We then show that the infinite dimensional trait space of different zooplankton trajectories can be projected onto a low dimensional space of generalized parameters and the genotype evolution dynamics can be easily followed using this low-dimensional space. Using this space of generalized parameters we explore the influence of mutagenesis on evolution of DVM, and we show that strong mutagenesis allows the coexistence of an infinitely large number of strategies whereas for weak mutagenesis the selection results in the extinction of most strategies, with the surviving strategies all staying close to the optimal strategy in the corresponding mutagenesis-free system.

The canonical equation of adaptive dynamics for life histories: from fitness-returns to selection gradients and Pontryagin's maximum principle

This paper should be read as addendum to Dieckmann et al. (J Theor Biol 241:370–389, 2006) and Parvinen et al. (J Math Biol 67: 509–533, 2013). Our goal is, using little more than high-school calculus, to (1) exhibit the form of the canonical equation of adaptive dynamics for classical life history problems, where the examples in Dieckmann et al. (J Theor Biol 241:370–389, 2006) and Parvinen et al. (J Math Biol 67: 509–533, 2013) are chosen such that they avoid a number of the problems that one gets in this most relevant of applications, (2) derive the fitness gradient occurring in the CE from simple fitness return arguments, (3) show explicitly that setting said fitness gradient equal to zero results in the classical marginal value principle from evolutionary ecology, (4) show that the latter in turn is equivalent to Pontryagin’s maximum principle, a well known equivalence that however in the literature is given either ex cathedra or is proven with more advanced tools, (5) connect the classical optimisation arguments of life history theory a little better to real biology (Mendelian populations with separate sexes subject to an environmental feedback loop), (6) make a minor improvement to the form of the CE for the examples in Dieckmann et al. and Parvinen et al.

Evolution of complex density-dependent dispersal strategies

The question of how dispersal behavior is adaptive and how it responds to changes in selection pressure is more relevant than ever, as anthropogenic habitat alteration and climate change accelerate around the world. In metapopulation models where local populations are large, and thus local population size is measured in densities, density-dependent dispersal is expected to evolve to a single-threshold strategy, in which individuals stay in patches with local population density smaller than a threshold value and move immediately away from patches with local population density larger than the threshold. Fragmentation tends to convert continuous populations into metapopulations and also to decrease local population sizes. Therefore we analyze a metapopulation model, where each patch can support only a relatively small local population and thus experience demographic stochasticity. We investigated the evolution of density-dependent dispersal, emigration and immigration, in two scenarios: adult and natal dispersal. We show that density-dependent emigration can also evolve to a nonmonotone, "triple-threshold" strategy. This interesting phenomenon results from an interplay between the direct and indirect benefits of dispersal and the costs of dispersal. We also found that, compared to juveniles, dispersing adults may benefit more from density-dependent vs. density-independent dispersal strategies.

Function-valued adaptive dynamics and optimal control theory

In this article we further develop the theory of adaptive dynamics of function-valued traits. Previous work has concentrated on models for which invasion fitness can be written as an integral in which the integrand for each argument value is a function of the strategy value at that argument value only. For this type of models of direct effect, singular strategies can be found using the calculus of variations, with singular strategies needing to satisfy Euler's equation with environmental feedback. In a broader, more mechanistically oriented class of models, the function-valued strategy affects a process described by differential equations, and fitness can be expressed as an integral in which the integrand for each argument value depends both on the strategy and on process variables at that argument value. In general, the calculus of variations cannot help analyzing this much broader class of models. Here we explain how to find singular strategies in this class of process-mediated models using optimal control theory. In particular, we show that singular strategies need to satisfy Pontryagin's maximum principle with environmental feedback. We demonstrate the utility of this approach by studying the evolution of strategies determining seasonal flowering schedules.

Impact of environmental covariation in growth and mortality on evolving maturation reaction norms

Maturation age and size have important fitness consequences through their effects on survival probabilities and body sizes. The evolution of maturation reaction norms in response to environmental covariation in growth and mortality is therefore a key subject of life-history theory. The eco-evolutionary model we present and analyze here incorporates critical features that earlier studies of evolving maturation reaction norms have often neglected: the trade-off between growth and reproduction, source-sink population structure, and population regulation through density-dependent growth and fecundity. We report the following findings. First, the evolutionarily optimal age at maturation can be decomposed into the sum of a density-dependent and a density-independent component. These components measure, respectively, the hypothetical negative age at which an individual's length would be 0 and the delay in maturation relative to this offset. Second, along any growth trajectory, individuals mature earlier when mortality is higher. This allows us to deduce, third, how the shapes of evolutionarily optimal maturation reaction norms depend on the covariation between growth and mortality (positive or negative, linear or curvilinear, and deterministic or probabilistic). Providing eco-evolutionary explanations for many alternative reaction-norm shapes, our results appear to be in good agreement with current empirical knowledge on maturation dynamics.

A differential game theoretical analysis of mechanistic models for territoriality

In this paper, elements of differential game theory are used to analyze a spatially explicit home range model for interacting wolf packs when movement behavior is uncertain. The model consists of a system of partial differential equations whose parameters reflect the movement behavior of individuals within each pack and whose steady-state solutions describe the patterns of space-use associated to each pack. By controlling the behavioral parameters in a spatially-dynamic fashion, packs adjust their patterns of movement so as to find a Nash-optimal balance between spreading their territory and avoiding conflict with hostile neighbors. On the mathematical side, we show that solving a nonzero-sum differential game corresponds to finding a non-invasible function-valued trait. From the ecological standpoint, when movement behavior is uncertain, the resulting evolutionarily stable equilibrium gives rise to a buffer-zone, or a no-wolf's land where deer are known to find refuge.

Evolution of age-dependent sex-reversal under adaptive dynamics

We investigate the evolution of the age (or size) at sex-reversal in a model of sequential hermaphroditism, by means of the function-valued adaptive dynamics. The trait is the probability law of the age at sex-reversal considered as a random variable. Our analysis starts with the ecological model which was first introduced and analyzed by Calsina and Ripoll (Math Biosci 208(2), 393-418, 2007). The structure of the population is extended to a genotype class and a new model for an invading/mutant population is introduced. The invasion fitness functional is derived from the ecological setting, and it turns out to be controlled by a formula of Shaw-Mohler type. The problem of finding evolutionarily stable strategies is solved by means of infinite-dimensional linear optimization. We have found that these strategies correspond to sex-reversal at a single particular age (or size) even if the set of feasible strategies is considerably broader and allows for a probabilistic sex-reversal. Several examples, including in addition the population-dynamical stability, are illustrated. For a special case, we can show that an unbeatable size at sex-reversal must be larger than 69.3% of the expected size at death.

Trait Substitution Sequence process and Canonical Equation for age-structured populations

We are interested in a stochastic model of trait and age-structured population undergoing mutation and selection. We start with a continuous time, discrete individual-centered population process. Taking the large population and rare mutations limits under a well-chosen time-scale separation condition, we obtain a jump process that generalizes the Trait Substitution Sequence process describing Adaptive Dynamics for populations without age structure. Under the additional assumption of small mutations, we derive an age-dependent ordinary differential equation that extends the Canonical Equation. These evolutionary approximations have never been introduced to our knowledge. They are based on ecological phenomena represented by PDEs that generalize the Gurtin-McCamy equation in Demography. Another particularity is that they involve an establishment probability, describing the probability of invasion of the resident population by the mutant one, that cannot always be computed explicitly. Examples illustrate how adding an age-structure enrich the modelling of structured population by including life history features such as senescence. In the cases considered, we establish the evolutionary approximations and study their long time behavior and the nature of their evolutionary singularities when computation is tractable. Numerical procedures and simulations are carried.

Evolutionary suicide

The great majority of species that lived on this earth have gone extinct. These extinctions are often explained by invoking changes in the environment, to which the species has been unable to adapt. Evolutionary suicide is an alternative explanation to such extinctions. It is an evolutionary process in which a viable population adapts in such a way that it can no longer persist. In this paper different models, where evolutionary suicide occurs are discussed, and the theory behind the phenomenon is reviewed.

The adaptive dynamics of function-valued traits

This study extends the framework of adaptive dynamics to function-valued traits. Such adaptive traits naturally arise in a great variety of settings: variable or heterogeneous environments, age-structured populations, phenotypic plasticity, patterns of growth and form, resource gradients, and in many other areas of evolutionary ecology. Adaptive dynamics theory allows analysing the long-term evolution of such traits under the density-dependent and frequency-dependent selection pressures resulting from feedback between evolving populations and their ecological environment. Starting from individual-based considerations, we derive equations describing the expected dynamics of a function-valued trait in asexually reproducing populations under mutation-limited evolution, thus generalizing the canonical equation of adaptive dynamics to function-valued traits. We explain in detail how to account for various kinds of evolutionary constraints on the adaptive dynamics of function-valued traits. To illustrate the utility of our approach, we present applications to two specific examples that address, respectively, the evolution of metabolic investment strategies along resource gradients, and the evolution of seasonal flowering schedules in temporally varying environments.