The war of attrition with random rewards

Game theory in biology: 50 years and onwards

Game theory in biology gained prominence 50 years ago, when Maynard Smith & Price formulated the concept of an evolutionarily stable strategy (ESS). Their aim was to explain why conflicts between animals of the same species usually are of a 'limited war' type, not causing serious injury. They emphasized that game theory is an alternative to previous ideas about group selection, which were used by ethologists to explain limited aggression. Subsequently, the ESS concept was applied to many phenomena with frequency dependence in the evolutionary success of strategies, including sex allocation, alternative mating types, contest behaviour and signalling, cooperation, and parental care. Both the analyses of signalling and cooperation were inspired by similar problems in economics and attracted much attention in biology. Here we give a perspective on which of the ambitions in the field have been achieved, with a focus on contest behaviour and cooperation. We evaluate whether the game-theoretical study of the evolution of cooperation has measured up to expectations in explaining the behaviour of non-human animals. We also point to potentially fruitful directions for the field, and emphasize the importance of incorporating realistic behavioural mechanisms into models. This article is part of the theme issue 'Half a century of evolutionary games: a synthesis of theory, application and future directions'.

The early rise and spread of evolutionary game theory: perspectives based on recollections of early workers

Though the first attempts to introduce game theory into evolutionary biology failed, new formalism by Maynard Smith and Price in 1973 had almost instant success. We use information supplied by early workers to analyse how and why evolutionary game theory (EGT) spread so rapidly in its earliest years. EGT was a major tool for the rapidly expanding discipline of behavioural ecology in the 1970s; each catalysed the other. The first models were applied to animal contests, and early workers sought to improve their biological reality to compare predictions with observations. Furthermore, it was quickly realized that EGT provided a general evolutionary modelling method; not only was it swiftly applied to diverse phenotypic adaptations in evolutionary biology, it also attracted researchers from other disciplines such as mathematics and economics, for which game theory was first devised. Lastly, we pay attention to exchanges with population geneticists, considering tensions between the two modelling methods, as well as efforts to bring them closer. This article is part of the theme issue 'Half a century of evolutionary games: a synthesis of theory, application and future directions'.

Persistent homology and the shape of evolutionary games

For nearly three decades, spatial games have produced a wealth of insights to the study of behavior and its relation to population structure. However, as different rules and factors are added or altered, the dynamics of spatial models often become increasingly complicated to interpret. To tackle this problem, we introduce persistent homology as a rigorous framework that can be used to both define and compute higher-order features of data in a manner which is invariant to parameter choices, robust to noise, and independent of human observation. Our work demonstrates its relevance for spatial games by showing how topological features of simulation data that persist over different spatial scales reflect the stability of strategies in 2D lattice games. To do so, we analyze the persistent homology of scenarios from two games: a Prisoner's Dilemma and a SIRS epidemic model. The experimental results show how the method accurately detects features that correspond to real aspects of the game dynamics. Unlike other tools that study dynamics of spatial systems, persistent homology can tell us something meaningful about population structure while remaining neutral about the underlying structure itself. Regardless of game complexity, since strategies either succeed or fail to conform to shapes of a certain topology there is much potential for the method to provide novel insights for a wide variety of spatially extended systems in biology, social science, and physics.

Attack versus defense: A strategic rationale for role differentiation in conflict

Is there a strategic mechanism that explains role-contingent differences in conflict behavior? I sketch a theory in which differences in optimal behavior for attackers and defenders arise under initially symmetric conditions through the dynamic accumulation of differences in the distributions of traits in the subpopulations of potential opponents.

The joint evolution of cooperation and competition

In nature, cooperation among individuals is often accompanied by competition among the same individuals for the cooperatively produced rewards. In such a situation, the evolution of cooperative and competitive investments influences each other, but previous theoretical studies mostly focused on either cooperation or competition. Here we consider a generic situation in which individuals cooperatively produce rewards according to the continuous snowdrift game, and then rewards are divided among cooperating individuals according to a generalized tug-of-war game. Using adaptive dynamics and numerical simulations, we investigated the joint evolution of two continuous traits, the investment in cooperation and in competition, respectively. We found that competition for the division of rewards promotes evolutionary branching, and hence polymorphism in both the cooperative and the competitive traits. In polymorphic populations, cooperation levels are positively correlated with competition levels among strains, so that cooperators tend to benefit disproportionately from the benefits produced. We also found that the mean cooperation level within the population is promoted by the competition. Our results show that coevolution of cooperation and competition has qualitatively different outcomes compared to the evolution of only cooperation or only competition, and suggest that it is important to simultaneously consider multiple aspects of social interactions.

Conflicts of interest improve collective computation of adaptive social structures

In many biological systems, the functional behavior of a group is collectively computed by the system's individual components. An example is the brain's ability to make decisions via the activity of billions of neurons. A long-standing puzzle is how the components' decisions combine to produce beneficial group-level outputs, despite conflicts of interest and imperfect information. We derive a theoretical model of collective computation from mechanistic first principles, using results from previous work on the computation of power structure in a primate model system. Collective computation has two phases: an information accumulation phase, in which (in this study) pairs of individuals gather information about their fighting abilities and make decisions about their dominance relationships, and an information aggregation phase, in which these decisions are combined to produce a collective computation. To model information accumulation, we extend a stochastic decision-making model-the leaky integrator model used to study neural decision-making-to a multiagent game-theoretic framework. We then test alternative algorithms for aggregating information-in this study, decisions about dominance resulting from the stochastic model-and measure the mutual information between the resultant power structure and the "true" fighting abilities. We find that conflicts of interest can improve accuracy to the benefit of all agents. We also find that the computation can be tuned to produce different power structures by changing the cost of waiting for a decision. The successful application of a similar stochastic decision-making model in neural and social contexts suggests general principles of collective computation across substrates and scales.

Evolutionary dynamics of a smoothed war of attrition game

In evolutionary game theory the War of Attrition game is intended to model animal contests which are decided by non-aggressive behavior, such as the length of time that a participant will persist in the contest. The classical War of Attrition game assumes that no errors are made in the implementation of an animal׳s strategy. However, it is inevitable in reality that such errors must sometimes occur. Here we introduce an extension of the classical War of Attrition game which includes the effect of errors in the implementation of an individual׳s strategy. This extension of the classical game has the important feature that the payoff is continuous, and as a consequence admits evolutionary behavior that is fundamentally different from that possible in the original game. We study the evolutionary dynamics of this new game in well-mixed populations both analytically using adaptive dynamics and through individual-based simulations, and show that there are a variety of possible outcomes, including simple monomorphic or dimorphic configurations which are evolutionarily stable and cannot occur in the classical War of Attrition game. In addition, we study the evolutionary dynamics of this extended game in a variety of spatially and socially structured populations, as represented by different complex network topologies, and show that similar outcomes can also occur in these situations.

Biological auctions with multiple rewards

The competition for resources among cells, individuals or species is a fundamental characteristic of evolution. Biological all-pay auctions have been used to model situations where multiple individuals compete for a single resource. However, in many situations multiple resources with various values exist and single reward auctions are not applicable. We generalize the model to multiple rewards and study the evolution of strategies. In biological all-pay auctions the bid of an individual corresponds to its strategy and is equivalent to its payment in the auction. The decreasingly ordered rewards are distributed according to the decreasingly ordered bids of the participating individuals. The reproductive success of an individual is proportional to its fitness given by the sum of the rewards won minus its payments. Hence, successful bidding strategies spread in the population. We find that the results for the multiple reward case are very different from the single reward case. While the mixed strategy equilibrium in the single reward case with more than two players consists of mostly low-bidding individuals, we show that the equilibrium can convert to many high-bidding individuals and a few low-bidding individuals in the multiple reward case. Some reward values lead to a specialization among the individuals where one subpopulation competes for the rewards and the other subpopulation largely avoids costly competitions. Whether the mixed strategy equilibrium is an evolutionarily stable strategy (ESS) depends on the specific values of the rewards.

Condition, not eyespan, predicts contest outcome in female stalk-eyed flies, Teleopsis dalmanni

In contests among males, body condition is often the key determinant of a successful outcome, with fighting ability signaled by so-called armaments, that is, exaggerated, condition-dependent traits. However, it is not known whether condition and exaggerated traits function in the same way in females. Here, we manipulated adult condition by varying larval nutrition in the stalk-eyed fly, Teleopsis dalmanni, a species in which eyespan is exaggerated in both sexes, and we measured the outcome of contests between females of similar or different body condition and relative eyespan. We found that females in higher condition, with both larger bodies and eyespan, won a higher proportion of encounters when competing against rivals of lower condition. However, when females were of equal condition, neither eyespan nor body length had an effect on the outcome of a contest. An analysis of previously published data revealed a similar pattern in males: individuals with large relative eyespan did not win significantly more encounters when competing with individuals of a similar body size. Contrary to expectations, and to previous findings in males, there was no clear effect of differences in body size or eyespan affecting contest duration in females. Taken together, our findings suggest that although eyespan can provide an honest indicator of condition, large eyespans provide no additional benefit to either sex in intrasexual aggressive encounters; body size is instead the most important factor.

Evidence for assessment disappears in mixed-sex contests of the crayfish, Orconectes virilis

During agonistic interactions, decisions about contest persistence can be informed by assessment of one’s own energy or time expenditure (self-assessment), one’s own expenditure combined with opponent inflicted costs (cumulative assessment), or through information exchange with an opponent (mutual assessment). Females and males can be expected to exhibit different strategies for contest resolution due to contrasting energetic requirements and resource valuation. We examined the assessment strategies crayfish employ during same-sex and mixed-sex fights. Two individuals interacted for 15 min, and fight duration and times spent at various intensity levels were quantified. Results indicated that both sexes employ a self-assessment strategy during same-sex fights. Evidence for assessment during mixed-sex fights was notably weaker, suggesting the resolution of mixed-sex fights involves different behavioural elements and/or sources of information. In species where mixed-sex fights are common year-round, the lack of common rules can lead to greater energy expenditure for both sexes.

Evolutionary dynamics of biological auctions

Many scenarios in the living world, where individual organisms compete for winning positions (or resources), have properties of auctions. Here we study the evolution of bids in biological auctions. For each auction, n individuals are drawn at random from a population of size N. Each individual makes a bid which entails a cost. The winner obtains a benefit of a certain value. Costs and benefits are translated into reproductive success (fitness). Therefore, successful bidding strategies spread in the population. We compare two types of auctions. In "biological all-pay auctions", the costs are the bid for every participating individual. In "biological second price all-pay auctions", the cost for everyone other than the winner is the bid, but the cost for the winner is the second highest bid. Second price all-pay auctions are generalizations of the "war of attrition" introduced by Maynard Smith. We study evolutionary dynamics in both types of auctions. We calculate pairwise invasion plots and evolutionarily stable distributions over the continuous strategy space. We find that the average bid in second price all-pay auctions is higher than in all-pay auctions, but the average cost for the winner is similar in both auctions. In both cases, the average bid is a declining function of the number of participants, n. The more individuals participate in an auction the smaller is the chance of winning, and thus expensive bids must be avoided.

The aggressive behavior of Nile tilapia introduced into novel environments with variation in enrichment

Many studies show environmental enrichment is correlated with benefits to captive animals; however, one should not always assume this positive relationship given that enrichment increases the amount of resources that a territorial animal must defend and possibly affects its aggressive dynamics. In this study, we tested if environmental enrichment affects aggressive interactions in the aggressive fish Nile tilapia (Oreochromis niloticus). We compared fights staged between pairs of male tilapia of similar size (= matched in resource holding potential) in a novel arena that was either barren or enriched, to examine whether enrichment enhances territory value in line with theoretical predictions, with the potential for compromised welfare. We evaluated time elapsed until the first attack (latency), frequency of aggressive interactions and fight duration. We detected fight dynamic differences at the pair level. Higher resource value generated increased aggression but had no effect on fight duration or latency. This conclusion is in line with game theory predictions concerning resource value and contradicts the theory that enrichment of the environment will serve welfare purposes.

Resource holding potential, subjective resource value, and game theoretical models of aggressiveness signalling

Empirical evidence suggests that aggressiveness (willingness to enter into, or escalate an aggressive interaction) may be more important than the ability to win fights in some species. Both empirical and theoretical traditions treat aggressiveness as a distinct property from the ability (RHP) or motivation (subjective resource value) to win a fight. I examine how these three traits are clearly distinct when modelled using a simple strategic model of escalation. I then examine game theoretical models of agonistic communication and demonstrate that models in which aggressiveness is signalled require: (1) a trait, aggressiveness, which is neither a correlate, nor consequence of RHP or motivation, (2) a handicap which negates any benefit to be gained through the use of a particular signal, and (3) the absence of any other asymmetry which could be used to assign roles to players. I conclude that it is unlikely that these assumptions are ever met, and that empirical examples of "aggressiveness" are far more likely to represent long-term differences in subjective resource value.

Time-dependent animal conflicts: 1. The symmetric case

Animal conflicts are often characterized by time-dependent strategy sets. This paper considers the following type of animal conflicts: a member of a group is at risk and needs the assistance of another member to be saved. As long as assistance is not provided, the individual which is at risk has a positive, time-dependent rate of dying. Each of the other group members is a potential helper. Assisting this individual accrues a cost, but losing him decreases the inclusive fitness of each group member. A potential helper's interval between the moment an individual finds itself at risk and the moment it assists is a random variable, hence its strategy is to choose the probability distribution for this random variable. Assuming that each of the potential helpers knows the others' strategies, we show that the ability to observe their realizations influences the evolutionarily stable strategies (ESS) of the game. According to our results, where the realizations can be observed ESS always exist: immediate assistance, no assistance and delayed assistance. Where the realizations cannot be observed ESS do not always exist, immediate assistance and no assistance are possible ESS, while delayed assistance cannot be an ESS. We apply our model to the n brothers' problem and to the parental investment conflict.

Time-dependent animal conflicts: 2. The asymmetric case

This paper presents an asymmetric game-theoretical model to the following type of animal conflicts: a member of a group is at risk and needs the help of another member to be saved. As long as assistance is not provided, this individual has a positive, time-dependent rate of dying. Assisting the individual which is at risk accrues a cost, but losing it decreases each member's inclusive fitness. A potential helper's interval between the moment a group member gets into trouble and the moment it assists is a random variable, hence its strategy is to choose the distribution of this random variable. In the asymmetric conflict all the potential helpers have identical strategy sets, but each plays a different role. For example, male or female and young or old. We consider both payoff-irrelevant asymmetry and payoff-relevant asymmetry and characterize each role's stable replies. The evolutionarily stable strategies (ESS) are computed, and the model is applied to the n brothers' problem. According to our results immediate assistance and no assistance are possible ESS both under payoff-relevant asymmetry and under payoff-relevant asymmetry.

Patch leaving strategies and superparasitism: an asymmetric generalized war of attrition

When several competitors deplete a patch, it can be advantageous for each of them to stay provided that others leave, whereas, on the other hand, staying longer decreases the expected payoff for everyone. This situation can be considered as a generalized war of attrition. Previous studies have shown that optimal patch leaving strategies become stochastic and the expected leaving time is much larger than predicted by the marginal value theorem when competitors interfere. The possibility of superparasitism, as occurs for example in parasitoids, induces such interference. In addition, it gives several complications. First, the payoff of females that have left the patch is affected by the ovipositions of the remaining individuals. Second, differences in the arrival time of females cause payoff-relevant asymmetries, since females that arrived early on have parasitized more hosts in a patch at the moment superparasitism starts than those that arrived later. We show that this can be modelled as an asymmetric generalized war of attrition, and derive global characteristics of the ESS for simultaneous decisions on when to start superparasitism and when to leave a patch.

Female sex pheromone-mediated effects on behavior and consequences of male competition in the shore crab (Carcinus maenas)

Exposure to receptive female pheromone elicits guarding behavior in shore crab males (Carcinus maenas), but little is known about the effects of sex pheromone on male competition or if the female plays an active role in mate choice. This study examined whether female pheromone enhanced agonistic behavior between males and what effects visual and chemical cues had on the rules and costs of such contests. We also investigated whether females exhibit a preference for males in terms of size. Under laboratory conditions, solitary male shore crabs engaged males who already had possession of a female. The visual and chemical presence of a receptive female had an impact on contest rules and costs. Fights were costly in terms of duration and of sustaining injury with either one or both crabs incurring injury in 40% of fights. To investigate the metabolic consequences of fighting over a perceived sexual resource (chemical cue only), fights were staged between pairs of size-matched males in the presence of water containing the female sex pheromone, water in which males had been kept, and untreated seawater. The duration and intensity of contests were greater when staged in the presence of the female pheromone compared with the two other treatments. Crabs that fought in the presence of female sex pheromone also had a greater accumulation of L-lactate and a reduction of glycogen stores. Fights were less costly in terms of injury with a single chemical cue compared with enhanced costs with a multiple sexual cue. The importance of female choice was determined by presenting postmolt females with different sized males. Males were kept in a fixed position, and the majority of females approached and performed courtship behavior to the largest males, demonstrating that females may be selective in terms of size.

The ESS in an asymmetric generalized war of attrition with mistakes in role perception

We derive the ESS for the generalized asymmetric war of attrition, where payoffs to contestants may vary in time and may depend on some characteristic, called the "role" of an individual. We use the same approach as Hammerstein & Parker (1982), who examined an asymmetric war of attrition. We consider two roles, A and B. Role A is assumed to be favoured with respect to payoffs. It is assumed that there is always a true asymmetry, so in each contest one individual has role A and the other has role B. It is assumed that roles are assigned to contestants at random and that they can make mistakes in role perception. It is shown that, under certain assumptions about shapes of payoff functions and probabilities of making mistakes, there is an ESS which can be characterized by two probability distributions with non-overlapping support. Individuals who perceive their role as A should choose larger persistence times. This ESS structure is similar to that in the asymmetric war of attrition. In that model, the resource values and the cost rates are constant. We consider situations where all these values may change in time and where rewards and costs may be equal after some finite time. Such shapes of payoff functions arise naturally in competitive patch depletion (Sjerps & Haccou, 1994a,b). As a result, the probability density functions that specify the conditional strategies are no longer necessarily negative exponentials (as in the war of attrition), but may have very different shapes. Furthermore, under some conditions there is a maximum persistence time, at which there can be an atom of probability. We give explicit expressions as well as numerical approximations for the ESS.

Threat display in birds

The study of threat displays has long been an area in which theory and empirical work have each spurred the other forward. Communication is currently the focus of great interest and effort on the part of modellers. A great deal that classical ethologists have accurately described about threat displays still lacks adequate explanation. Here we review the empirical literature on the use of threat displays by birds competing for small valued resources, both to refocus theoretical attention upon the key characteristics of threat and to assess the degree to which current theory explains these characteristics. We aim to demonstrate that threat displays communicate information about aggressive motivation, but are not handicaps. Handicap models predict a single graded display, while the vast majority of studies report repertoires of about four to six discrete threats for any given species. These displays vary with motivational and strategic considerations, and may be demonstrated to rank consistently on a scale of willingness to escalate, thus providing information about aggressive motivation. We conclude by identifying those features of avian threat displays that have not been adequately explained, in the hope that this reexamination of empirical data will help focus theoretical attention on these issues.

Discrete conventional signalling of a continuous variable

In aggressive interactions, animals often use a discrete set of signals, while the properties being signalled are likely to be continuous, for example fighting ability or value of victory. Here we investigate a particular model of fighting that allows for conventional signalling of subjective resource value to occur. Perfect signalling and no signalling are not evolutionarily stable strategies (ESSs) in the model. Instead, we find ESSs in which partial information is communicated, with discrete displays signalling a range of values rather than a precise one. The result also indicates that communication should be more precise in conflicts over small resources. Signalling strategies can exist in fighting because of the common interest in avoiding injuries, but communication is likely to be limited because of the fundamental conflict over the resource. Our results reflect a compromise between these two factors. Data allowing for a thorough test of the model are lacking; however, existing data seem consistent with the obtained theoretical results. Copyright 1998 The Association for the Study of Animal Behaviour.

Evolutionarily [corrected] stable strategies: a review of basic theory

Widely successful in applied population biology, the Evolutionarily Stable Strategy concept remains controversial because of the severe restrictions present in its original formulation. We review theory which explores and relaxes these restrictions, finding the concept to be quite robust and adaptable, incorporating considerations such as genetics, population diversity, environmental variability, and mutation.