Coaction versus reciprocity in continuous-time models of cooperation

Nepotism mediates enforced cooperation in asymmetric negotiations

In cooperative societies, group members typically exchange different commodities among each other, which involves an incessant negotiation process. How is the conflict of fitness interests resolved in this continual bargaining process between unequal partners, so that maintaining the cooperative interaction is the best option for all parties involved? Theory predicts that relatedness between group members may alleviate the conflict of fitness interests, thereby promoting the evolution of cooperation. To evaluate the relative importance of relatedness and direct fitness effects in the negotiation process, we experimentally manipulated both the relatedness and mutual behavioral responses of dominant breeders and subordinate helpers in the cooperatively breeding cichlid fish Neolamprologus pulcher. Results show that coercion by breeders is crucial for the performance of alloparental egg care by helpers, but that kinship significantly decreases the need for coercion as predicted by theory. This illustrates the relative importance of kinship and enforcement in the bargaining process.

Human and macaque pairs employ different coordination strategies in a transparent decision game

Many real-world decisions in social contexts are made while observing a partner's actions. To study dynamic interactions during such decisions, we developed a setup where two agents seated face-to-face to engage in game-theoretical tasks on a shared transparent touchscreen display ('transparent games'). We compared human and macaque pairs in a transparent version of the coordination game 'Bach-or-Stravinsky', which entails a conflict about which of two individually-preferred opposing options to choose to achieve coordination. Most human pairs developed coordinated behavior and adopted dynamic turn-taking to equalize the payoffs. All macaque pairs converged on simpler, static coordination. Remarkably, two animals learned to coordinate dynamically after training with a human confederate. This pair selected the faster agent's preferred option, exhibiting turn-taking behavior that was captured by modeling the visibility of the partner's action before one's own movement. Such competitive turn-taking was unlike the prosocial turn-taking in humans, who equally often initiated switches to and from their preferred option. Thus, the dynamic coordination is not restricted to humans but can occur on the background of different social attitudes and cognitive capacities in rhesus monkeys. Overall, our results illustrate how action visibility promotes the emergence and maintenance of coordination when agents can observe and time their mutual actions.

The smell of cooperation: rats increase helpful behaviour when receiving odour cues of a conspecific performing a cooperative task

Reciprocity can explain cooperative behaviour among non-kin, where individuals help others depending on their experience in previous interactions. Norway rats (Rattus norvegicus) cooperate reciprocally according to direct and generalized reciprocity. In a sequence of four consecutive experiments, we show that odour cues from a cooperating conspecific are sufficient to induce the altruistic help of rats in a food-exchange task. When rats were enabled to help a non-cooperative partner while receiving olfactory information from a rat helping a conspecific in a different room, they helped their non-cooperative partner as if it was a cooperative one. We further show that the cues inducing altruistic behaviour are released during the act of cooperation and do not depend on the identity of the cue provider. Remarkably, olfactory cues seem to be more important for cooperation decisions than experiencing a cooperative act per se. This suggests that rats may signal their cooperation propensity to social partners, which increases their chances to receive help in return.

Modeling the formation of social conventions from embodied real-time interactions

What is the role of real-time control and learning in the formation of social conventions? To answer this question, we propose a computational model that matches human behavioral data in a social decision-making game that was analyzed both in discrete-time and continuous-time setups. Furthermore, unlike previous approaches, our model takes into account the role of sensorimotor control loops in embodied decision-making scenarios. For this purpose, we introduce the Control-based Reinforcement Learning (CRL) model. CRL is grounded in the Distributed Adaptive Control (DAC) theory of mind and brain, where low-level sensorimotor control is modulated through perceptual and behavioral learning in a layered structure. CRL follows these principles by implementing a feedback control loop handling the agent’s reactive behaviors (pre-wired reflexes), along with an Adaptive Layer that uses reinforcement learning to maximize long-term reward. We test our model in a multi-agent game-theoretic task in which coordination must be achieved to find an optimal solution. We show that CRL is able to reach human-level performance on standard game-theoretic metrics such as efficiency in acquiring rewards and fairness in reward distribution.

Emergence and suppression of cooperation by action visibility in transparent games

Real-world agents, humans as well as animals, observe each other during interactions and choose their own actions taking the partners' ongoing behaviour into account. Yet, classical game theory assumes that players act either strictly sequentially or strictly simultaneously without knowing each other's current choices. To account for action visibility and provide a more realistic model of interactions under time constraints, we introduce a new game-theoretic setting called transparent games, where each player has a certain probability of observing the partner's choice before deciding on its own action. By means of evolutionary simulations, we demonstrate that even a small probability of seeing the partner's choice before one's own decision substantially changes the evolutionary successful strategies. Action visibility enhances cooperation in an iterated coordination game, but reduces cooperation in a more competitive iterated Prisoner's Dilemma. In both games, "Win-stay, lose-shift" and "Tit-for-tat" strategies are predominant for moderate transparency, while a "Leader-Follower" strategy emerges for high transparency. Our results have implications for studies of human and animal social behaviour, especially for the analysis of dyadic and group interactions.

Quantitative Matching of Clutch Size in Reciprocating Hermaphroditic Worms

Reciprocity [1] is one of the most controversial evolutionary explanations of cooperation among non-kin [2, 3]. For some authors, cognitive capacity of non-human organisms is limiting, and more parsimonious mechanisms should apply [3-5]; for others, the debate is mainly semantic [2, 6], and empirical evidence can be found in a wide range of taxa [7]. However, while the ability to alternate cooperative behaviors does not settle the reciprocity controversy, the capacity to adjust cooperative behavior to the value of received help could prove decisive. Marine polychaete worms Ophryotrocha diadema, as several simultaneous hermaphrodites, do not self-fertilize and have unilateral mating (i.e., they behave either as females or as males during each mating event). They are also external fertilizers and thus cannot store allosperm, which contribute to make them ideal model organisms to investigate reciprocity, since partners usually alternate sexual roles with each other, repeatedly exchanging egg clutch of variable size [8-12]. However, whether the alternation of sexual roles is the result of conditional reciprocity rather than by-product reciprocity has never been tested [13]. Here, we show that O. diadema worms reciprocate eggs conditionally to the partner's behavior and adjust the quality of cooperation according to that of their partners. Moreover, only egg reciprocation offers similar fitness returns via both the female and the male function with respect to non-reciprocating laying strategies. These results document that fine-tuned forms of conditional reciprocity can emerge in cognitively unsophisticated animals, broadening the criteria to recognize conditional reciprocity among animals.

Cheating and punishment in cooperative animal societies

Cheaters-genotypes that gain a selective advantage by taking the benefits of the social contributions of others while avoiding the costs of cooperating-are thought to pose a major threat to the evolutionary stability of cooperative societies. In order for cheaters to undermine cooperation, cheating must be an adaptive strategy: cheaters must have higher fitness than cooperators, and their behaviour must reduce the fitness of their cooperative partners. It is frequently suggested that cheating is not adaptive because cooperators have evolved mechanisms to punish these behaviours, thereby reducing the fitness of selfish individuals. However, a simpler hypothesis is that such societies arise precisely because cooperative strategies have been favoured over selfish ones-hence, behaviours that have been interpreted as 'cheating' may not actually result in increased fitness, even when they go unpunished. Here, we review the empirical evidence for cheating behaviours in animal societies, including cooperatively breeding vertebrates and social insects, and we ask whether such behaviours are primarily limited by punishment. Our review suggests that both cheating and punishment are probably rarer than often supposed. Uncooperative individuals typically have lower, not higher, fitness than cooperators; and when evidence suggests that cheating may be adaptive, it is often limited by frequency-dependent selection rather than by punishment. When apparently punitive behaviours do occur, it remains an open question whether they evolved in order to limit cheating, or whether they arose before the evolution of cooperation.

The Formation of Social Conventions in Real-Time Environments

Why are some behaviors governed by strong social conventions while others are not? We experimentally investigate two factors contributing to the formation of conventions in a game of impure coordination: the continuity of interaction within each round of play (simultaneous vs. real-time) and the stakes of the interaction (high vs. low differences between payoffs). To maximize efficiency and fairness in this game, players must coordinate on one of two equally advantageous equilibria. In agreement with other studies manipulating continuity of interaction, we find that players who were allowed to interact continuously within rounds achieved outcomes with greater efficiency and fairness than players who were forced to make simultaneous decisions. However, the stability of equilibria in the real-time condition varied systematically and dramatically with stakes: players converged on more stable patterns of behavior when stakes are high. To account for this result, we present a novel analysis of the dynamics of continuous interaction and signaling within rounds. We discuss this previously unconsidered interaction between within-trial and across-trial dynamics as a form of social canalization. When stakes are low in a real-time environment, players can satisfactorily coordinate 'on the fly', but when stakes are high there is increased pressure to establish and adhere to shared expectations that persist across rounds.

Negotiation and appeasement can be more effective drivers of sociality than kin selection

Two alternative frameworks explain the evolution of cooperation in the face of conflicting interests. Conflicts can be alleviated by kinship, the alignment of interests by virtue of shared genes, or by negotiation strategies, allowing mutually beneficial trading of services or commodities. Although negotiation often occurs in kin-structured populations, the interplay of kin- and negotiation-based mechanisms in the evolution of cooperation remains an unresolved issue. Inspired by the biology of a cooperatively breeding fish, we developed an individual-based simulation model to study the evolution of negotiation-based cooperation in relation to different levels of genetic relatedness. We show that the evolution of negotiation strategies leads to an equilibrium where subordinates appease dominants by conditional cooperation, resulting in high levels of help and low levels of aggression. This negotiation-based equilibrium can be reached both in the absence of relatedness and in a kin-structured population. However, when relatedness is high, evolution often ends up in an alternative equilibrium where subordinates help their kin unconditionally. The level of help at this kin-selected equilibrium is considerably lower than at the negotiation-based equilibrium, and it corresponds to a level reached when responsiveness is prevented from evolving in the simulations. A mathematical invasion analysis reveals that, quite generally, the alignment of payoffs due to the relatedness of interaction partners tends to impede selection for harsh but effective punishment of defectors. Hence kin structure will often hamper rather than facilitate the evolution of productive cooperation.

Correlated pay-offs are key to cooperation

The general belief that cooperation and altruism in social groups result primarily from kin selection has recently been challenged, not least because results from cooperatively breeding insects and vertebrates have shown that groups may be composed mainly of non-relatives. This allows testing predictions of reciprocity theory without the confounding effect of relatedness. Here, we review complementary and alternative evolutionary mechanisms to kin selection theory and provide empirical examples of cooperative behaviour among unrelated individuals in a wide range of taxa. In particular, we focus on the different forms of reciprocity and on their underlying decision rules, asking about evolutionary stability, the conditions selecting for reciprocity and the factors constraining reciprocal cooperation. We find that neither the cognitive requirements of reciprocal cooperation nor the often sequential nature of interactions are insuperable stumbling blocks for the evolution of reciprocity. We argue that simple decision rules such as 'help anyone if helped by someone' should get more attention in future research, because empirical studies show that animals apply such rules, and theoretical models find that they can create stable levels of cooperation under a wide range of conditions. Owing to its simplicity, behaviour based on such a heuristic may in fact be ubiquitous. Finally, we argue that the evolution of exchange and trading of service and commodities among social partners needs greater scientific focus.

Coordinated vigilance provides evidence for direct reciprocity in coral reef fishes

Reciprocity is frequently assumed to require complex cognitive abilities. Therefore, it has been argued that reciprocity may be restricted to animals that can meet these demands. Here, we provide evidence for the potential presence of direct reciprocity in teleost fishes. We demonstrate that in pairs of coral reef rabbitfishes (f. Siganidae), one fish frequently assumes an upright vigilance position in the water column, while the partner forages in small crevices in the reef substratum. Both behaviours are strongly coordinated and partners regularly alternate their positions, resulting in a balanced distribution of foraging activity. Compared to solitary individuals, fishes in pairs exhibit longer vigilance bouts, suggesting that the help provided to the partner is costly. In turn, fishes in pairs take more consecutive bites and penetrate deeper into crevices than solitary individuals, suggesting that the safety provided by a vigilant partner may outweigh initial costs by increasing foraging efficiency. Thus, the described system appears to meet all of the requirements for direct reciprocity. We argue that the nature of rabbitfish pairs provides favourable conditions for the establishment of direct reciprocity, as continuous interaction with the same partner, simultaneous needs, interdependence, and communication relax the cognitive demands of reciprocal cooperation.

Information Transfer between Generations Linked to Biodiversity in Rock-Paper-Scissors Games

Ecological processes, such as reproduction, mobility, and interaction between species, play important roles in the maintenance of biodiversity. Classically, the cyclic dominance of species has been modelled using the nonhierarchical interactions among competing species, represented by the “Rock-Paper-Scissors” (RPS) game. Here we propose a cascaded channel model for analyzing the existence of biodiversity in the RPS game. The transition between successive generations is modelled as communication of information over a noisy communication channel. The rate of transfer of information over successive generations is studied using mutual information and it is found that “greedy” information transfer between successive generations may lead to conditions for extinction. This generalized framework can be used to study biodiversity in any number of interacting species, ecosystems with unequal rates for different species, and also competitive networks.

Cyclic dominance in evolutionary games: a review

Rock is wrapped by paper, paper is cut by scissors and scissors are crushed by rock. This simple game is popular among children and adults to decide on trivial disputes that have no obvious winner, but cyclic dominance is also at the heart of predator-prey interactions, the mating strategy of side-blotched lizards, the overgrowth of marine sessile organisms and competition in microbial populations. Cyclical interactions also emerge spontaneously in evolutionary games entailing volunteering, reward, punishment, and in fact are common when the competing strategies are three or more, regardless of the particularities of the game. Here, we review recent advances on the rock-paper-scissors (RPS) and related evolutionary games, focusing, in particular, on pattern formation, the impact of mobility and the spontaneous emergence of cyclic dominance. We also review mean-field and zero-dimensional RPS models and the application of the complex Ginzburg-Landau equation, and we highlight the importance and usefulness of statistical physics for the successful study of large-scale ecological systems. Directions for future research, related, for example, to dynamical effects of coevolutionary rules and invasion reversals owing to multi-point interactions, are also outlined.