The evolution of conditional moral assessment in indirect reciprocity

A mechanistic model of gossip, reputations, and cooperation

Social reputations facilitate cooperation: those who help others gain a good reputation, making them more likely to receive help themselves. But when people hold private views of one another, this cycle of indirect reciprocity breaks down, as disagreements lead to the perception of unjustified behavior that ultimately undermines cooperation. Theoretical studies often assume population-wide agreement about reputations, invoking rapid gossip as an endogenous mechanism for reaching consensus. However, the theory of indirect reciprocity lacks a mechanistic description of how gossip actually generates consensus. Here, we develop a mechanistic model of gossip-based indirect reciprocity that incorporates two alternative forms of gossip: exchanging information with randomly selected peers or consulting a single gossip source. We show that these two forms of gossip are mathematically equivalent under an appropriate transformation of parameters. We derive an analytical expression for the minimum amount of gossip required to reach sufficient consensus and stabilize cooperation. We analyze how the amount of gossip necessary for cooperation depends on the benefits and costs of cooperation, the assessment rule (social norm), and errors in reputation assessment, strategy execution, and gossip transmission. Finally, we show that biased gossip can either facilitate or hinder cooperation, depending on the direction and magnitude of the bias. Our results contribute to the growing literature on cooperation facilitated by communication, and they highlight the need to study strategic interactions coupled with the spread of social information.

Indirect reciprocity with Bayesian reasoning and biases

Reputations can foster cooperation by indirect reciprocity: if I am good to you then others will be good to me. But this mechanism for cooperation in one-shot interactions only works when people agree on who is good and who is bad. Errors in actions or assessments can produce disagreements about reputations, which can unravel the positive feedback loop between social standing and pro-social behaviour. Cooperators can end up punished and defectors rewarded. Public reputation systems and empathy are two possible mechanisms to promote agreement about reputations. Here we suggest an alternative: Bayesian reasoning by observers. By taking into account the probabilities of errors in action and observation and their prior beliefs about the prevalence of good people in the population, observers can use Bayesian reasoning to determine whether or not someone is good. To study this scenario, we develop an evolutionary game theoretical model in which players use Bayesian reasoning to assess reputations, either publicly or privately. We explore this model analytically and numerically for five social norms (Scoring, Shunning, Simple Standing, Staying, and Stern Judging). We systematically compare results to the case when agents do not use reasoning in determining reputations. We find that Bayesian reasoning reduces cooperation relative to non-reasoning, except in the case of the Scoring norm. Under Scoring, Bayesian reasoning can promote coexistence of three strategic types. Additionally, we study the effects of optimistic or pessimistic biases in individual beliefs about the degree of cooperation in the population. We find that optimism generally undermines cooperation whereas pessimism can, in some cases, promote cooperation.

Indirect reciprocity with abductive reasoning

Indirect reciprocity is a reputational mechanism through which cooperative behavior can be promoted amongst a group of individuals. However, in order for this mechanism to effectively do so, cheating must be appropriately punished and cooperating appropriately rewarded. Errors in assessments and actions can hinder this process. In such a setting, individuals might try to reason about evidence to assign reputations given the possibility of errors. Here, we consider a well-established theory of reasoning used to combine evidence, abductive reasoning, as a possible means by which such errors can be circumvented. Specifically, we use Dempster-Shafer theory to model individuals who account for possible errors by combining information about their beliefs about the status of the population and the errors rates and then choose the simplest scenario that could explain their observations in the context of these beliefs. We investigate the effectiveness of abductive reasoning at promoting cooperation for five social norms: Scoring, Shunning, Simple Standing, Staying, and Stern Judging. We find that, generally, abductive reasoning can outperform non-reasoning models at ameliorating the effects of the aforementioned challenges and promote higher levels of cooperation under low-error conditions. However, for high-error conditions, we find that abductive reasoning can undermine cooperation. Furthermore, we also find that a degree of bias towards believing previously held reputations can help sustain cooperation.

When do stereotypes undermine indirect reciprocity?

Social reputations provide a powerful mechanism to stimulate human cooperation, but observing individual reputations can be cognitively costly. To ease this burden, people may rely on proxies such as stereotypes, or generalized reputations assigned to groups. Such stereotypes are less accurate than individual reputations, and so they could disrupt the positive feedback between altruistic behavior and social standing, undermining cooperation. How do stereotypes impact cooperation by indirect reciprocity? We develop a theoretical model of group-structured populations in which individuals are assigned either individual reputations based on their own actions or stereotyped reputations based on their groups' behavior. We find that using stereotypes can produce either more or less cooperation than using individual reputations, depending on how widely reputations are shared. Deleterious outcomes can arise when individuals adapt their propensity to stereotype. Stereotyping behavior can spread and can be difficult to displace, even when it compromises collective cooperation and even though it makes a population vulnerable to invasion by defectors. We discuss the implications of our results for the prevalence of stereotyping and for reputation-based cooperation in structured populations.

Individuals reciprocate negative actions revealing negative upstream reciprocity

Indirect reciprocity is widely recognized as a mechanism for explaining cooperation and can be divided into two sub-concepts: downstream and upstream reciprocity. Downstream reciprocity is supported by reputation; if someone sees you helping someone else, the person who sees this will think higher of you, and you will be more likely to be helped. Upstream reciprocity is helping someone because you are being helped by somebody else, which often happens in everyday life and experimental games. This paper focuses on the behavior of "take" and examines negative upstream reciprocity using an upstream reciprocity framework. The term "take" is defined as "to steal rather than give resources to others." "If something is taken from you, do you take from others?" is an important extension for indirect reciprocity research; subsequently, this paper discusses experiments conducted on whether negative upstream reciprocity is chained and what causes it. The results demonstrated differences between positive and negative upstream reciprocity. In analyzing the data of nearly 600 participants to determine the extent to which negative upstream reciprocity is observed and the causes of negative upstream reciprocity, the study found that If individual A takes resources from individual B, then B is more likely to take resources from a third-party, individual C. Notably, some causes of positive upstream reciprocity were found to have no effect or the opposite effect on negative upstream reciprocity. The results also demonstrate that the first person to take can cause a chain reaction. This paper demonstrates the importance of the first person not taking from someone else and suggests the need to consider various behavioral options for future research on cooperation.

Evolution of norms for judging social behavior

Reputations provide a powerful mechanism to sustain cooperation, as individuals cooperate with those of good social standing. But how should someone's reputation be updated as we observe their social behavior, and when will a population converge on a shared norm for judging behavior? Here, we develop a mathematical model of cooperation conditioned on reputations, for a population that is stratified into groups. Each group may subscribe to a different social norm for assessing reputations and so norms compete as individuals choose to move from one group to another. We show that a group initially comprising a minority of the population may nonetheless overtake the entire population-especially if it adopts the Stern Judging norm, which assigns a bad reputation to individuals who cooperate with those of bad standing. When individuals do not change group membership, stratifying reputation information into groups tends to destabilize cooperation, unless individuals are strongly insular and favor in-group social interactions. We discuss the implications of our results for the structure of information flow in a population and for the evolution of social norms of judgment.

Evolutionary stability of cooperation in indirect reciprocity under noisy and private assessment

Indirect reciprocity is a mechanism that explains large-scale cooperation in humans. In indirect reciprocity, individuals use reputations to choose whether or not to cooperate with a partner and update others' reputations. A major question is how the rules to choose their actions and the rules to update reputations evolve. In the public reputation case where all individuals share the evaluation of others, social norms called Simple Standing (SS) and Stern Judging (SJ) have been known to maintain cooperation. However, in the case of private assessment where individuals independently evaluate others, the mechanism of maintenance of cooperation is still largely unknown. This study theoretically shows for the first time that cooperation by indirect reciprocity can be evolutionarily stable under private assessment. Specifically, we find that SS can be stable, but SJ can never be. This is intuitive because SS can correct interpersonal discrepancies in reputations through its simplicity. On the other hand, SJ is too complicated to avoid an accumulation of errors, which leads to the collapse of cooperation. We conclude that moderate simplicity is a key to stable cooperation under the private assessment. Our result provides a theoretical basis for the evolution of human cooperation.

Quantitative assessment can stabilize indirect reciprocity under imperfect information

The field of indirect reciprocity investigates how social norms can foster cooperation when individuals continuously monitor and assess each other's social interactions. By adhering to certain social norms, cooperating individuals can improve their reputation and, in turn, receive benefits from others. Eight social norms, known as the "leading eight," have been shown to effectively promote the evolution of cooperation as long as information is public and reliable. These norms categorize group members as either 'good' or 'bad'. In this study, we examine a scenario where individuals instead assign nuanced reputation scores to each other, and only cooperate with those whose reputation exceeds a certain threshold. We find both analytically and through simulations that such quantitative assessments are error-correcting, thus facilitating cooperation in situations where information is private and unreliable. Moreover, our results identify four specific norms that are robust to such conditions, and may be relevant for helping to sustain cooperation in natural populations.

The complexity of human cooperation under indirect reciprocity

Indirect reciprocity (IR) is a key mechanism to understand cooperation among unrelated individuals. It involves reputations and complex information processing, arising from social interactions. By helping someone, individuals may improve their reputation, which may be shared in a population and change the predisposition of others to reciprocate in the future. The reputation of individuals depends, in turn, on social norms that define a good or bad action, offering a computational and mathematical appealing way of studying the evolution of moral systems. Over the years, theoretical and empirical research has unveiled many features of cooperation under IR, exploring norms with varying degrees of complexity and information requirements. Recent results suggest that costly reputation spread, interaction observability and empathy are determinants of cooperation under IR. Importantly, such characteristics probably impact the level of complexity and information requirements for IR to sustain cooperation. In this review, we present and discuss those recent results. We provide a synthesis of theoretical models and discuss previous conclusions through the lens of evolutionary game theory and cognitive complexity. We highlight open questions and suggest future research in this domain. This article is part of the theme issue 'The language of cooperation: reputation and honest signalling'.

The evolution of indirect reciprocity under action and assessment generosity

Indirect reciprocity is a mechanism for the evolution of cooperation based on social norms. This mechanism requires that individuals in a population observe and judge each other's behaviors. Individuals with a good reputation are more likely to receive help from others. Previous work suggests that indirect reciprocity is only effective when all relevant information is reliable and publicly available. Otherwise, individuals may disagree on how to assess others, even if they all apply the same social norm. Such disagreements can lead to a breakdown of cooperation. Here we explore whether the predominantly studied 'leading eight' social norms of indirect reciprocity can be made more robust by equipping them with an element of generosity. To this end, we distinguish between two kinds of generosity. According to assessment generosity, individuals occasionally assign a good reputation to group members who would usually be regarded as bad. According to action generosity, individuals occasionally cooperate with group members with whom they would usually defect. Using individual-based simulations, we show that the two kinds of generosity have a very different effect on the resulting reputation dynamics. Assessment generosity tends to add to the overall noise and allows defectors to invade. In contrast, a limited amount of action generosity can be beneficial in a few cases. However, even when action generosity is beneficial, the respective simulations do not result in full cooperation. Our results suggest that while generosity can favor cooperation when individuals use the most simple strategies of reciprocity, it is disadvantageous when individuals use more complex social norms.

Adherence to public institutions that foster cooperation

Humans typically consider altruism a moral good and condition their social behavior on the moral reputations of others. Indirect reciprocity explains how social norms and reputations support cooperation: individuals cooperate with others who are considered good. Indirect reciprocity works when an institution monitors and publicly broadcasts moral reputations. Here we develop a theory of adherence to public monitoring in societies where individuals are, at first, independently responsible for evaluating the reputations of their peers. Using a mathematical model, we show that adherence to an institution of moral assessment can evolve and promote cooperation under four different social norms, including norms that previous studies found to perform poorly. We determine how an institution's size and its degree of tolerance towards anti-social behavior affect the rate of cooperation. Public monitoring serves to eliminate disagreements about reputations, which increases cooperation and payoffs, so that adherence evolves by social contagion and remains robust against displacement.

Emotion expressions shape human social norms and reputations

The emergence of pro-social behaviors remains a key open challenge across disciplines. In this context, there is growing evidence that expressing emotions may foster human cooperation. However, it remains unclear how emotions shape individual choices and interact with other cooperation mechanisms. Here, we provide a comprehensive experimental analysis of the interplay of emotion expressions with two important mechanisms: direct and indirect reciprocity. We show that cooperation in an iterated prisoner's dilemma emerges from the combination of the opponent's initial reputation, past behaviors, and emotion expressions. Moreover, all factors influenced the social norm adopted when assessing the action of others - i.e., how their counterparts' reputations are updated - thus, reflecting longer-term consequences. We expose a new class of emotion-based social norms, where emotions are used to forgive those that defect but also punish those that cooperate. These findings emphasize the importance of emotion expressions in fostering, directly and indirectly, cooperation in society.

Two ways to overcome the three social dilemmas of indirect reciprocity

Indirect reciprocity is one of the main principles of evolving cooperation in a social dilemma situation. In reciprocity, a positive score is given to cooperative behaviour while a negative score is given to non-cooperative behaviour, and the dilemma is resolved by selectively cooperating only with those with positive scores. However, many studies have shown that non-cooperation with those who have not cooperated also downgrades one's reputation; they have called this situation the scoring dilemma. To address this dilemma, the notion of justified punishments has been considered. The notion of justified punishment allows good individuals who defect against bad co-players to keep their standing. Despite numerous studies on justified punishment, it is unknown whether this solution leads to a new type of dilemma because reputations may be downgraded when the intent of punishment is not correctly communicated. The dilemma of punishment has so far been rarely analysed, and thus, the complete solution of the mechanism for evolving cooperation using the principle of indirect reciprocity has not been found yet. Here, we identify sufficient conditions to overcome each of the three dilemmas including the dilemma of punishment to maintain stable cooperation by using the framework of evolutionary game theory. This condition includes the principle of detecting free riders, which resolves the social dilemma, the principle of justification, which resolves the scoring dilemma, and the principle of generosity, which resolves the dilemma of punishment. A norm that satisfies these principles can stably maintain social cooperation. Our insights may offer a general assessment principle that applies to a wide range of subjects, from individual actions to national decisions.

A Review of Theoretical Studies on Indirect Reciprocity

Despite the accumulation of research on indirect reciprocity over the past 30 years and the publication of over 100,000 related papers, there are still many issues to be addressed. Here, we look back on the research that has been done on indirect reciprocity and identify the issues that have been resolved and the ones that remain to be resolved. This manuscript introduces indirect reciprocity in the context of the evolution of cooperation, basic models of social dilemma situations, the path taken in the elaboration of mathematical analysis using evolutionary game theory, the discovery of image scoring norms, and the breakthroughs brought about by the analysis of the evolutionary instability of the norms. Moreover, it presents key results obtained by refining the assessment function, resolving the punishment dilemma, and presenting a complete solution to the social dilemma problem. Finally, it discusses the application of indirect reciprocity in various disciplines.

Justified defection is neither justified nor unjustified in indirect reciprocity

Indirect reciprocity is one of the major mechanisms in the evolution of human cooperation. In indirect reciprocity, social norms with which individuals distinguish good people from bad people play essential roles. Despite extensive studies on the evolution of cooperation in indirect reciprocity, little is known about which social norms people actually adopt. Here we reveal what kind of norms are adopted by people in indirect reciprocal situations in daily life by using scenario-based experiments. The results showed that people evaluated "justified defection" as neither good nor bad and withheld their evaluation. Theoretically, social norms that evaluate justified defection as good are required for cooperation to be stable. However, the norm that people actually adopted deviates from the theoretical predictions. Our results indicate the necessity to reconsider the justification of "justified defection" in the evolution of cooperation.

Evolution of empathetic moral evaluation

Social norms can promote cooperation by assigning reputations to individuals based on their past actions. A good reputation indicates that an individual is likely to reciprocate. A large body of research has established norms of moral assessment that promote cooperation, assuming reputations are objective. But without a centralized institution to provide objective evaluation, opinions about an individual’s reputation may differ across a population. In this setting we study the role of empathy–the capacity to form moral evaluations from another person’s perspective. We show that empathy tends to foster cooperation by reducing the rate of unjustified defection. The norms of moral evaluation previously considered most socially beneficial depend on high levels of empathy, whereas different norms maximize social welfare in populations incapable of empathy. Finally, we show that empathy itself can evolve through social contagion. We conclude that a capacity for empathy is a key component for sustaining cooperation in societies.

When meerkats have pups, they employ an individual to stand guard and warn the others of potential dangers and predators, putting their own life at risk. What seems like a selfless act is actually a common behavior found throughout the animal kingdom. But rather than acting out of concern for another ones wellbeing, it is considered to be an altruistic behavior towards kin, where an individual sacrifices its own reproductive success for the sake of the reproductive fitness of its entire clan.

In human societies, however, people often act altruistically towards unrelated individuals and have developed sophisticated systems of moral evaluation to decide who is worthy of cooperation and likely to reciprocate a favor. In other words, individuals will only help those who have a good reputation for being altruistic themselves. However, for this system to work, reputations need to be public knowledge, and societies need to agree on everyones reputations. But what happens when opinions about an individual's reputation are private and vary across a population?

Now, Radzvilavicius et al. wanted to find out whether altruism can emerge when people have different opinions about each others moral reputations. To do so, they used a so-called evolutionary game theory a mathematical description of how strategies change in a population over time. In their model, each individual could decide if they wanted to pay a personal cost to create a benefit for another individual. Each participant decided whether to act altruistically based on the reputation of the recipient; observers could update the individuals reputation based on their behavior.

The mathematical model revealed that when people are more empathetic and able to put themselves in someone elses shoes, altruism tends to spread over time. When people take into account different opinions and form moral judgements from another person's perspective, the population can sustain a higher level of cooperation. Moreover, the capacity for taking another person's perspective can itself evolve and remain stable in a population meaning that those individuals who evaluate each other empathetically tend to do better, and empathy spreads through social influence.

These findings can help us understand how empathy might have evolved in societies that value reputation as a means of reciprocity. A next step could be to test the theory developed by Radzvilavicius et al. in manipulative experiments, or to compare the theory to field data on reputations and behavior in online interactions.

Indirect reciprocity with private, noisy, and incomplete information

Indirect reciprocity is a mechanism for cooperation based on shared moral systems and individual reputations. It assumes that members of a community routinely observe and assess each other and that they use this information to decide who is good or bad, and who deserves cooperation. When information is transmitted publicly, such that all community members agree on each other's reputation, previous research has highlighted eight crucial moral systems. These "leading-eight" strategies can maintain cooperation and resist invasion by defectors. However, in real populations individuals often hold their own private views of others. Once two individuals disagree about their opinion of some third party, they may also see its subsequent actions in a different light. Their opinions may further diverge over time. Herein, we explore indirect reciprocity when information transmission is private and noisy. We find that in the presence of perception errors, most leading-eight strategies cease to be stable. Even if a leading-eight strategy evolves, cooperation rates may drop considerably when errors are common. Our research highlights the role of reliable information and synchronized reputations to maintain stable moral systems.

Experimental evidence of selective inattention in reputation-based cooperation

Reputation-based cooperation is often observed in modern society. People gain several types of information by assessing others. Among these, the most important information is the actions of people and those of their recipients. However, almost all studies assume that people consider all of the information they receive. This assumption is extreme, and people engaging in reputation-based cooperation may not pay attention to some information, i.e., they may display selective inattention. We demonstrate that subjects’ decision-making in relation to cooperative action depends on the content of the information they receive about their recipients. Our results show that subjects either consider or ignore information depending on the content of that information. When their recipients had cooperated previously, subjects cooperated without considering the information they received. When the recipients had played before with those who had bad reputations, subjects did not use that information, regardless of whether it was disclosed proactively. In other cases, subjects considered information on both the previous actions of recipients and those of the recipients’ own recipients. We found that subjects did not always use the information to make decisions, although they willingly received information about their recipients. This supports the proposition that selective inattention occurs in reputation-based cooperation.

A solution for private assessment in indirect reciprocity using solitary observation

Although indirect reciprocity is a fundamental mechanism in the evolution of human cooperation, most studies assume public assessment in which individuals are not permitted to obtain private assessments of others. Existing studies on private assessment have used individual-based simulations because of the analytical difficulty involved. Here, we develop an analytical method using solitary observation to solve private assessment in indirect reciprocity problem without any approximation. In this study, we formulate a model of solitary observation and calculate the replicator dynamics systems of five leading norms of indirect reciprocity. Indirect reciprocity in private assessment provides a different result to that in public assessment. According to the existence proofs of cooperative evolutionarily stable (CES) points in the system, strict norms (stern judging and shunning) have no CES point in private assessment, while they do in public assessment. Image scoring does not change the system regardless of the assessment types because it does not use second-order information. In tolerant norms (simple standing and staying), the CES points move to co-existence of norms and unconditional cooperators. Despite the fact that there is no central coercive assessment system in private assessment, the average cooperation rate at the CES points in private assessment is greater than that in public assessment. This is because private assessment gives unconditional cooperators a role. Our results also show the superiority of the staying norm. Compared with simple standing, staying has three advantages in private assessment: a higher cooperation rate, easiness of invasion into defectors, and robustness to maintain cooperative evolutionarily stable situations. Our results are applicable to general social dilemmas in relation to private information. Under some dilemmas, norms or assessment rules should be carefully chosen to enable cooperation to evolve.

Tolerant indirect reciprocity can boost social welfare through solidarity with unconditional cooperators in private monitoring

Indirect reciprocity is an important mechanism for resolving social dilemmas. Previous studies explore several types of assessment rules that are evolutionarily stable for keeping cooperation regimes. However, little is known about the effects of private information on social systems. Most indirect reciprocity studies assume public monitoring in which individuals share a single assessment for each individual. Here, we consider a private monitoring system that loosens such an unnatural assumption. We explore the stable norms in the private system using an individual-based simulation. We have three main findings. First, narrow and unstable cooperation: cooperation in private monitoring becomes unstable and the restricted norms cannot maintain cooperative regimes while they can in public monitoring. Second, stable coexistence of discriminators and unconditional cooperators: under private monitoring, unconditional cooperation can play a role in keeping a high level of cooperation in tolerant norm situations. Finally, Pareto improvement: private monitoring can achieve a higher cooperation rate than does public monitoring.

A norm knockout method on indirect reciprocity to reveal indispensable norms

Although various norms for reciprocity-based cooperation have been suggested that are evolutionarily stable against invasion from free riders, the process of alternation of norms and the role of diversified norms remain unclear in the evolution of cooperation. We clarify the co-evolutionary dynamics of norms and cooperation in indirect reciprocity and also identify the indispensable norms for the evolution of cooperation. Inspired by the gene knockout method, a genetic engineering technique, we developed the norm knockout method and clarified the norms necessary for the establishment of cooperation. The results of numerical investigations revealed that the majority of norms gradually transitioned to tolerant norms after defectors are eliminated by strict norms. Furthermore, no cooperation emerges when specific norms that are intolerant to defectors are knocked out.

Evolution of cooperation under indirect reciprocity and arbitrary exploration rates

Cooperation has been recognized as an evolutionary puzzle since Darwin, and remains identified as one of the biggest challenges of the XXIst century. Indirect Reciprocity (IR), a key mechanism that humans employ to cooperate with each other, establishes that individual behaviour depends on reputations, which in turn evolve depending on social norms that classify behaviours as good or bad. While it is well known that different social norms give rise to distinct cooperation levels, it remains unclear how the performance of each norm is influenced by the random exploration of new behaviours, often a key component of social dynamics where a plethora of stimuli may compel individuals to deviate from pre-defined behaviours. Here we study, for the first time, the impact of varying degrees of exploration rates - the likelihood of spontaneously adopting another strategy, akin to a mutation probability in evolutionary dynamics - in the emergence of cooperation under IR. We show that high exploration rates may either improve or harm cooperation, depending on the underlying social norm at work. Regarding some of the most popular social norms studied to date, we find that cooperation under Simple-standing and Image-score is enhanced by high exploration rates, whereas the opposite occurs for Stern-judging and Shunning.