Improving Collective Estimations Using Resistance to Social Influence

Promoting Erroneous Divergent Opinions Increases the Wisdom of Crowds

The aggregation of many lay judgments generates surprisingly accurate estimates. This phenomenon, called the "wisdom of crowds," has been demonstrated in domains such as medical decision-making and financial forecasting. Previous research identified two factors driving this effect: the accuracy of individual assessments and the diversity of opinions. Most available strategies to enhance the wisdom of crowds have focused on improving individual accuracy while neglecting the potential of increasing opinion diversity. Here, we study a complementary approach to reduce collective error by promoting erroneous divergent opinions. This strategy proposes to anchor half of the crowd to a small value and the other half to a large value before eliciting and averaging all estimates. Consistent with our mathematical modeling, four experiments (N = 1,362 adults) demonstrated that this method is effective for estimation and forecasting tasks. Beyond the practical implications, these findings offer new theoretical insights into the epistemic value of collective decision-making.

Crowdsourcing Knowledge Production of COVID-19 Information on Japanese Wikipedia in the Face of Uncertainty: Empirical Analysis

BACKGROUND

A worldwide overabundance of information comprising misinformation, rumors, and propaganda concerning COVID-19 has been observed in addition to the pandemic. By addressing this data confusion, Wikipedia has become an important source of information.

OBJECTIVE

This study aimed to investigate how the editors of Wikipedia have handled COVID-19-related information. Specifically, it focused on 2 questions: What were the knowledge preferences of the editors who participated in producing COVID-19-related information? and How did editors with different knowledge preferences collaborate?

METHODS

This study used a large-scale data set, including >2 million edits in the histories of 1857 editors who edited 133 articles related to COVID-19 on Japanese Wikipedia. Machine learning methods, including graph neural network methods, Bayesian inference, and Granger causality analysis, were used to establish the editors' topic proclivity and collaboration patterns.

RESULTS

Overall, 3 trends were observed. Two groups of editors were involved in the production of information on COVID-19. One group had a strong preference for sociopolitical topics (social-political group), and the other group strongly preferred scientific and medical topics (scientific-medical group). The social-political group played a central role (contributing 16,544,495/23,485,683, 70.04% of bits of content and 57,969/76,673, 75.61% of the references) in the information production part of the COVID-19 articles on Wikipedia, whereas the scientific-medical group played only a secondary role. The severity of the pandemic in Japan activated the editing behaviors of the social-political group, leading them to contribute more to COVID-19 information production on Wikipedia while simultaneously deactivating the editing behaviors of the scientific-medical group, resulting in their less contribution to COVID-19 information production on Wikipedia (Pearson correlation coefficient=0.231; P<.001).

CONCLUSIONS

The results of this study showed that lay experts (ie, Wikipedia editors) in the fields of science and medicine tended to remain silent when facing high scientific uncertainty related to the pandemic. Considering the high quality of the COVID-19-related articles on Japanese Wikipedia, this research also suggested that the sidelining of the science and medicine editors in discussions is not necessarily a problem. Instead, the social and political context of the issues with high scientific uncertainty is more important than the scientific discussions that support accuracy.

Social influence makes outlier opinions in online reviews offer more helpful information

Identifying helpful information from large-scale online reviews has become a core issue in studies on harnessing wisdom-of-crowds. We investigated whether online reviews expressing dissenting opinions (i.e., outlier reviews) can provide helpful information. Using statistical and simulation methods with a large-scale dataset, we found that, compared with other online reviews, outlier reviews were deemed more helpful because they provided more sufficient, neutral, and concise information. To interpret these results, we considered that in collective behaviours, a prevalent social psychological process-conformity (i.e., changing one's behaviour in response to pressure from others)-pressured reviewers expressing dissenting opinions. This motivated them to provide more convincing evidence (i.e., sufficient, neutral, and concise information). This study offers a simple yet effective approach for eliciting helpful information from many online reviews and deepens the understanding of the mechanism underlying collective online behaviour. Specifically, conformity was considered to cause biases in the collective behaviour of humans; however, this study revealed that conformity can elicit valuable outcomes in collective behaviour.

Casting votes of antecedents play a key role in successful sequential decision-making

Aggregation of opinions often results in high decision-making accuracy, owing to the collective intelligence effect. Studies on group decisions have examined the optimum weights for opinion aggregation to maximise accuracy. In addition to the optimum weights of opinions, the impact of the correlation among opinions on collective intelligence is a major issue in collective decision-making. We investigated how individuals should weigh the opinions of others and their own to maximise their accuracy in sequential decision-making. In our sequential decision-making model, each person makes a primary choice, observes his/her predecessors' opinions, and makes a final choice, which results in the person's answer correlating with those of others. We developed an algorithm to find casting voters whose primary choices are determinative of their answers and revealed that decision accuracy is maximised by considering only the abilities of the preceding casting voters. We also found that for individuals with heterogeneous abilities, the order of decision-making has a significant impact on the correlation between their answers and their accuracies. This could lead to a counter-intuitive phenomenon whereby, in sequential decision-making, respondents are, on average, more accurate when less reliable individuals answer earlier and more reliable individuals answer later.

The network science of collective intelligence

In the last few years, breakthroughs in computational and experimental techniques have produced several key discoveries in the science of networks and human collective intelligence. This review presents the latest scientific findings from two key fields of research: collective problem-solving and the wisdom of the crowd. I demonstrate the core theoretical tensions separating these research traditions and show how recent findings offer a new synthesis for understanding how network dynamics alter collective intelligence, both positively and negatively. I conclude by highlighting current theoretical problems at the forefront of research on networked collective intelligence, as well as vital public policy challenges that require new research efforts.

The motor Wisdom of the Crowd

Wisdom of the Crowd is the aggregation of many individual estimates to obtain a better collective one. Because of its enormous social potential, this effect has been thoroughly investigated, but predominantly on tasks that involve rational thinking (such as estimating a number). Here we tested this effect in the context of drawing geometrical shapes, which still enacts cognitive processes but mainly involves visuomotor control. We asked more than 700 school students to trace five patterns shown on a touchscreen and then aggregated their individual trajectories to improve the match with the original pattern. Our results show the characteristics of the strongest examples of Wisdom of the Crowd. First, the aggregate trajectory can be up to 5 times more accurate than the individual ones. Second, this great improvement requires aggregating trajectories from different individuals (rather than trials from the same individual). Third, the aggregate trajectory outperforms more than 99% of individual trajectories. Fourth, while older individuals outperform younger ones, a crowd of young individuals outperforms the average older one. These results demonstrate for the first time Wisdom of the Crowd in the realm of motor control, opening the door to further studies of human and also animal behavioural trajectories and their mechanistic underpinnings.

Predictive models for human-AI nexus in group decision making

Machine learning (ML) and artificial intelligence (AI) have had a profound impact on our lives. Domains like health and learning are naturally helped by human-AI interactions and decision making. In these areas, as ML algorithms prove their value in making important decisions, humans add their distinctive expertise and judgment on social and interpersonal issues that need to be considered in tandem with algorithmic inputs of information. Some questions naturally arise. What rules and regulations should be invoked on the employment of AI, and what protocols should be in place to evaluate available AI resources? What are the forms of effective communication and coordination with AI that best promote effective human-AI teamwork? In this review, we highlight factors that we believe are especially important in assembling and managing human-AI decision making in a group setting.

Crowd control: Reducing individual estimation bias by sharing biased social information

Cognitive biases are widespread in humans and animals alike, and can sometimes be reinforced by social interactions. One prime bias in judgment and decision-making is the human tendency to underestimate large quantities. Previous research on social influence in estimation tasks has generally focused on the impact of single estimates on individual and collective accuracy, showing that randomly sharing estimates does not reduce the underestimation bias. Here, we test a method of social information sharing that exploits the known relationship between the true value and the level of underestimation, and study if it can counteract the underestimation bias. We performed estimation experiments in which participants had to estimate a series of quantities twice, before and after receiving estimates from one or several group members. Our purpose was threefold: to study (i) whether restructuring the sharing of social information can reduce the underestimation bias, (ii) how the number of estimates received affects the sensitivity to social influence and estimation accuracy, and (iii) the mechanisms underlying the integration of multiple estimates. Our restructuring of social interactions successfully countered the underestimation bias. Moreover, we find that sharing more than one estimate also reduces the underestimation bias. Underlying our results are a human tendency to herd, to trust larger estimates than one's own more than smaller estimates, and to follow disparate social information less. Using a computational modeling approach, we demonstrate that these effects are indeed key to explain the experimental results. Overall, our results show that existing knowledge on biases can be used to dampen their negative effects and boost judgment accuracy, paving the way for combating other cognitive biases threatening collective systems.

Impact of sharing full versus averaged social information on social influence and estimation accuracy

The recent developments of social networks and recommender systems have dramatically increased the amount of social information shared in human communities, challenging the human ability to process it. As a result, sharing aggregated forms of social information is becoming increasingly popular. However, it is unknown whether sharing aggregated information improves people's judgments more than sharing the full available information. Here, we compare the performance of groups in estimation tasks when social information is fully shared versus when it is first averaged and then shared. We find that improvements in estimation accuracy are comparable in both cases. However, our results reveal important differences in subjects' behaviour: (i) subjects follow the social information more when receiving an average than when receiving all estimates, and this effect increases with the number of estimates underlying the average; (ii) subjects follow the social information more when it is higher than their personal estimate than when it is lower. This effect is stronger when receiving all estimates than when receiving an average. We introduce a model that sheds light on these effects, and confirms their importance for explaining improvements in estimation accuracy in all treatments.

Accuracy-Risk Trade-Off Due to Social Learning in Crowd-Sourced Financial Predictions

A critical question relevant to the increasing importance of crowd-sourced-based finance is how to optimize collective information processing and decision-making. Here, we investigate an often under-studied aspect of the performance of online traders: beyond focusing on just accuracy, what gives rise to the trade-off between risk and accuracy at the collective level? Answers to this question will lead to designing and deploying more effective crowd-sourced financial platforms and to minimizing issues stemming from risk such as implied volatility. To investigate this trade-off, we conducted a large online Wisdom of the Crowd study where 2037 participants predicted the prices of real financial assets (S&P 500, WTI Oil and Gold prices). Using the data collected, we modeled the belief update process of participants using models inspired by Bayesian models of cognition. We show that subsets of predictions chosen based on their belief update strategies lie on a Pareto frontier between accuracy and risk, mediated by social learning. We also observe that social learning led to superior accuracy during one of our rounds that occurred during the high market uncertainty of the Brexit vote.

The wisdom of stalemates: consensus and clustering as filtering mechanisms for improving collective accuracy

Groups of organisms, from bacteria to fish schools to human societies, depend on their ability to make accurate decisions in an uncertain world. Most models of collective decision-making assume that groups reach a consensus during a decision-making bout, often through simple majority rule. In many natural and sociological systems, however, groups may fail to reach consensus, resulting in stalemates. Here, we build on opinion dynamics and collective wisdom models to examine how stalemates may affect the wisdom of crowds. For simple environments, where individuals have access to independent sources of information, we find that stalemates improve collective accuracy by selectively filtering out incorrect decisions (an effect we call stalemate filtering). In complex environments, where individuals have access to both shared and independent information, this effect is even more pronounced, restoring the wisdom of crowds in regions of parameter space where large groups perform poorly when making decisions using majority rule. We identify network properties that tune the system between consensus and accuracy, providing mechanisms by which animals, or evolution, could dynamically adjust the collective decision-making process in response to the reward structure of the possible outcomes. Overall, these results highlight the adaptive potential of stalemate filtering for improving the decision-making abilities of group-living animals.

Multi-model and network inference based on ensemble estimates: avoiding the madness of crowds

Recent progress in theoretical systems biology, applied mathematics and computational statistics allows us to compare the performance of different candidate models at describing a particular biological system quantitatively. Model selection has been applied with great success to problems where a small number-typically less than 10-of models are compared, but recent studies have started to consider thousands and even millions of candidate models. Often, however, we are left with sets of models that are compatible with the data, and then we can use ensembles of models to make predictions. These ensembles can have very desirable characteristics, but as I show here are not guaranteed to improve on individual estimators or predictors. I will show in the cases of model selection and network inference when we can trust ensembles, and when we should be cautious. The analyses suggest that the careful construction of an ensemble-choosing good predictors-is of paramount importance, more than had perhaps been realized before: merely adding different methods does not suffice. The success of ensemble network inference methods is also shown to rest on their ability to suppress false-positive results. A Jupyter notebook which allows carrying out an assessment of ensemble estimators is provided.

The impact of incorrect social information on collective wisdom in human groups

A major problem resulting from the massive use of social media is the potential spread of incorrect information. Yet, very few studies have investigated the impact of incorrect information on individual and collective decisions. We performed experiments in which participants had to estimate a series of quantities, before and after receiving social information. Unbeknownst to them, we controlled the degree of inaccuracy of the social information through 'virtual influencers', who provided some incorrect information. We find that a large proportion of individuals only partially follow the social information, thus resisting incorrect information. Moreover, incorrect information can help improve group performance more than correct information, when going against a human underestimation bias. We then design a computational model whose predictions are in good agreement with the empirical data, and sheds light on the mechanisms underlying our results. Besides these main findings, we demonstrate that the dispersion of estimates varies a lot between quantities, and must thus be considered when normalizing and aggregating estimates of quantities that are very different in nature. Overall, our results suggest that incorrect information does not necessarily impair the collective wisdom of groups, and can even be used to dampen the negative effects of known cognitive biases.

Communication speeds up but impairs the consensus decision in a dyadic colour estimation task

Communication plays an important role in consensus decision-making which pervades our daily life. However, the exact role of communication in consensus formation is not clear. Here, to study the effects of communication on consensus formation, we designed a dyadic colour estimation task, where a pair of isolated participants repeatedly estimated the colours of discs until they reached a consensus or completed eight estimations, either with or without communication. We show that participants' estimates gradually approach each other, reaching towards a consensus, and these are enhanced with communication. We also show that dyadic consensus estimation is on average better than individual estimation. Surprisingly, consensus estimation without communication generally outperforms that with communication, indicating that communication impairs the improvement of consensus estimation. However, without communication, it takes longer to reach a consensus. Moreover, participants who partially cooperate with each other tend to result in better overall consensus. Taken together, we have identified the effect of communication on the dynamics of consensus formation, and the results may have implications on group decision-making in general.

Adaptive social networks promote the wisdom of crowds

Social networks continuously change as new ties are created and existing ones fade. It is widely acknowledged that our social embedding has a substantial impact on what information we receive and how we form beliefs and make decisions. However, most empirical studies on the role of social networks in collective intelligence have overlooked the dynamic nature of social networks and its role in fostering adaptive collective intelligence. Therefore, little is known about how groups of individuals dynamically modify their local connections and, accordingly, the topology of the network of interactions to respond to changing environmental conditions. In this paper, we address this question through a series of behavioral experiments and supporting simulations. Our results reveal that, in the presence of plasticity and feedback, social networks can adapt to biased and changing information environments and produce collective estimates that are more accurate than their best-performing member. To explain these results, we explore two mechanisms: 1) a global-adaptation mechanism where the structural connectivity of the network itself changes such that it amplifies the estimates of high-performing members within the group (i.e., the network "edges" encode the computation); and 2) a local-adaptation mechanism where accurate individuals are more resistant to social influence (i.e., adjustments to the attributes of the "node" in the network); therefore, their initial belief is disproportionately weighted in the collective estimate. Our findings substantiate the role of social-network plasticity and feedback as key adaptive mechanisms for refining individual and collective judgments.

Insights about collective decision-making at the genetic level

By living in a collective, individuals can share and aggregate information to base their decisions on the many rather than on the one, thereby increasing accuracy. But a collective can also be defined at the molecular level. In the following, we reason that genes, by working collectively, share fundamental features with social organisms, which ends, without invoking cognition, in wiser responses. For that, we compile into a single picture the terms redundancy, stochastic resonance, intrinsic and extrinsic noise, and cross-regulation.

Counteracting estimation bias and social influence to improve the wisdom of crowds

Aggregating multiple non-expert opinions into a collective estimate can improve accuracy across many contexts. However, two sources of error can diminish collective wisdom: individual estimation biases and information sharing between individuals. Here, we measure individual biases and social influence rules in multiple experiments involving hundreds of individuals performing a classic numerosity estimation task. We first investigate how existing aggregation methods, such as calculating the arithmetic mean or the median, are influenced by these sources of error. We show that the mean tends to overestimate, and the median underestimate, the true value for a wide range of numerosities. Quantifying estimation bias, and mapping individual bias to collective bias, allows us to develop and validate three new aggregation measures that effectively counter sources of collective estimation error. In addition, we present results from a further experiment that quantifies the social influence rules that individuals employ when incorporating personal estimates with social information. We show that the corrected mean is remarkably robust to social influence, retaining high accuracy in the presence or absence of social influence, across numerosities and across different methods for averaging social information. Using knowledge of estimation biases and social influence rules may therefore be an inexpensive and general strategy to improve the wisdom of crowds.

The zebrafish subcortical social brain as a model for studying social behavior disorders

Social behaviors are essential for the survival and reproduction of social species. Many, if not most, neuropsychiatric disorders in humans are either associated with underlying social deficits or are accompanied by social dysfunctions. Traditionally, rodent models have been used to model these behavioral impairments. However, rodent assays are often difficult to scale up and adapt to high-throughput formats, which severely limits their use for systems-level science. In recent years, an increasing number of studies have used zebrafish (Danio rerio) as a model system to study social behavior. These studies have demonstrated clear potential in overcoming some of the limitations of rodent models. In this Review, we explore the evolutionary conservation of a subcortical social brain between teleosts and mammals as the biological basis for using zebrafish to model human social behavior disorders, while summarizing relevant experimental tools and assays. We then discuss the recent advances gleaned from zebrafish social behavior assays, the applications of these assays to studying related disorders, and the opportunities and challenges that lie ahead.

The wisdom of partisan crowds

Theories in favor of deliberative democracy are based on the premise that social information processing can improve group beliefs. While research on the "wisdom of crowds" has found that information exchange can increase belief accuracy on noncontroversial factual matters, theories of political polarization imply that groups will become more extreme-and less accurate-when beliefs are motivated by partisan political bias. A primary concern is that partisan biases are associated not only with more extreme beliefs, but also with a diminished response to social information. While bipartisan networks containing both Democrats and Republicans are expected to promote accurate belief formation, politically homogeneous networks are expected to amplify partisan bias and reduce belief accuracy. To test whether the wisdom of crowds is robust to partisan bias, we conducted two web-based experiments in which individuals answered factual questions known to elicit partisan bias before and after observing the estimates of peers in a politically homogeneous social network. In contrast to polarization theories, we found that social information exchange in homogeneous networks not only increased accuracy but also reduced polarization. Our results help generalize collective intelligence research to political domains.

Adolescents show collective intelligence which can be driven by a geometric mean rule of thumb

How effective groups are in making decisions is a long-standing question in studying human and animal behaviour. Despite the limited social and cognitive abilities of younger people, skills which are often required for collective intelligence, studies of group performance have been limited to adults. Using a simple task of estimating the number of sweets in jars, we show in two experiments that adolescents at least as young as 11 years old improve their estimation accuracy after a period of group discussion, demonstrating collective intelligence. Although this effect was robust to the overall distribution of initial estimates, when the task generated positively skewed estimates, the geometric mean of initial estimates gave the best fit to the data compared to other tested aggregation rules. A geometric mean heuristic in consensus decision making is also likely to apply to adults, as it provides a robust and well-performing rule for aggregating different opinions. The geometric mean rule is likely to be based on an intuitive logarithmic-like number representation, and our study suggests that this mental number scaling may be beneficial in collective decisions.

Adversity magnifies the importance of social information in decision-making

Decision-making theories explain animal behaviour, including human behaviour, as a response to estimations about the environment. In the case of collective behaviour, they have given quantitative predictions of how animals follow the majority option. However, they have so far failed to explain that in some species and contexts social cohesion increases when conditions become more adverse (i.e. individuals choose the majority option with higher probability when the estimated quality of all available options decreases). We have found that this failure is due to modelling simplifications that aided analysis, like low levels of stochasticity or the assumption that only one choice is the correct one. We provide a more general but simple geometric framework to describe optimal or suboptimal decisions in collectives that gives insight into three different mechanisms behind this effect. The three mechanisms have in common that the private information acts as a gain factor to social information: a decrease in the privately estimated quality of all available options increases the impact of social information, even when social information itself remains unchanged. This increase in the importance of social information makes it more likely that agents will follow the majority option. We show that these results quantitatively explain collective behaviour in fish and experiments of social influence in humans.

Reciprocity of social influence

Humans seek advice, via social interaction, to improve their decisions. While social interaction is often reciprocal, the role of reciprocity in social influence is unknown. Here, we tested the hypothesis that our influence on others affects how much we are influenced by them. Participants first made a visual perceptual estimate and then shared their estimate with an alleged partner. Then, in alternating trials, the participant either revised their decisions or observed how the partner revised theirs. We systematically manipulated the partner’s susceptibility to influence from the participant. We show that participants reciprocated influence with their partner by gravitating toward the susceptible (but not insusceptible) partner’s opinion. In further experiments, we showed that reciprocity is both a dynamic process and is abolished when people believed that they interacted with a computer. Reciprocal social influence is a signaling medium for human-to-human communication that goes beyond aggregation of evidence for decision improvement.

Humans give and receive social influence—e.g. advice—in many situations, but it is not known whether social influence is a reciprocal process, like trade. Here, the authors show that people are more likely to follow a partner's advice if that partner has previously complied with their advice.

Emergence of opinion leaders in reference networks

Individuals often refer to opinions of others when they make decisions in the real world. Our question is how the people’s reference structure self-organizes when people try to provide correct answers by referring to more accurate agents. We constructed an adaptive network model, in which each node represents an agent and each directed link represents a reference. In every iteration round within our model, each agent makes a decision sequentially by following the majority of the reference partners’ opinions and rewires a reference link to a partner if the partner’s performance falls below a given threshold. The value of this threshold is common for all agents and represents the performance assessment severity of the population. We found that the reference network self-organizes into a heterogeneous one with a nearly exponential in-degree (the number of followers) distribution, where reference links concentrate around agents with high intrinsic ability. In this heterogeneous network, the decision-making accuracy of agents improved on average. However, the proportion of agents who provided correct answers showed strong temporal fluctuation compared to that observed in the case in which each agent refers to randomly selected agents. We also found a counterintuitive phenomenon in which reference links concentrate more around high-ability agents and the population became smarter on average when the rewiring threshold was set lower than when it was set higher.

How social information can improve estimation accuracy in human groups

In our digital and connected societies, the development of social networks, online shopping, and reputation systems raises the questions of how individuals use social information and how it affects their decisions. We report experiments performed in France and Japan, in which subjects could update their estimates after having received information from other subjects. We measure and model the impact of this social information at individual and collective scales. We observe and justify that, when individuals have little prior knowledge about a quantity, the distribution of the logarithm of their estimates is close to a Cauchy distribution. We find that social influence helps the group improve its properly defined collective accuracy. We quantify the improvement of the group estimation when additional controlled and reliable information is provided, unbeknownst to the subjects. We show that subjects' sensitivity to social influence permits us to define five robust behavioral traits and increases with the difference between personal and group estimates. We then use our data to build and calibrate a model of collective estimation to analyze the impact on the group performance of the quantity and quality of information received by individuals. The model quantitatively reproduces the distributions of estimates and the improvement of collective performance and accuracy observed in our experiments. Finally, our model predicts that providing a moderate amount of incorrect information to individuals can counterbalance the human cognitive bias to systematically underestimate quantities and thereby improve collective performance.

Can Simple Transmission Chains Foster Collective Intelligence in Binary-Choice Tasks?

In many social systems, groups of individuals can find remarkably efficient solutions to complex cognitive problems, sometimes even outperforming a single expert. The success of the group, however, crucially depends on how the judgments of the group members are aggregated to produce the collective answer. A large variety of such aggregation methods have been described in the literature, such as averaging the independent judgments, relying on the majority or setting up a group discussion. In the present work, we introduce a novel approach for aggregating judgments—the transmission chain—which has not yet been consistently evaluated in the context of collective intelligence. In a transmission chain, all group members have access to a unique collective solution and can improve it sequentially. Over repeated improvements, the collective solution that emerges reflects the judgments of every group members. We address the question of whether such a transmission chain can foster collective intelligence for binary-choice problems. In a series of numerical simulations, we explore the impact of various factors on the performance of the transmission chain, such as the group size, the model parameters, and the structure of the population. The performance of this method is compared to those of the majority rule and the confidence-weighted majority. Finally, we rely on two existing datasets of individuals performing a series of binary decisions to evaluate the expected performances of the three methods empirically. We find that the parameter space where the transmission chain has the best performance rarely appears in real datasets. We conclude that the transmission chain is best suited for other types of problems, such as those that have cumulative properties.