Collective cognition in humans: groups outperform their best members in a sentence reconstruction task

Collective Intelligence Increases Diagnostic Accuracy in a General Practice Setting

BACKGROUND

General practitioners (GPs) work in an ill-defined environment where diagnostic errors are prevalent. Previous research indicates that aggregating independent diagnoses can improve diagnostic accuracy in a range of settings. We examined whether aggregating independent diagnoses can also improve diagnostic accuracy for GP decision making. In addition, we investigated the potential benefit of such an approach in combination with a decision support system (DSS).

METHODS

We simulated virtual groups using data sets from 2 previously published studies. In study 1, 260 GPs independently diagnosed 9 patient cases in a vignette-based study. In study 2, 30 GPs independently diagnosed 12 patient actors in a patient-facing study. In both data sets, GPs provided diagnoses in a control condition and/or DSS condition(s). Each GP's diagnosis, confidence rating, and years of experience were entered into a computer simulation. Virtual groups of varying sizes (range: 3-9) were created, and different collective intelligence rules (plurality, confidence, and seniority) were applied to determine each group's final diagnosis. Diagnostic accuracy was used as the performance measure.

RESULTS

Aggregating independent diagnoses by weighing them equally (i.e., the plurality rule) substantially outperformed average individual accuracy, and this effect increased with increasing group size. Selecting diagnoses based on confidence only led to marginal improvements, while selecting based on seniority reduced accuracy. Combining the plurality rule with a DSS further boosted performance.

DISCUSSION

Combining independent diagnoses may substantially improve a GP's diagnostic accuracy and subsequent patient outcomes. This approach did, however, not improve accuracy in all patient cases. Therefore, future work should focus on uncovering the conditions under which collective intelligence is most beneficial in general practice.

HIGHLIGHTS

We examined whether aggregating independent diagnoses of GPs can improve diagnostic accuracy.Using data sets of 2 previously published studies, we composed virtual groups of GPs and combined their independent diagnoses using 3 collective intelligence rules (plurality, confidence, and seniority).Aggregating independent diagnoses by weighing them equally substantially outperformed average individual GP accuracy, and this effect increased with increasing group size.Combining independent diagnoses may substantially improve GP's diagnostic accuracy and subsequent patient outcomes.

Crowdsourcing a diagnosis? Exploring the accuracy of the size and type of group diagnosis: an experimental study

BACKGROUND

The consultation process, where a clinician seeks an opinion from another clinician, is foundational in medicine. However, the effectiveness of group diagnosis has not been studied.

OBJECTIVE

To compare individual diagnosis to group diagnosis on two dimensions: group size (n=3 or 6) and group process (interactive or artificial groups).

METHODOLOGY

Thirty-six internal or emergency medicine residents participated in the study. Initially, each resident worked through four written cases on their own, providing a primary diagnosis and a differential diagnosis. Next, participants formed into groups of three. Using a videoconferencing platform, they worked through four additional cases, collectively providing a single primary diagnosis and differential diagnosis. The process was repeated using a group of six with four new cases. Cases were all counterbalanced. Retrospectively, nominal (ie, artificial) groups were formed by aggregating individual participant data into subgroups of three and six and analytically computing scores. Presence of the correct diagnosis as primary diagnosis or included in the differential diagnosis, as well as the number of diagnoses mentioned, was calculated for all conditions. Means were compared using analysis of variance.

RESULTS

For both authentic and nominal groups, the diagnostic accuracy of group diagnosis was superior to individual for both the primary diagnosis and differential diagnosis. However, there was no improvement in diagnostic accuracy when comparing a group of three to a group of six. Interactive and nominal groups were equivalent; however, this may be an artefact of the method used to combine data.

CONCLUSIONS

Group diagnosis improves diagnostic accuracy. However, a larger group is not necessarily superior to a smaller group. In this study, interactive group discussion does not result in improved diagnostic accuracy.

A cultural evolutionary theory that explains both gradual and punctuated change

Cumulative cultural evolution (CCE) occurs among humans who may be presented with many similar options from which to choose, as well as many social influences and diverse environments. It is unknown what general principles underlie the wide range of CCE dynamics and whether they can all be explained by the same unified paradigm. Here, we present a scalable evolutionary model of discrete choice with social learning, based on a few behavioural science assumptions. This paradigm connects the degree of transparency in social learning to the human tendency to imitate others. Computer simulations and quantitative analysis show the interaction of three primary factors—information transparency, popularity bias and population size—drives the pace of CCE. The model predicts a stable rate of evolutionary change for modest degrees of popularity bias. As popularity bias grows, the transition from gradual to punctuated change occurs, with maladaptive subpopulations arising on their own. When the popularity bias gets too severe, CCE stops. This provides a consistent framework for explaining the rich and complex adaptive dynamics taking place in the real world, such as modern digital media.

The motor Wisdom of the Crowd

Wisdom of the Crowd is the aggregation of many individual estimates to obtain a better collective one. Because of its enormous social potential, this effect has been thoroughly investigated, but predominantly on tasks that involve rational thinking (such as estimating a number). Here we tested this effect in the context of drawing geometrical shapes, which still enacts cognitive processes but mainly involves visuomotor control. We asked more than 700 school students to trace five patterns shown on a touchscreen and then aggregated their individual trajectories to improve the match with the original pattern. Our results show the characteristics of the strongest examples of Wisdom of the Crowd. First, the aggregate trajectory can be up to 5 times more accurate than the individual ones. Second, this great improvement requires aggregating trajectories from different individuals (rather than trials from the same individual). Third, the aggregate trajectory outperforms more than 99% of individual trajectories. Fourth, while older individuals outperform younger ones, a crowd of young individuals outperforms the average older one. These results demonstrate for the first time Wisdom of the Crowd in the realm of motor control, opening the door to further studies of human and also animal behavioural trajectories and their mechanistic underpinnings.

Problem-solving in groups of common marmosets (Callithrix jacchus): more than the sum of its parts

Human hypercooperativity and the emergence of division of labor enables us to solve problems not only effectively within a group but also collectively. Collective problem-solving occurs when groups perform better than the additive performance of separate individuals. Currently, it is unknown whether this is unique to humans. To investigate the evolutionary origin of collective problem-solving and potential precursors, we propose a continuum of group effects on problem-solving, from simple to complex ones, eventually culminating in collective problem-solving. We tested captive common marmosets with a series of problem-solving tasks, either alone or in a group. To test whether the performance of a group was more than the sum of its parts, we compared real groups to virtual groups (pooled scores of animals tested alone). Marmosets in real groups were both more likely to solve problems than marmosets within the virtual groups and to do so faster. Although individuals within real groups approached the problem faster, a reduction in neophobia was not sufficient to explain the greater success. Success within real groups arose because animals showed higher perseverance, especially after a fellow group member had found the solution in complex tasks. These results are consistent with the idea that group problem-solving evolved alongside a continuum, with performance improving beyond baseline as societies move from social tolerance to opportunities for diffusion of information to active exchange of information. We suggest that increasing interdependence and the adoption of cooperative breeding pushed our ancestors up this scale.

Pooling decisions decreases variation in response bias and accuracy

Decision makers in contexts as diverse as medical, judicial, and political decision making are known to differ substantially in response bias and accuracy, and these differences are a major factor undermining the reliability and fairness of the respective decision systems. Using theoretical modeling and empirical testing across five domains, we show that collective systems based on pooling decisions robustly overcome this important but as of now unresolved problem of experts' heterogeneity. In breast and skin cancer diagnostics and fingerprint analysis, we find that pooling the decisions of five experts reduces the variation in sensitivity among decision makers by 52%, 54%, and 41%, respectively. Similar reductions are achieved for specificity and response bias, and in other domains. Thus, although outcomes in individual decision systems are highly variable and at the mercy of individual decision makers, collective systems based on pooling decrease this variation, thereby promoting reliability, fairness, and possibly even trust.

•

Professional decision makers typically differ in their response bias and accuracy

•

Such differences undermine the reliability and fairness of decision systems

•

Pooling decisions reduces such variation in response bias and accuracy

•

This occurred in cancer diagnostics, fingerprint analysis, and forecasting

Science-Driven Societal Transformation, Part I: Worldview

Humanity faces serious social and environmental problems, including climate change and biodiversity loss. Increasingly, scientists, global policy experts, and the general public conclude that incremental approaches to reduce risk are insufficient and transformative change is needed across all sectors of society. However, the meaning of transformation is still unsettled in the literature, as is the proper role of science in fostering it. This paper is the first in a three-part series that adds to the discussion by proposing a novel science-driven research-and-development program aimed at societal transformation. More than a proposal, it offers a perspective and conceptual framework from which societal transformation might be approached. As part of this, it advances a formal mechanics with which to model and understand self-organizing societies of individuals. While acknowledging the necessity of reform to existing societal systems (e.g., governance, economic, and financial systems), the focus of the series is on transformation understood as systems change or systems migration—the de novo development of and migration to new societal systems. The series provides definitions, aims, reasoning, worldview, and a theory of change, and discusses fitness metrics and design principles for new systems. This first paper proposes a worldview, built using ideas from evolutionary biology, complex systems science, cognitive sciences, and information theory, which is intended to serve as the foundation for the R&D program. Subsequent papers in the series build on the worldview to address fitness metrics, system design, and other topics.

Larger group sizes facilitate the emergence and spread of innovations in a group-living bird

The benefits of group living have traditionally been attributed to risk dilution or the efficient exploitation of resources; individuals in social groups may therefore benefit from access to valuable information. If sociality facilitates access to information, then individuals in larger groups may be predicted to solve novel problems faster than individuals in smaller groups. Additionally, larger group sizes may facilitate the subsequent spread of innovations within animal groups, as has been proposed for human societies. We presented a novel foraging task (where a food reward could be accessed by pushing a self-shutting sliding door) to 16 groups of wild, cooperatively breeding Australian magpies, Cracticus tibicen dorsalis, ranging in size from two to 11 individuals. We found a nonlinear decline in the time taken for the innovative behaviour to emerge with increasing group size, and social information use facilitated the transmission of novel behaviour, with it spreading more quickly in larger than smaller groups. This study provides important evidence for a nonlinear relationship between group size and the emergence of innovation (and its subsequent transmission) in a wild population of animals. Further work investigating the scope and strength of group size-innovation relationships, and the mechanisms underpinning them, will help us understand the potential advantages of living in larger social groups.

Collective Choice, Collaboration, and Communication

This article reviews recent empirical research on collective choice and collaborative problem solving. Much of the collective choice research focuses on hidden profiles. A hidden profile exists when group members individually have information favoring suboptimal choices but the group collectively has information favoring an optimal choice. Groups are notoriously bad at discovering optimal choices when information is distributed to create a hidden profile. Reviewed work identifies informational structures, individual processing biases, and social motivations that inhibit and facilitate the discovery of hidden profiles. The review of collaborative problem-solving research is framed by Larson's concept of synergy. Synergy refers to performance gains that are attributable to collaboration. Recent research has addressed factors that result in groups performing as well as their best member (weak synergy) and better than their best member (strong synergy). Communication dynamics underlying both collective choice and collaborative problem solving are discussed.

Combining independent decisions increases diagnostic accuracy of reading lumbosacral radiographs and magnetic resonance imaging

Diagnosing the causes of low back pain is a challenging task, prone to errors. A novel approach to increase diagnostic accuracy in medical decision making is collective intelligence, which refers to the ability of groups to outperform individual decision makers in solving problems. We investigated whether combining the independent ratings of chiropractors, chiropractic radiologists and medical radiologists can improve diagnostic accuracy when interpreting diagnostic images of the lumbosacral spine. Evaluations were obtained from two previously published studies: study 1 consisted of 13 raters independently rating 300 lumbosacral radiographs; study 2 consisted of 14 raters independently rating 100 lumbosacral magnetic resonance images. In both studies, raters evaluated the presence of "abnormalities", which are indicators of a serious health risk and warrant immediate further examination. We combined independent decisions of raters using a majority rule which takes as final diagnosis the decision of the majority of the group. We compared the performance of the majority rule to the performance of single raters. Our results show that with increasing group size (i.e., increasing the number of independent decisions) both sensitivity and specificity increased in both data-sets, with groups consistently outperforming single raters. These results were found for radiographs and MR image reading alike. Our findings suggest that combining independent ratings can improve the accuracy of lumbosacral diagnostic image reading.

Dynamical networks of influence in small group discussions

In many domains of life, business and management, numerous problems are addressed by small groups of individuals engaged in face-to-face discussions. While research in social psychology has a long history of studying the determinants of small group performances, the internal dynamics that govern a group discussion are not yet well understood. Here, we rely on computational methods based on network analyses and opinion dynamics to describe how individuals influence each other during a group discussion. We consider the situation in which a small group of three individuals engages in a discussion to solve an estimation task. We propose a model describing how group members gradually influence each other and revise their judgments over the course of the discussion. The main component of the model is an influence network-a weighted, directed graph that determines the extent to which individuals influence each other during the discussion. In simulations, we first study the optimal structure of the influence network that yields the best group performances. Then, we implement a social learning process by which individuals adapt to the past performance of their peers, thereby affecting the structure of the influence network in the long run. We explore the mechanisms underlying the emergence of efficient or maladaptive networks and show that the influence network can converge towards the optimal one, but only when individuals exhibit a social discounting bias by downgrading the relative performances of their peers. Finally, we find a late-speaker effect, whereby individuals who speak later in the discussion are perceived more positively in the long run and are thus more influential. The numerous predictions of the model can serve as a basis for future experiments, and this work opens research on small group discussion to computational social sciences.

Criticality triggers the emergence of collective intelligence in groups

A spinlike model mimicking human behavior in groups is employed to investigate the dynamics of the decision-making process. Within the model, the temporal evolution of the state of systems is governed by a time-continuous Markov chain. The transition rates of the resulting master equation are defined in terms of the change of interaction energy between the neighboring agents (change of the level of conflict) and the change of a locally defined agent fitness. Three control parameters can be identified: (i) the social interaction strength βJ measured in units of social temperature, (ii) the level of confidence β^{'} that each individual has on his own expertise, and (iii) the level of knowledge p that identifies the expertise of each member. Based on these three parameters, the phase diagrams of the system show that a critical transition front exists where a sharp and concurrent change in fitness and consensus takes place. We show that at the critical front, the information leakage from the fitness landscape to the agents is maximized. This event triggers the emergence of the collective intelligence of the group, and in the end it leads to a dramatic improvement in the decision-making performance of the group. The effect of size M of the system is also investigated, showing that, depending on the value of the control parameters, increasing M may be either beneficial or detrimental.

Can Simple Transmission Chains Foster Collective Intelligence in Binary-Choice Tasks?

In many social systems, groups of individuals can find remarkably efficient solutions to complex cognitive problems, sometimes even outperforming a single expert. The success of the group, however, crucially depends on how the judgments of the group members are aggregated to produce the collective answer. A large variety of such aggregation methods have been described in the literature, such as averaging the independent judgments, relying on the majority or setting up a group discussion. In the present work, we introduce a novel approach for aggregating judgments—the transmission chain—which has not yet been consistently evaluated in the context of collective intelligence. In a transmission chain, all group members have access to a unique collective solution and can improve it sequentially. Over repeated improvements, the collective solution that emerges reflects the judgments of every group members. We address the question of whether such a transmission chain can foster collective intelligence for binary-choice problems. In a series of numerical simulations, we explore the impact of various factors on the performance of the transmission chain, such as the group size, the model parameters, and the structure of the population. The performance of this method is compared to those of the majority rule and the confidence-weighted majority. Finally, we rely on two existing datasets of individuals performing a series of binary decisions to evaluate the expected performances of the three methods empirically. We find that the parameter space where the transmission chain has the best performance rarely appears in real datasets. We conclude that the transmission chain is best suited for other types of problems, such as those that have cumulative properties.

Boosting medical diagnostics by pooling independent judgments

Collective intelligence refers to the ability of groups to outperform individual decision makers when solving complex cognitive problems. Despite its potential to revolutionize decision making in a wide range of domains, including medical, economic, and political decision making, at present, little is known about the conditions underlying collective intelligence in real-world contexts. We here focus on two key areas of medical diagnostics, breast and skin cancer detection. Using a simulation study that draws on large real-world datasets, involving more than 140 doctors making more than 20,000 diagnoses, we investigate when combining the independent judgments of multiple doctors outperforms the best doctor in a group. We find that similarity in diagnostic accuracy is a key condition for collective intelligence: Aggregating the independent judgments of doctors outperforms the best doctor in a group whenever the diagnostic accuracy of doctors is relatively similar, but not when doctors' diagnostic accuracy differs too much. This intriguingly simple result is highly robust and holds across different group sizes, performance levels of the best doctor, and collective intelligence rules. The enabling role of similarity, in turn, is explained by its systematic effects on the number of correct and incorrect decisions of the best doctor that are overruled by the collective. By identifying a key factor underlying collective intelligence in two important real-world contexts, our findings pave the way for innovative and more effective approaches to complex real-world decision making, and to the scientific analyses of those approaches.

Hospitality Invites Sociability, Which Builds Cohesion: a Model for the Role of Main Streets in Population Mental Health

The aim of this study was to investigate the contribution of main streets to community social cohesion, a factor important to health. Prior work suggests that casual contact in public space, which we call "sociability," facilitates more sustained social bonds in the community. We appropriate the term "hospitality" to describe a main street's propensity to support a density of such social interactions. Hospitality is a result of the integrity and complex contents of the main street and surrounding area. We examine this using a typology we term "box-circle-line" to represent the streetscape (the box), the local neighborhood (the circle), and the relationship to the regional network of streets (the line). Through field visits to 50 main streets in New Jersey and elsewhere, and a systematic qualitative investigation of main streets in a densely interconnected urban region (Essex County, New Jersey), we observed significant variation in main street hospitality, which generally correlated closely with sociability. Physical elements such as street wall, neighborhood elements such as connectivity, inter-community elements such as access and perceived welcome, and socio-political elements such as investment and racial discrimination were identified as relevant to main street hospitality. We describe the box-circle-line as a theoretical model for main street hospitality that links these various factors and provides a viable framework for further research into main street hospitality, particularly with regard to geographic health disparities.

Fitness landscapes among many options under social influence

Cultural learning represents a novel problem in that an optimal decision depends not only on intrinsic utility of the decision/behavior but also on transparency of costs and benefits, the degree of social versus individual learning, and the relative popularity of each possible choice in a population. In terms of a fitness-landscape function, this recursive relationship means that multiple equilibria can exist. Here we use discrete-choice theory to construct a fitness-landscape function for a bi-axial decision-making map that plots the magnitude of social influence in the learning process against the costs and payoffs of decisions. Specifically, we use econometric and statistical methods to estimate not only the fitness function but also movements along the map axes. To search for these equilibria, we employ a hill-climbing algorithm that leads to the expected values of optimal decisions, which we define as peaks on the fitness landscape. We illustrate how estimation of a measure of transparency, a measure of social influence, and the associated fitness landscape can be accomplished using panel data sets.

Self-organized flexible leadership promotes collective intelligence in human groups

Collective intelligence refers to the ability of groups to outperform individual decision-makers. At present, relatively little is known about the mechanisms promoting collective intelligence in natural systems. We here test a novel mechanism generating collective intelligence: self-organization according to information quality. We tested this mechanism by performing simulated predator detection experiments using human groups. By continuously tracking the personal information of all members prior to collective decisions, we found that individuals adjusted their response time during collective decisions to the accuracy of their personal information. When individuals possessed accurate personal information, they decided quickly during collective decisions providing accurate information to the other group members. By contrast, when individuals had inaccurate personal information, they waited longer, allowing them to use social information before making a decision. Individuals deciding late during collective decisions had an increased probability of changing their decision leading to increased collective accuracy. Our results thus show that groups can self-organize according to the information accuracy of their members, thereby promoting collective intelligence. Interestingly, we find that individuals flexibly acted both as leader and as follower depending on the quality of their personal information at any particular point in time.

Collective behaviour, uncertainty and environmental change

A central aspect of cultural evolutionary theory concerns how human groups respond to environmental change. Although we are painting with a broad brush, it is fair to say that prior to the twenty-first century, adaptation often happened gradually over multiple human generations, through a combination of individual and social learning, cumulative cultural evolution and demographic shifts. The result was a generally resilient and sustainable population. In the twenty-first century, however, considerable change happens within small portions of a human generation, on a vastly larger range of geographical and population scales and involving a greater degree of horizontal learning. As a way of gauging the complexity of societal response to environmental change in a globalized future, we discuss several theoretical tools for understanding how human groups adapt to uncertainty. We use our analysis to estimate the limits of predictability of future societal change, in the belief that knowing when to hedge bets is better than relying on a false sense of predictability.

Disentangling the impact of social groups on response times and movement dynamics in evacuations

Crowd evacuations are paradigmatic examples for collective behaviour, as interactions between individuals lead to the overall movement dynamics. Approaches assuming that all individuals interact in the same way have significantly improved our understanding of pedestrian crowd evacuations. However, this scenario is unlikely, as many pedestrians move in social groups that are based on friendship or kinship. We test how the presence of social groups affects the egress time of individuals and crowds in a representative crowd evacuation experiment. Our results suggest that the presence of social groups increases egress times and that this is largely due to differences at two stages of evacuations. First, individuals in social groups take longer to show a movement response at the start of evacuations, and, second, they take longer to move into the vicinity of the exits once they have started to move towards them. Surprisingly, there are no discernible time differences between the movement of independent individuals and individuals in groups directly in front of the exits. We explain these results and discuss their implications. Our findings elucidate behavioural differences between independent individuals and social groups in evacuations. Such insights are crucial for the control of crowd evacuations and for planning mass events.

Ants work harder during consensus decision-making in small groups

Individuals derive many benefits from being social, one of which is improved accuracy of decision-making, the so-called 'wisdom of the crowds' effect. This advantage arises because larger groups can pool information from more individuals. At present, limited empirical data indicate that larger groups outperform smaller ones during consensus decision-making in human and non-human animals. Inaccurate decisions can lead to significant costs, and we might therefore expect individuals in small groups to employ mechanisms to compensate for the lack of numbers. Small groups may be able to maintain decision accuracy if individuals are better informed than those in larger groups and/or by increasing the proportion of the group involved in collective decision-making relative to larger groups. In this study, we use interactive computer vision software to investigate individual contributions to consensus decision-making during house-hunting in different sized groups of the ant Myrmecina nipponica. We show that individuals in small colonies invest greater effort in the consensus decision process than those in large colonies and should be better informed as a result. This may act to ameliorate the limitations of group size, but could leave smaller groups more susceptible to additional stresses.