Task complexity moderates group synergy

When combinations of humans and AI are useful: A systematic review and meta-analysis

Inspired by the increasing use of artificial intelligence (AI) to augment humans, researchers have studied human-AI systems involving different tasks, systems and populations. Despite such a large body of work, we lack a broad conceptual understanding of when combinations of humans and AI are better than either alone. Here we addressed this question by conducting a preregistered systematic review and meta-analysis of 106 experimental studies reporting 370 effect sizes. We searched an interdisciplinary set of databases (the Association for Computing Machinery Digital Library, the Web of Science and the Association for Information Systems eLibrary) for studies published between 1 January 2020 and 30 June 2023. Each study was required to include an original human-participants experiment that evaluated the performance of humans alone, AI alone and human-AI combinations. First, we found that, on average, human-AI combinations performed significantly worse than the best of humans or AI alone (Hedges' g = -0.23; 95% confidence interval, -0.39 to -0.07). Second, we found performance losses in tasks that involved making decisions and significantly greater gains in tasks that involved creating content. Finally, when humans outperformed AI alone, we found performance gains in the combination, but when AI outperformed humans alone, we found losses. Limitations of the evidence assessed here include possible publication bias and variations in the study designs analysed. Overall, these findings highlight the heterogeneity of the effects of human-AI collaboration and point to promising avenues for improving human-AI systems.

How large language models can reshape collective intelligence

Collective intelligence underpins the success of groups, organizations, markets and societies. Through distributed cognition and coordination, collectives can achieve outcomes that exceed the capabilities of individuals-even experts-resulting in improved accuracy and novel capabilities. Often, collective intelligence is supported by information technology, such as online prediction markets that elicit the 'wisdom of crowds', online forums that structure collective deliberation or digital platforms that crowdsource knowledge from the public. Large language models, however, are transforming how information is aggregated, accessed and transmitted online. Here we focus on the unique opportunities and challenges this transformation poses for collective intelligence. We bring together interdisciplinary perspectives from industry and academia to identify potential benefits, risks, policy-relevant considerations and open research questions, culminating in a call for a closer examination of how large language models affect humans' ability to collectively tackle complex problems.

Reconsidering Leadership Development: From Programs to Developmental Systems

We argue for reconsidering leadership development based on open systems theory and systems design principles. A primary advantage of open systems thinking is that it encourages holistic approaches to development and avoids episodic program-based training and piecemeal thinking. The latter approaches are both limited and limiting yet tend to be the preferred approach to leadership development in organizations. Open systems approaches to development offer numerous advantages both conceptually and pragmatically, especially through the incorporation of ongoing feedback cycles. Core practices that define a leadership development system are presented and implications are discussed.

The Effects of Group Composition and Dynamics on Collective Performance

As organizations gravitate to group-based structures, the problem of improving performance through judicious selection of group members has preoccupied scientists and managers alike. However, which individual attributes best predict group performance remains poorly understood. Here, we describe a preregistered experiment in which we simultaneously manipulated four widely studied attributes of group compositions: skill level, skill diversity, social perceptiveness, and cognitive style diversity. We find that while the average skill level of group members, skill diversity, and social perceptiveness are significant predictors of group performance, skill level dominates all other factors combined. Additionally, we explore the relationship between patterns of collaborative behavior and performance outcomes and find that any potential gains in solution quality from additional communication between the group members are outweighed by the overhead time cost, leading to lower overall efficiency. However, groups exhibiting more "turn-taking" behavior are considerably faster and thus more efficient. Finally, contrary to our expectation, we find that group compositional factors (i.e., skill level and social perceptiveness) are not associated with the amount of communication between group members nor turn-taking dynamics.

Network structure shapes the impact of diversity in collective learning

It is widely believed that diversity arising from different skills enhances the performance of teams, and in particular, their ability to learn and innovate. However, diversity has also been associated with negative effects on the communication and coordination within collectives. Yet, despite the importance of diversity as a concept, we still lack a mechanistic understanding of how its impact is shaped by the underlying social network. To fill this gap, we model skill diversity within a simple model of collective learning and show that its effect on collective performance differs depending on the complexity of the task and the network density. In particular, we find that diversity consistently impairs performance in simple tasks. In contrast, in complex tasks, link density modifies the effect of diversity: while homogeneous populations outperform diverse ones in sparse networks, the opposite is true in dense networks, where diversity boosts collective performance. Our findings also provide insight on how to forge teams in an increasingly interconnected world: the more we are connected, the more we can benefit from diversity to solve complex problems.

Organizational system thinking as a cognitive framework to meet climate targets

System thinking is a crucial cognitive framework to enable individual pro-environmental behavioral changes. Indeed, a large body of literature has shown a significant and positive association between individuals' system thinking capacities and perceptions of the threat posed by climate change. However, individual behavioral changes play a limited role in addressing climate change compared to large organizations involved in a significantly larger share of economic activities. Do organizations exhibit system thinking capacities? Here, we conjecture that system thinking is a cognitive framework observable at an aggregated group level and, therefore, organizations, not just individuals, can exhibit characteristic levels of system thinking. We conceptualize a definition of organizational system thinking and develop an empirical method to estimate it using a large body of textual data from business organizations. Then, we show that system thinking organizations are more likely to lower emissions and align them with the pathways required to meet the climate targets set by the Paris Agreement. Finally, we discussed the theoretical and policy implication of our study. Overall, our results suggest that system thinking is a relevant organization-level cognitive framework that can help organizations align their emissions with global climate targets.

Flexible social inference facilitates targeted social learning when rewards are not observable

Groups coordinate more effectively when individuals are able to learn from others' successes. But acquiring such knowledge is not always easy, especially in real-world environments where success is hidden from public view. We suggest that social inference capacities may help bridge this gap, allowing individuals to update their beliefs about others' underlying knowledge and success from observable trajectories of behaviour. We compared our social inference model against simpler heuristics in three studies of human behaviour in a collective-sensing task. Experiment 1 demonstrated that average performance improved as a function of group size at a rate greater than predicted by heuristic models. Experiment 2 introduced artificial agents to evaluate how individuals selectively rely on social information. Experiment 3 generalized these findings to a more complex reward landscape. Taken together, our findings provide insight into the relationship between individual social cognition and the flexibility of collective behaviour.

The Social Route to Abstraction: Interaction and Diversity Enhance Performance and Transfer in a Rule-Based Categorization Task

Capacities for abstract thinking and problem-solving are central to human cognition. Processes of abstraction allow the transfer of experiences and knowledge between contexts helping us make informed decisions in new or changing contexts. While we are often inclined to relate such reasoning capacities to individual minds and brains, they may in fact be contingent on human-specific modes of collaboration, dialogue, and shared attention. In an experimental study, we test the hypothesis that social interaction enhances cognitive processes of rule-induction, which in turn improves problem-solving performance. Through three sessions of increasing complexity, individuals and groups were presented with a problem-solving task requiring them to categorize a set of visual stimuli. To assess the character of participants' problem representations, after each training session, they were presented with a transfer task involving stimuli that differed in appearance, but shared relations among features with the training set. Besides, we compared participants' categorization behaviors to simulated agents relying on exemplar learning. We found that groups performed superior to individuals and agents in the training sessions and were more likely to correctly generalize their observations in the transfer phase, especially in the high complexity session, suggesting that groups more effectively induced underlying categorization rules from the stimuli than individuals and agents. Crucially, variation in performance among groups was predicted by semantic diversity in members' dialogical contributions, suggesting a link between social interaction, cognitive diversity, and abstraction.

Multidisciplinary learning through collective performance favors decentralization

Many models of learning in teams assume that team members can share solutions or learn concurrently. However, these assumptions break down in multidisciplinary teams where team members often complete distinct, interrelated pieces of larger tasks. Such contexts make it difficult for individuals to separate the performance effects of their own actions from the actions of interacting neighbors. In this work, we show that individuals can overcome this challenge by learning from network neighbors through mediating artifacts (like collective performance assessments). When neighbors' actions influence collective outcomes, teams with different networks perform relatively similarly to one another. However, varying a team's network can affect performance on tasks that weight individuals' contributions by network properties. Consequently, when individuals innovate (through "exploring" searches), dense networks hurt performance slightly by increasing uncertainty. In contrast, dense networks moderately help performance when individuals refine their work (through "exploiting" searches) by efficiently finding local optima. We also find that decentralization improves team performance across a battery of 34 tasks. Our results offer design principles for multidisciplinary teams within which other forms of learning prove more difficult.

A guide to megastudies

How can behavioral insights best be leveraged to solve pressing policy challenges? Because research studies are typically designed to test the validity of a particular idea, surprisingly little is known about the relative efficacy of different approaches to changing behavior in any given policy context. We discuss megastudies as a research approach that can surmount this and other obstacles to developing optimal behaviorally informed policy interventions. We define a megastudy as "a massive field experiment in which many different treatments are tested synchronously in one large sample using a common, objectively measured outcome." We summarize this apples-to-apples approach to research and lay out recommendations, limitations, and promising future directions for scholars who might want to conduct or evaluate megastudies.

The network science of collective intelligence

In the last few years, breakthroughs in computational and experimental techniques have produced several key discoveries in the science of networks and human collective intelligence. This review presents the latest scientific findings from two key fields of research: collective problem-solving and the wisdom of the crowd. I demonstrate the core theoretical tensions separating these research traditions and show how recent findings offer a new synthesis for understanding how network dynamics alter collective intelligence, both positively and negatively. I conclude by highlighting current theoretical problems at the forefront of research on networked collective intelligence, as well as vital public policy challenges that require new research efforts.

What Makes Inventions Become Traditions?

Although anthropology was the first academic discipline to investigate cultural change, many other disciplines have made noteworthy contributions to understanding what influences the adoption of new behaviors. Drawing on a broad, interdisciplinary literature covering both humans and nonhumans, we examine ( a) which features of behavioral traits make them more transmissible, ( b) which individual characteristics of inventors promote copying of their inventions, ( c) which characteristics of individuals make them more prone to adopting new behaviors, ( d) which characteristics of dyadic relationships promote cultural transmission, ( e) which properties of groups (e.g., network structures) promote transmission of traits, and ( f) which characteristics of groups promote retention, rather than extinction, of cultural traits. One of anthropology's strengths is its readiness to adopt and improve theories and methods from other disciplines, integrating them into a more holistic approach; hence, we identify approaches that might be particularly useful to biological and cultural anthropologists, and knowledge gaps that should be filled.

The distribution of initial estimates moderates the effect of social influence on the wisdom of the crowd

Whether, and under what conditions, groups exhibit “crowd wisdom” has been a major focus of research across the social and computational sciences. Much of this work has focused on the role of social influence in promoting the wisdom of the crowd versus leading the crowd astray and has resulted in conflicting conclusions about how social network structure determines the impact of social influence. Here, we demonstrate that it is not enough to consider the network structure in isolation. Using theoretical analysis, numerical simulation, and reanalysis of four experimental datasets (totaling 2885 human subjects), we find that the wisdom of crowds critically depends on the interaction between (i) the centralization of the social influence network and (ii) the distribution of the initial individual estimates. By adopting a framework that integrates both the structure of the social influence and the distribution of the initial estimates, we bring previously conflicting results under one theoretical framework and clarify the effects of social influence on the wisdom of crowds.

Overcoming Individual Limitations Through Distributed Computation: Rational Information Accumulation in Multigenerational Populations

Many of the computational problems people face are difficult to solve under the limited time and cognitive resources available to them. Overcoming these limitations through social interaction is one of the most distinctive features of human intelligence. In this paper, we show that information accumulation in multigenerational social networks can be produced by a form of distributed Bayesian inference that allows individuals to benefit from the experience of previous generations while expending little cognitive effort. In doing so, we provide a criterion for assessing the rationality of a population that extends traditional analyses of the rationality of individuals. We tested the predictions of this analysis in two highly controlled behavioral experiments where the social transmission structure closely matched the assumptions of our model. Participants made decisions on simple categorization tasks that relied on and contributed to accumulated knowledge. Success required these microsocieties to accumulate information distributed across people and time. Our findings illustrate how in certain settings, distributed computation at the group level can pool information and resources, allowing limited individuals to perform effectively on complex tasks.

The potential for effective reasoning guides children's preference for small group discussion over crowdsourcing

Communication between social learners can make a group collectively "wiser" than any individual, but conformist tendencies can also distort collective judgment. We asked whether intuitions about when communication is likely to improve or distort collective judgment could allow social learners to take advantage of the benefits of communication while minimizing the risks. In three experiments (n = 360), 7- to 10-year old children and adults decided whether to refer a question to a small group for discussion or "crowdsource" independent judgments from individual advisors. For problems affording the kind of 'demonstrative' reasoning that allows a group member to reliably correct errors made by even a majority, all ages preferred to consult the discussion group, even compared to a crowd ten times as large-consistent with past research suggesting that discussion groups regularly outperform even their best members for reasoning problems. In contrast, we observed a consistent developmental shift towards crowdsourcing independent judgments when reasoning by itself was insufficient to conclusively answer a question. Results suggest sophisticated intuitions about the nature of social influence and collective intelligence may guide our social learning strategies from early in development.

Task complexity moderates group synergy

Scientists and managers alike have been preoccupied with the question of whether and, if so, under what conditions groups of interacting problem solvers outperform autonomous individuals. Here we describe an experiment in which individuals and groups were evaluated on a series of tasks of varying complexity. We find that groups are as fast as the fastest individual and more efficient than the most efficient individual when the task is complex but not when the task is simple. We then precisely quantify synergistic gains and process losses associated with interacting groups, finding that the balance between the two depends on complexity. Our study has the potential to reconcile conflicting findings about group synergy in previous work.

Complexity—defined in terms of the number of components and the nature of the interdependencies between them—is clearly a relevant feature of all tasks that groups perform. Yet the role that task complexity plays in determining group performance remains poorly understood, in part because no clear language exists to express complexity in a way that allows for straightforward comparisons across tasks. Here we avoid this analytical difficulty by identifying a class of tasks for which complexity can be varied systematically while keeping all other elements of the task unchanged. We then test the effects of task complexity in a preregistered two-phase experiment in which 1,200 individuals were evaluated on a series of tasks of varying complexity (phase 1) and then randomly assigned to solve similar tasks either in interacting groups or as independent individuals (phase 2). We find that interacting groups are as fast as the fastest individual and more efficient than the most efficient individual for complex tasks but not for simpler ones. Leveraging our highly granular digital data, we define and precisely measure group process losses and synergistic gains and show that the balance between the two switches signs at intermediate values of task complexity. Finally, we find that interacting groups generate more solutions more rapidly and explore the solution space more broadly than independent problem solvers, finding higher-quality solutions than all but the highest-scoring individuals.