Exploration versus exploitation in space, mind, and society

Free will: An Example of the Dopaminergic System

Neuroscience has convinced people that much of their behavior is determined by causes unknown to them and beyond their control. However, are advances in neuroscience truly a prerequisite for such beliefs? Robert Kane's theory of ultimate responsibility is libertarian theory. Its innovative nature makes it possible to discuss the neurophysiological basis of its postulates. Using the functions of the midbrain dopaminergic system as an example, this article provides an overview of this neurophysiological basis. According to Kane, if we are to be ultimately responsible for our wills as well as for our actions, some actions in our lives must lack sufficient motives and causes. These are self-forming actions. Dopamine is hypothesized to mediate self-forming action execution. Dopamine not only mediates action but also ensures synaptic plasticity in the brain, that is, learning from action; hence, dopamine changes the acting individual and provides the formation of our own wills. The basal ganglia, which are the main target of dopamine in the brain, act through parallel pathways and are involved in decision-making processes. Dopamine is also involved in the regulation of the neurodynamical properties of prefrontal cortex networks with random spiking noise. It can be assumed that the activity of the dopaminergic system is closely related to the physiological basis of free will.

Balancing exploration and exploitation? The impact of cost and inhibitory control on information gathering in early childhood

The information-seeking behaviour of adults focuses on optimizing the gathering and utilizing information to minimize search costs. In contrast, children tend to engage in information search during decision-making with less consideration for costs. This difference in behaviour is believed to be linked to the development of executive functions. Therefore, the present study aimed to investigate the relationship between executive function and cost-related information-gathering behaviour. We assessed 56 children aged 4-6 years, involving three tasks: an information-gathering task, an inhibitory control and a working memory task. In the information-gathering task, children participated in both non-cost and cost conditions, where they were given the opportunity to freely gather information or incur a cost to acquire information. The findings revealed that children with higher inhibitory control tended to gather less information when a cost was involved. This highlights the important role of inhibitory control in shaping information-seeking behaviour in early childhood.

Temporal Patterns of Angular Displacement of Endosomes: Insights into Motor Protein Exchange Dynamics

The material transport system, facilitated by motor proteins, plays a vital role in maintaining a non-equilibrium cellular state. However, understanding the temporal coordination of motor protein activity requires an advanced imaging technique capable of measuring 3D angular displacement in real-time. In this study, a Fourier transform-based plasmonic dark-field microscope has been developed using anisotropic nanoparticles, enabling the prolonged and simultaneous observation of endosomal lateral and rotational motion. A sequence of discontinuous 3D angular displacements has been observed during the pause and run phases of transport. Notably, a serially correlated temporal pattern in the intermittent rotational events has been demonstrated during the tug-of-war mechanism, indicating Markovian switching between the exploitational and explorational modes of motor protein exchange prior to resuming movement. Alterations in transition frequency and the exploitation-to-exploration ratio upon dynein inhibitor treatment highlight the relationship between disrupted motor coordination and reduced endosomal transport efficiency. Collectively, these results suggest the importance of orchestrated temporal motor protein patterns for efficient cellular transport.

A Comparison of Rapid Rule-Learning Strategies in Humans and Monkeys

Interspecies comparisons are key to deriving an understanding of the behavioral and neural correlates of human cognition from animal models. We perform a detailed comparison of the strategies of female macaque monkeys to male and female humans on a variant of the Wisconsin Card Sorting Test (WCST), a widely studied and applied task that provides a multiattribute measure of cognitive function and depends on the frontal lobe. WCST performance requires the inference of a rule change given ambiguous feedback. We found that well-trained monkeys infer new rules three times more slowly than minimally instructed humans. Input-dependent hidden Markov model-generalized linear models were fit to their choices, revealing hidden states akin to feature-based attention in both species. Decision processes resembled a win-stay, lose-shift strategy with interspecies similarities as well as key differences. Monkeys and humans both test multiple rule hypotheses over a series of rule-search trials and perform inference-like computations to exclude candidate choice options. We quantitatively show that perseveration, random exploration, and poor sensitivity to negative feedback account for the slower task-switching performance in monkeys.

Exploration-Exploitation and Suicidal Behavior in Borderline Personality Disorder and Depression

Importance

Clinical theory and behavioral studies suggest that people experiencing suicidal crisis are often unable to find constructive solutions or incorporate useful information into their decisions, resulting in premature convergence on suicide and neglect of better alternatives. However, prior studies of suicidal behavior have not formally examined how individuals resolve the tradeoffs between exploiting familiar options and exploring potentially superior alternatives.

Objective

To investigate exploration and exploitation in suicidal behavior from the formal perspective of reinforcement learning.

Design, Setting, and Participants

Two case-control behavioral studies of exploration-exploitation of a large 1-dimensional continuous space and a 21-day prospective ambulatory study of suicidal ideation were conducted between April 2016 and March 2022. Participants were recruited from inpatient psychiatric units, outpatient clinics, and the community in Pittsburgh, Pennsylvania, and underwent laboratory and ambulatory assessments. Adults diagnosed with borderline personality disorder (BPD) and midlife and late-life major depressive disorder (MDD) were included, with each sample including demographically equated groups with a history of high-lethality suicide attempts, low-lethality suicide attempts, individuals with BPD or MDD but no suicide attempts, and control individuals without psychiatric disorders. The MDD sample also included a subgroup with serious suicidal ideation.

Main Outcomes and Measures

Behavioral (model-free and model-derived) indices of exploration and exploitation, suicide attempt lethality (Beck Lethality Scale), and prospectively assessed suicidal ideation.

Results

The BPD group included 171 adults (mean [SD] age, 30.55 [9.13] years; 135 [79%] female). The MDD group included 143 adults (mean [SD] age, 62.03 [6.82] years; 81 [57%] female). Across the BPD (χ23 = 50.68; P < .001) and MDD (χ24 = 36.34; P < .001) samples, individuals with high-lethality suicide attempts discovered fewer options than other groups as they were unable to shift away from unrewarded options. In contrast, those with low-lethality attempts were prone to excessive behavioral shifts after rewarded and unrewarded actions. No differences were seen in strategic early exploration or in exploitation. Among 84 participants with BPD in the ambulatory study, 56 reported suicidal ideation. Underexploration also predicted incident suicidal ideation (χ21 = 30.16; P < .001), validating the case-control results prospectively. The findings were robust to confounds, including medication exposure, affective state, and behavioral heterogeneity.

Conclusions and Relevance

The findings suggest that narrow exploration and inability to abandon inferior options are associated with serious suicidal behavior and chronic suicidal thoughts. By contrast, individuals in this study who engaged in low-lethality suicidal behavior displayed a low threshold for taking potentially disadvantageous actions.

Knowing what to know: Implications of the choice of prior distribution on the behavior of adaptive design optimization

Adaptive design optimization (ADO) is a state-of-the-art technique for experimental design (Cavagnaro et al., 2010). ADO dynamically identifies stimuli that, in expectation, yield the most information about a hypothetical construct of interest (e.g., parameters of a cognitive model). To calculate this expectation, ADO leverages the modeler's existing knowledge, specified in the form of a prior distribution. Informative priors align with the distribution of the focal construct in the participant population. This alignment is assumed by ADO's internal assessment of expected information gain. If the prior is instead misinformative, i.e., does not align with the participant population, ADO's estimates of expected information gain could be inaccurate. In many cases, the true distribution that characterizes the participant population is unknown, and experimenters rely on heuristics in their choice of prior and without an understanding of how this choice affects ADO's behavior. Our work introduces a mathematical framework that facilitates investigation of the consequences of the choice of prior distribution on the efficiency of experiments designed using ADO. Through theoretical and empirical results, we show that, in the context of prior misinformation, measures of expected information gain are distinct from the correctness of the corresponding inference. Through a series of simulation experiments, we show that, in the case of parameter estimation, ADO nevertheless outperforms other design methods. Conversely, in the case of model selection, misinformative priors can lead inference to favor the wrong model, and rather than mitigating this pitfall, ADO exacerbates it.

Exploration, exploitation, and development: Developmental shifts in decision-making

Decision-making requires balancing exploration with exploitation, yet children are highly exploratory, with exploration decreasing with development. Less is known about what drives these changes. We examined the development of decision-making in 188 three- to eight-year-old children (M = 64 months; 98 girls) and 26 adults (M = 19 years; 13 women). Children were recruited from ethnically diverse suburban middle-class neighborhoods of Columbus, Ohio, USA. Results indicate that mature reward-based choices emerge relatively late in development, with children tending to over-explore. Computational modeling suggests that this exploration is systematic rather than random, as children tend to avoid repeating choices made on the previous trial. This pattern of exploration (reminiscent of novelty preference) decreased with development, whereas the tendency to exploit increased.

Simple autonomous agents can enhance creative semantic discovery by human groups

Innovation is challenging, and theory and experiments indicate that groups may be better able to identify and preserve innovations than individuals. But innovation within groups faces its own challenges, including groupthink and truncated diffusion. We performed experiments involving a game in which people search for ideas in various conditions: alone, in networked social groups, or in networked groups featuring autonomous agents (bots). The objective was to search a semantic space of 20,000 nouns with defined similarities for an arbitrary noun with the highest point value. Participants (N = 1875) were embedded in networks (n = 125) of 15 nodes to which we sometimes added 2 bots. The bots had 3 possible strategies: they shared a random noun generated by their immediate neighbors, or a noun most similar from among those identified, or a noun least similar. We first confirm that groups are better able to explore a semantic space than isolated individuals. Then we show that when bots that share the most similar noun operate in groups facing a semantic space that is relatively easy to navigate, group performance is superior. Simple autonomous agents with interpretable behavior can affect the capacity for creative discovery of human groups.

Motor Strategies: The Role of Active Behavior in Spatial Hearing Research

When completing a task, the ability to implement behavioral strategies to solve it in an effective and cognitively less-demanding way is extremely adaptive for humans. This behavior makes it possible to accumulate evidence and test one's own predictions about the external world. In this work, starting from examples in the field of spatial hearing research, I analyze the importance of considering motor strategies in perceptual tasks, and I stress the urgent need to create ecological experimental settings, which are essential in allowing the implementation of such behaviors and in measuring them. In particular, I will consider head movements as an example of strategic behavior implemented to solve acoustic space-perception tasks.

Neural mechanisms of information seeking

We ubiquitously seek information to make better decisions. Particularly in the modern age, when more information is available at our fingertips than ever, the information we choose to collect determines the quality of our decisions. Decision neuroscience has long adopted empirical approaches where the information available to decision-makers is fully controlled by the researchers, leaving neural mechanisms of information seeking less understood. Although information seeking has long been studied in the context of the exploration-exploitation trade-off, recent studies have widened the scope to investigate more overt information seeking in a way distinct from other decision processes. Insights gained from these studies, accumulated over the last few years, raise the possibility that information seeking is driven by the reward system signaling the subjective value of information. In this piece, we review findings from the recent studies, highlighting the conceptual and empirical relationships between distinct literatures, and discuss future research directions necessary to establish a more comprehensive understanding of how individuals seek information as a part of value-based decision-making.

Performance errors during rodent learning reflect a dynamic choice strategy

Humans, even as infants, use cognitive strategies, such as exploration and hypothesis testing, to learn about causal interactions in the environment. In animal learning studies, however, it is challenging to disentangle higher-order behavioral strategies from errors arising from imperfect task knowledge or inherent biases. Here, we trained head-fixed mice on a wheel-based auditory two-choice task and exploited the intra- and inter-animal variability to understand the drivers of errors during learning. During learning, performance errors are dominated by a choice bias, which, despite appearing maladaptive, reflects a dynamic strategy. Early in learning, mice develop an internal model of the task contingencies such that violating their expectation of reward on correct trials (by using short blocks of non-rewarded "probe" trials) leads to an abrupt shift in strategy. During the probe block, mice behave more accurately with less bias, thereby using their learned stimulus-action knowledge to test whether the outcome contingencies have changed. Despite having this knowledge, mice continued to exhibit a strong choice bias during reinforced trials. This choice bias operates on a timescale of tens to hundreds of trials with a dynamic structure, shifting between left, right, and unbiased epochs. Biased epochs also coincided with faster motor kinematics. Although bias decreased across learning, expert mice continued to exhibit short bouts of biased choices interspersed with longer bouts of unbiased choices and higher performance. These findings collectively suggest that during learning, rodents actively probe their environment in a structured manner to refine their decision-making and maintain long-term flexibility.

Dynamic diversity is the answer to proxy failure

We argue that a diverse and dynamic pool of agents mitigates proxy failure. Proxy modularity plays a key role in the ongoing production of diversity. We review examples from a range of scales.

Approximate information for efficient exploration-exploitation strategies

This paper addresses the exploration-exploitation dilemma inherent in decision-making, focusing on multiarmed bandit problems. These involve an agent deciding whether to exploit current knowledge for immediate gains or explore new avenues for potential long-term rewards. We here introduce a class of algorithms, approximate information maximization (AIM), which employs a carefully chosen analytical approximation to the gradient of the entropy to choose which arm to pull at each point in time. AIM matches the performance of Thompson sampling, which is known to be asymptotically optimal, as well as that of Infomax from which it derives. AIM thus retains the advantages of Infomax while also offering enhanced computational speed, tractability, and ease of implementation. In particular, we demonstrate how to apply it to a 50-armed bandit game. Its expression is tunable, which allows for specific optimization in various settings, making it possible to surpass the performance of Thompson sampling at short and intermediary times.

The structure and development of explore-exploit decision making

A critical component of human learning reflects the balance people must achieve between focusing on the utility of what they know versus openness to what they have yet to experience. How individuals decide whether to explore new options versus exploit known options has garnered growing interest in recent years. Yet, the component processes underlying decisions to explore and whether these processes change across development remain poorly understood. By contrasting a variety of tasks that measure exploration in slightly different ways, we found that decisions about whether to explore reflect (a) random exploration that is not explicitly goal-directed and (b) directed exploration to purposefully reduce uncertainty. While these components similarly characterized the decision-making of both youth and adults, younger participants made decisions that were less strategic, but more exploratory and flexible, than those of adults. These findings are discussed in terms of how people adapt to and learn from changing environments over time.Data has been made available in the Open Science Foundation platform (osf.io).

The generative neural microdynamics of cognitive processing

The entorhinal cortex and hippocampus form a recurrent network that informs many cognitive processes, including memory, planning, navigation, and imagination. Neural recordings from these regions reveal spatially organized population codes corresponding to external environments and abstract spaces. Aligning the former cognitive functionalities with the latter neural phenomena is a central challenge in understanding the entorhinal-hippocampal circuit (EHC). Disparate experiments demonstrate a surprising level of complexity and apparent disorder in the intricate spatiotemporal dynamics of sequential non-local hippocampal reactivations, which occur particularly, though not exclusively, during immobile pauses and rest. We review these phenomena with a particular focus on their apparent lack of physical simulative realism. These observations are then integrated within a theoretical framework and proposed neural circuit mechanisms that normatively characterize this neural complexity by conceiving different regimes of hippocampal microdynamics as neuromarkers of diverse cognitive computations.

The journey within: mental navigation as a novel framework for understanding psychotherapeutic transformation

BACKGROUND

Despite the demonstrated efficacy of psychotherapy, the precise mechanisms that drive therapeutic transformations have posed a challenge and still remain unresolved. Here, we suggest a potential solution to this problem by introducing a framework based on the concept of mental navigation. It refers to our ability to navigate our cognitive space of thoughts, ideas, concepts, and memories, similar to how we navigate physical space. We start by analyzing the neural, cognitive, and experiential constituents intrinsic to mental navigation. Subsequently, we posit that the metaphoric spatial language we employ to articulate introspective experiences (e.g., "unexplored territory" or "going in circles") serves as a robust marker of mental navigation.

METHODS

Using large text corpora, we compared the utilization of spatial language between transcripts of psychotherapy sessions (≈ 12 M. words), casual everyday conversations (≈ 12 M. words), and fictional dialogues in movies (≈ 14 M. words). We also examined 110 psychotherapy transcripts qualitatively to discern patterns and dynamics associated with mental navigation.

RESULTS

We found a notable increase in the utilization of spatial metaphors during psychotherapy compared to casual everyday dialogues (U = 192.0, p = .001, d = 0.549) and fictional conversations (U = 211, p < .001, d = 0.792). In turn, analyzing the usage of non-spatial metaphors, we did not find significant differences between the three datasets (H = 0.682, p = 0.710). The qualitative analysis highlighted specific examples of mental navigation at play.

CONCLUSION

Mental navigation might underlie the psychotherapy process and serve as a robust framework for understanding the transformative changes it brings about.

Exploration versus exploitation decisions in the human brain: A systematic review of functional neuroimaging and neuropsychological studies

Thoughts and actions are often driven by a decision to either explore new avenues with unknown outcomes, or to exploit known options with predictable outcomes. Yet, the neural mechanisms underlying this exploration-exploitation trade-off in humans remain poorly understood. This is attributable to variability in the operationalization of exploration and exploitation as psychological constructs, as well as the heterogeneity of experimental protocols and paradigms used to study these choice behaviours. To address this gap, here we present a comprehensive review of the literature to investigate the neural basis of explore-exploit decision-making in humans. We first conducted a systematic review of functional magnetic resonance imaging (fMRI) studies of exploration-versus exploitation-based decision-making in healthy adult humans during foraging, reinforcement learning, and information search. Eleven fMRI studies met inclusion criterion for this review. Adopting a network neuroscience framework, synthesis of the findings across these studies revealed that exploration-based choice was associated with the engagement of attentional, control, and salience networks. In contrast, exploitation-based choice was associated with engagement of default network brain regions. We interpret these results in the context of a network architecture that supports the flexible switching between externally and internally directed cognitive processes, necessary for adaptive, goal-directed behaviour. To further investigate potential neural mechanisms underlying the exploration-exploitation trade-off we next surveyed studies involving neurodevelopmental, neuropsychological, and neuropsychiatric disorders, as well as lifespan development, and neurodegenerative diseases. We observed striking differences in patterns of explore-exploit decision-making across these populations, again suggesting that these two decision-making modes are supported by independent neural circuits. Taken together, our review highlights the need for precision-mapping of the neural circuitry and behavioural correlates associated with exploration and exploitation in humans. Characterizing exploration versus exploitation decision-making biases may offer a novel, trans-diagnostic approach to assessment, surveillance, and intervention for cognitive decline and dysfunction in normal development and clinical populations.

Information foraging with an oracle

During ecological decisions, such as when foraging for food or selecting a weekend activity, we often have to balance the costs and benefits of exploiting known options versus exploring novel ones. Here, we ask how individuals address such cost-benefit tradeoffs during tasks in which we can either explore by ourselves or seek external advice from an oracle (e.g., a domain expert or recommendation system). To answer this question, we designed two studies in which participants chose between inquiring (at a cost) for expert advice from an oracle, or to search for options without guidance, under manipulations affecting the optimal choice. We found that participants showed a greater propensity to seek expert advice when it was instrumental to increase payoff (study A), and when it reduced choice uncertainty, above and beyond payoff maximization (study B). This latter result was especially apparent in participants with greater trait-level intolerance of uncertainty. Taken together, these results suggest that we seek expert advice for both economic goals (i.e., payoff maximization) and epistemic goals (i.e., uncertainty minimization) and that our decisions to ask or not ask for advice are sensitive to cost-benefit tradeoffs.

Evidence for optimal semantic search throughout adulthood

As people age, they learn and store new knowledge in their semantic memory. Despite learning a tremendous amount of information, people can still recall information relevant to the current situation with ease. To accomplish this, the mind must efficiently organize and search a vast store of information. It also must continue to retrieve information effectively despite changes in cognitive mechanisms due to healthy aging, including a general slowing in information processing and a decline in executive functioning. How effectively does the mind of an individual adjust its search to account for changes due to aging? We tested 746 people ages 25 through 69 on a semantic fluency task (free listing animals) and found that, on average, retrieval follows an optimal path through semantic memory. Participants tended to list a sequence of semantically related animals (e.g., lion, tiger, puma) before switching to a semantically unrelated animal (e.g., whale). We found that the timing of these transitions to semantically unrelated animals was remarkably consistent with an optimal strategy for maximizing the overall rate of retrieval (i.e., the number of animals listed per unit time). Age did not affect an individual's deviation from the optimal strategy given their general performance, suggesting that people adapt and continue to search memory optimally throughout their lives. We argue that this result is more likely due to compensating for a general slowing than a decline in executive functioning.

Effort Foraging Task reveals positive correlation between individual differences in the cost of cognitive and physical effort in humans

Effort-based decisions, in which people weigh potential future rewards against effort costs required to achieve those rewards involve both cognitive and physical effort, though the mechanistic relationship between them is not yet understood. Here, we use an individual differences approach to isolate and measure the computational processes underlying effort-based decisions and test the association between cognitive and physical domains. Patch foraging is an ecologically valid reward rate maximization problem with well-developed theoretical tools. We developed the Effort Foraging Task, which embedded cognitive or physical effort into patch foraging, to quantify the cost of both cognitive and physical effort indirectly, by their effects on foraging choices. Participants chose between harvesting a depleting patch, or traveling to a new patch that was costly in time and effort. Participants' exit thresholds (reflecting the reward they expected to receive by harvesting when they chose to travel to a new patch) were sensitive to cognitive and physical effort demands, allowing us to quantify the perceived effort cost in monetary terms. The indirect sequential choice style revealed effort-seeking behavior in a minority of participants (preferring high over low effort) that has apparently been missed by many previous approaches. Individual differences in cognitive and physical effort costs were positively correlated, suggesting that these are perceived and processed in common. We used canonical correlation analysis to probe the relationship of task measures to self-reported affect and motivation, and found correlations of cognitive effort with anxiety, cognitive function, behavioral activation, and self-efficacy, but no similar correlations with physical effort.

Body appearance values modulate risk aversion in eating restriction

The understanding of eating disorders is hindered by the lack of integration between existing psychosocial and neurobiological approaches. We address this problem by developing a novel transdiagnostic and computational approach to eating restriction decisions. We first validated a novel paradigm which extends an established monetary risk task to involve body stimuli with psychosocial values. We used advanced behavioral data analysis of a large (total N = 539) sample of women from across the eating restraint spectrum, including those with anorexia nervosa (AN; n = 31), recovered from AN (n = 23), and subclinical women with varying levels of eating restraint (n = 485), obtained from an online experiment, public event, and laboratory-based study. We found that social and motivational values regarding body appearance have a significant effect on value-based, decision making in eating restriction. Subsequently, validated descriptive and predictive advanced computational modeling indicated that these behaviors are driven by an aversion to risk rather than loss, with desirable body outcomes being associated with less risk aversion, and undesirable body outcomes linked to greater risk aversion. These findings indicate that cognitive and social factors influence eating decisions by distinct mechanisms. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Efficiency traps beyond the climate crisis: exploration-exploitation trade-offs and rebound effects

Higher levels of economic activity are often accompanied by higher energy use and consumption of natural resources. As fossil fuels still account for 80% of the global energy mix, energy consumption remains closely linked to greenhouse gas (GHG) emissions and thus to climate change. Under the assumption of sufficiently elastic demand, this reality of global economic development based on permanent growth of economic activity, brings into play the Jevons Paradox, which hypothesises that increases in the efficiency of resource use leads to increases in resource consumption. Previous research on the rebound effects has limitations, including a lack of studies on the connection between reinforcement learning and environmental consequences. This paper develops a mathematical model and computer simulator to study the effects of micro-level exploration-exploitation strategies on efficiency, consumption and sustainability, considering different levels of direct and indirect rebound effects. Our model shows how optimal exploration-exploitation strategies for increasing efficiency can lead to unsustainable development patterns if they are not accompanied by demand reduction measures, which are essential for mitigating climate change. Moreover, our paper speaks to the broader issue of efficiency traps by highlighting how indirect rebound effects not only affect primary energy (PE) consumption and GHG emissions, but also resource consumption in other domains. By linking these issues together, our study sheds light on the complexities and interdependencies involved in achieving sustainable development goals. This article is part of the theme issue 'Climate change adaptation needs a science of culture'.

Cognitive Archeology and the Attentional System: An Evolutionary Mismatch for the Genus Homo

Brain evolution is a key topic in evolutionary anthropology. Unfortunately, in this sense the fossil record can usually support limited anatomical and behavioral inferences. Nonetheless, information from fossil species is, in any case, particularly valuable, because it represents the only direct proof of cerebral and behavioral changes throughout the human phylogeny. Recently, archeology and psychology have been integrated in the field of cognitive archeology, which aims to interpret current cognitive models according to the evidence we have on extinct human species. In this article, such evidence is reviewed in order to consider whether and to what extent the archeological record can supply information regarding changes of the attentional system in different taxa of the human genus. In particular, behavioral correlates associated with the fronto-parietal system and working memory are employed to consider recent changes in our species, Homo sapiens, and a mismatch between attentional and visuospatial ability is hypothesized. These two functional systems support present-moment awareness and mind-wandering, respectively, and their evolutionary unbalance can explain a structural sensitivity to psychological distress in our species.

Mnemicity: A Cognitive Gadget?

Episodic representations can be entertained either as "remembered" or "imagined"-as outcomes of experience or as simulations of such experience. Here, we argue that this feature is the product of a dedicated cognitive function: the metacognitive capacity to determine the mnemicity of mental event simulations. We argue that mnemicity attribution should be distinguished from other metacognitive operations (such as reality monitoring) and propose that this attribution is a "cognitive gadget"-a distinctively human ability made possible by cultural learning. Cultural learning is a type of social learning in which traits are inherited through social interaction. In the case of mnemicity, one culturally learns to discriminate metacognitive "feelings of remembering" from other perceptual, emotional, action-related, and metacognitive feelings; to interpret feelings of remembering as indicators of memory rather than imagination; and to broadcast the interpreted feelings in culture- and context-specific ways, such as "I was there" or "I witnessed it myself." We review evidence from the literature on memory development and scaffolding, metacognitive learning and teaching, as well as cross-cultural psychology in support of this view before pointing out various open questions about the nature and development of mnemicity highlighted by our account.

A shared novelty-seeking basis for creativity and curiosity

Curiosity and creativity are central pillars of human growth and invention. Although they have been studied extensively in isolation, the relationship between them has not yet been established. We propose that both curiosity and creativity emanate from the same mechanism of novelty seeking. We first present a synthesis showing that curiosity and creativity are affected similarly by a number of key cognitive faculties such as memory, cognitive control, attention, and reward. We then review empirical evidence from neuroscience research, indicating that the same brain regions are involved in both curiosity and creativity, focusing on the interplay between three major brain networks: the default mode network, the salience network, and the executive control network. After substantiating the link between curiosity and creativity, we propose a novelty-seeking model (NSM) that underlies them and suggests that the manifestation of the NSM is governed by one's state of mind.

Dynamic pathogen detection and social feedback shape collective hygiene in ants

Cooperative disease defense emerges as group-level collective behavior, yet how group members make the underlying individual decisions is poorly understood. Using garden ants and fungal pathogens as an experimental model, we derive the rules governing individual ant grooming choices and show how they produce colony-level hygiene. Time-resolved behavioral analysis, pathogen quantification, and probabilistic modeling reveal that ants increase grooming and preferentially target highly-infectious individuals when perceiving high pathogen load, but transiently suppress grooming after having been groomed by nestmates. Ants thus react to both, the infectivity of others and the social feedback they receive on their own contagiousness. While inferred solely from momentary ant decisions, these behavioral rules quantitatively predict hour-long experimental dynamics, and synergistically combine into efficient colony-wide pathogen removal. Our analyses show that noisy individual decisions based on only local, incomplete, yet dynamically-updated information on pathogen threat and social feedback can lead to potent collective disease defense.

Age effects in second language acquisition: Expanding the emergentist account

In 2005, Science magazine designated the problem of accounting for difficulties in L2 (second language) learning as one of the 125 outstanding challenges facing scientific research. A maturationally-based sensitive period has long been the favorite explanation for why ultimate foreign language attainment declines with age-of-acquisition. However, no genetic or neurobiological mechanisms for limiting language learning have yet been identified. At the same time, we know that cognitive, social, and motivational factors change in complex ways across the human lifespan. Emergentist theory provides a framework for relating these changes to variation in the success of L2 learning. The great variability in patterns of learning, attainment, and loss across ages, social groups, and linguistic levels provides the core motivation for the emergentist approach. Our synthesis incorporates three groups of factors which change systematically with age: environmental supports, cognitive abilities, and motivation for language learning. This extended emergentist account explains why and when second language succeeds for some children and adults and fails for others.

Exploratory preferences explain the human fascination for imaginary worlds in fictional stories

Imaginary worlds are present and often central in many of the most culturally successful modern narrative fictions, be it in novels (e.g., Harry Potter), movies (e.g., Star Wars), video games (e.g., The Legend of Zelda), graphic novels (e.g., One Piece) and TV series (e.g., Game of Thrones). We propose that imaginary worlds are popular because they activate exploratory preferences that evolved to help us navigate the real world and find new fitness-relevant information. Therefore, we hypothesize that the attraction to imaginary worlds is intrinsically linked to the desire to explore novel environments and that both are influenced by the same underlying factors. Notably, the inter-individual and cross-cultural variability of the preference for imaginary worlds should follow the inter-individual and cross-cultural variability of exploratory preferences (with the personality trait Openness-to-experience, age, sex, and ecological conditions). We test these predictions with both experimental and computational methods. For experimental tests, we run a pre-registered online experiment about movie preferences (N = 230). For computational tests, we leverage two large cultural datasets, namely the Internet Movie Database (N = 9424 movies) and the Movie Personality Dataset (N = 3.5 million participants), and use machine-learning algorithms (i.e., random forest and topic modeling). In all, consistent with how the human preference for spatial exploration adaptively varies, we provide empirical evidence that imaginary worlds appeal more to more explorative people, people higher in Openness-to-experience, younger individuals, males, and individuals living in more affluent environments. We discuss the implications of these findings for our understanding of the cultural evolution of narrative fiction and, more broadly, the evolution of human exploratory preferences.

Early adversity and the development of explore-exploit tradeoffs

Childhood adversity can have wide-ranging and long-lasting effects on later life. But what are the mechanisms that are responsible for these effects? This article brings together the cognitive science literature on explore-exploit tradeoffs, the empirical literature on early adversity, and the literature in evolutionary biology on 'life history' to explain how early experience influences later life. We propose one potential mechanism: early experiences influence 'hyperparameters' that determine the balance between exploration and exploitation. Adversity might accelerate a shift from exploration to exploitation, with broad and enduring effects on the adult brain and mind. These effects may be produced by life-history adaptations that use early experience to tailor development and learning to the likely future states of an organism and its environment.

Maladaptive explore/exploit trade-offs in schizophrenia

Schizophrenia is a complex disorder that remains poorly understood, particularly at the systems level. In this opinion article we argue that the explore/exploit trade-off concept provides a holistic and ecologically valid framework to resolve some of the apparent paradoxes that have emerged within schizophrenia research. We review recent evidence suggesting that fundamental explore/exploit behaviors may be maladaptive in schizophrenia during physical, visual, and cognitive foraging. We also describe how theories from the broader optimal foraging literature, such as the marginal value theorem (MVT), could provide valuable insight into how aberrant processing of reward, context, and cost/effort evaluations interact to produce maladaptive responses.

Beyond collective intelligence: Collective adaptation

We develop a conceptual framework for studying collective adaptation in complex socio-cognitive systems, driven by dynamic interactions of social integration strategies, social environments and problem structures. Going beyond searching for 'intelligent' collectives, we integrate research from different disciplines and outline modelling approaches that can be used to begin answering questions such as why collectives sometimes fail to reach seemingly obvious solutions, how they change their strategies and network structures in response to different problems and how we can anticipate and perhaps change future harmful societal trajectories. We discuss the importance of considering path dependence, lack of optimization and collective myopia to understand the sometimes counterintuitive outcomes of collective adaptation. We call for a transdisciplinary, quantitative and societally useful social science that can help us to understand our rapidly changing and ever more complex societies, avoid collective disasters and reach the full potential of our ability to organize in adaptive collectives.

Pathologist pupil dilation reflects experience level and difficulty in diagnosing medical images

Purpose: Digital whole slide imaging allows pathologists to view slides on a computer screen instead of under a microscope. Digital viewing allows for real-time monitoring of pathologists' search behavior and neurophysiological responses during the diagnostic process. One particular neurophysiological measure, pupil diameter, could provide a basis for evaluating clinical competence during training or developing tools that support the diagnostic process. Prior research shows that pupil diameter is sensitive to cognitive load and arousal, and it switches between exploration and exploitation of a visual image. Different categories of lesions in pathology pose different levels of challenge, as indicated by diagnostic disagreement among pathologists. If pupil diameter is sensitive to the perceived difficulty in diagnosing biopsies, eye-tracking could potentially be used to identify biopsies that may benefit from a second opinion. Approach: We measured case onset baseline-corrected (phasic) and uncorrected (tonic) pupil diameter in 90 pathologists who each viewed and diagnosed 14 digital breast biopsy cases that cover the diagnostic spectrum from benign to invasive breast cancer. Pupil data were extracted from the beginning of viewing and interpreting of each individual case. After removing 122 trials ( < 10 % ) with poor eye-tracking quality, 1138 trials remained. We used multiple linear regression with robust standard error estimates to account for dependent observations within pathologists. Results: We found a positive association between the magnitude of phasic dilation and subject-centered difficulty ratings and between the magnitude of tonic dilation and untransformed difficulty ratings. When controlling for case diagnostic category, only the tonic-difficulty relationship persisted. Conclusions: Results suggest that tonic pupil dilation may indicate overall arousal differences between pathologists as they interpret biopsy cases and could signal a need for additional training, experience, or automated decision aids. Phasic dilation is sensitive to characteristics of biopsies that tend to elicit higher difficulty ratings and could indicate a need for a second opinion.

Replay and compositional computation

Replay in the brain has been viewed as rehearsal or, more recently, as sampling from a transition model. Here, we propose a new hypothesis: that replay is able to implement a form of compositional computation where entities are assembled into relationally bound structures to derive qualitatively new knowledge. This idea builds on recent advances in neuroscience, which indicate that the hippocampus flexibly binds objects to generalizable roles and that replay strings these role-bound objects into compound statements. We suggest experiments to test our hypothesis, and we end by noting the implications for AI systems which lack the human ability to radically generalize past experience to solve new problems.

On the social nature of abstraction: cognitive implications of interaction and diversity

The human capacity for abstraction is remarkable. We effortlessly form abstract representations from varied experiences, generalizing and flexibly transferring experiences and knowledge between contexts, which can facilitate reasoning, problem solving and learning across many domains. The cognitive process of abstraction, however, is often portrayed and investigated as an individual process. This paper addresses how cognitive processes of abstraction-together with other aspects of human reasoning and problem solving-are fundamentally shaped and modulated by online social interaction. Starting from a general distinction between convergent thinking, divergent thinking and processes of abstraction, we address how social interaction shapes information processing differently depending on cognitive demands, social coordination and task ecologies. In particular, we suggest that processes of abstraction are facilitated by the interactive sharing and integration of varied individual experiences. To this end, we also discuss how the dynamics of group interactions vary as a function of group composition; that is, in terms of the similarity and diversity between the group members. We conclude by outlining the role of cognitive diversity in interactive processes and consider the importance of group diversity in processes of abstraction. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.

Sampling statistics are like story creation: a network analysis of parent-toddler exploratory play

Actions in the world elicit data for learning and do so in a stream of interconnected events. Here, we provide evidence on how toddlers with their parent sample information by acting on toys during exploratory play. We observed 10 min of free-flowing and unconstrained object exploration of by toddlers (mean age 21 months) and parents in a room with many available objects (n = 32). Borrowing concepts and measures from the study of narratives, we found that the toy selections are not a string of unrelated events but exhibit a suite of what we call coherence statistics: Zipfian distributions, burstiness and a network structure. We discuss the transient memory processes that underlie the moment-to-moment toy selections that create this coherence and the role of these statistics in the development of abstract and generalizable systems of knowledge. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.

Narrowing the coordination solution space during motor learning standardizes individual patterns of search strategy but diversifies learning rates

Constraints on practice can benefit motor learning by guiding the learner towards efficient coordination patterns, but can also narrow the potential solution space of coordination and control. The aim of this paper was to investigate whether narrowing the solution space through more restrictive task constraints limits the expression of potential exploratory behaviours during the learning process, identified using Drifting Markov Models. In a breaststroke swimming task, the change in interlimb coordination of 7 learners practicing for 16 lessons over 2 months was analysed to quantify motor exploration and identify periods of metastable regimes of coordination. Results showed that the observed exploratory dynamics were highly individual both in terms of range of exploration and in the patterns of search. The more restrictive task constraints did not impair the amount of exploration but rather channelled the exploration around a few selected patterns. In addition, restraining the nature of the exploratory process increased the inter-individual differences of the learning rate. Although manipulating the task constraints during learning can help learners to escape from the behavioural consequences of their intrinsic dynamics, maintaining a broad solution space for a diversity of coordination patterns to emerge was key to fostering effective exploration of individual coordination solutions.

The value of spatial experience and group size for ant colonies in direct competition

Animals often search for food more efficiently with experience. However, the contribution of experience to foraging success under direct competition has rarely been examined. Here we used colonies of an individually foraging desert ant to investigate the value of spatial experience. First, we trained worker groups of equal numbers to solve either a complex or a simple maze. We then tested pairs of both groups against one another in reaching a food reward. This task required solving the same complex maze that one of the groups had been trained in, to determine which group would exploit better the food reward. The worker groups previously trained in the complex mazes reached the food reward faster and more of these workers fed on the food than those trained in simple mazes, but only in the intermediate size group. To determine the relative importance of group size versus spatial experience in exploiting food patches, we then tested smaller trained worker groups against larger untrained ones. The larger groups outcompeted the smaller ones, despite the latter's advantage of spatial experience. The contribution of spatial experience, as found here, appears to be small, and depends on group size: an advantage of a few workers of the untrained group over the trained group negates its benefits.

Microfoundations of Adaptive Search in Complex Tasks: The Role of Cognitive Abilities and Styles

Problem-solving in complex environments requires a cognitively demanding search for task solutions. Managing this search process presents a major challenge in organizations. We contribute to the literature on this topic by providing new evidence on the cognitive antecedents that shape how individuals search when engaged in complex problem-solving tasks. We present results from three laboratory studies, wherein 335 individuals solved a complex task. In doing so, they generated behavioral data coupled with survey-based measurements of the individuals’ cognitive styles and performance-based tests of their cognitive abilities. Our data analysis contributes to the current literature by documenting systematic heterogeneity in the persistence and distance of search that can be explained by the participants’ level of creativity, attention to detail, and executive functions. We extend the research on the microfoundations of adaptive search by linking cognitive antecedents with a complex search task, widening our insight into what search behavior certain cognitive microfoundations lead to, and showing how managers can more effectively shape organizational search.

History: This paper has been accepted for the Organization Science Special Issue on Experiments in Organizational Theory.

Supplemental Material: The online appendix is available at https://doi.org/10.1287/orsc.2023.1654 .

Comparing rapid rule-learning strategies in humans and monkeys

Inter-species comparisons are key to deriving an understanding of the behavioral and neural correlates of human cognition from animal models. We perform a detailed comparison of macaque monkey and human strategies on an analogue of the Wisconsin Card Sort Test, a widely studied and applied multi-attribute measure of cognitive function, wherein performance requires the inference of a changing rule given ambiguous feedback. We found that well-trained monkeys rapidly infer rules but are three times slower than humans. Model fits to their choices revealed hidden states akin to feature-based attention in both species, and decision processes that resembled a Win-stay lose-shift strategy with key differences. Monkeys and humans test multiple rule hypotheses over a series of rule-search trials and perform inference-like computations to exclude candidates. An attention-set based learning stage categorization revealed that perseveration, random exploration and poor sensitivity to negative feedback explain the under-performance in monkeys.

Social Search and Resource Clustering as Emergent Stable States

Social search has stably evolved across various species and is often used by humans to search for resources (such as food, information, social partners). In turn, these resources frequently come distributed in patches or clusters. In the current work, we use an ecologically inspired agent-based model to investigate whether social search and clustering are stable outcomes of the dynamical mutual interactions between the two. While previous research has studied unidirectional influences of social search on resource clustering and vice versa, the current work investigates the consequential patterns emerging from their two-way interactions over time. In our model, consumers evolved search strategies (ranging from competitive to social) as adaptations to their environmental resource structures, and resources varied in distributions (ranging from random to clustered) that were shaped by agents' consumption patterns. Across four experiments, we systematically analyzed the patterns of influence that search strategies and environment structure have on each other to identify stable attractor states of both. In Experiment 1, we fixed resource clustering at various levels and observed its influence on social search, and in Experiment 2, we observed the influence of social search on resource distribution. In both these experiments we found that increasing levels of one variable produced increases in the other; however, at very high levels of the manipulated variable, the dependent variable tended to fall. Finally in Experiments 3 and 4, we studied the dynamics that arose when resource clustering and social search could both change and mutually influence each other, finding that low levels of social search and clustering were stable attractor states. Our simple 2D model yielded results that qualitatively resemble those across a wide range of search domains (from physical search for food to abstract search for information), highlighting some stable outcomes of mutually interacting consumer/resource systems.

Cognitive networks detect structural patterns and emotional complexity in suicide notes

Communicating one's mindset means transmitting complex relationships between concepts and emotions. Using network science and word co-occurrences, we reconstruct conceptual associations as communicated in 139 genuine suicide notes, i.e., notes left by individuals who took their lives. We find that, despite their negative context, suicide notes are surprisingly positively valenced. Through emotional profiling, their ending statements are found to be markedly more emotional than their main body: The ending sentences in suicide notes elicit deeper fear/sadness but also stronger joy/trust and anticipation than the main body. Furthermore, by using data from the Emotional Recall Task, we model emotional transitions within these notes as co-occurrence networks and compare their structure against emotional recalls from mentally healthy individuals. Supported by psychological literature, we introduce emotional complexity as an affective analog of structural balance theory, measuring how elementary cycles (closed triads) of emotion co-occurrences mix positive, negative and neutral states in narratives and recollections. At the group level, authors of suicide narratives display a higher complexity than healthy individuals, i.e., lower levels of coherently valenced emotional states in triads. An entropy measure identified a similar tendency for suicide notes to shift more frequently between contrasting emotional states. Both the groups of authors of suicide notes and healthy individuals exhibit less complexity than random expectation. Our results demonstrate that suicide notes possess highly structured and contrastive narratives of emotions, more complex than expected by null models and healthy populations.

Compositional Sequence Generation in the Entorhinal-Hippocampal System

Neurons in the medial entorhinal cortex exhibit multiple, periodically organized, firing fields which collectively appear to form an internal representation of space. Neuroimaging data suggest that this grid coding is also present in other cortical areas such as the prefrontal cortex, indicating that it may be a general principle of neural functionality in the brain. In a recent analysis through the lens of dynamical systems theory, we showed how grid coding can lead to the generation of a diversity of empirically observed sequential reactivations of hippocampal place cells corresponding to traversals of cognitive maps. Here, we extend this sequence generation model by describing how the synthesis of multiple dynamical systems can support compositional cognitive computations. To empirically validate the model, we simulate two experiments demonstrating compositionality in space or in time during sequence generation. Finally, we describe several neural network architectures supporting various types of compositionality based on grid coding and highlight connections to recent work in machine learning leveraging analogous techniques.

Balancing novelty and appropriateness leads to creative associations in children

Creative problem solving is a fundamental skill of human cognition and is conceived as a search process whereby a novel and appropriate solution is generated. However, it is unclear whether children are able to balance novelty and appropriateness to generate creative solutions and what are the underlying computational mechanisms. Here, we asked children, ranging from 10 to 11 years old, to perform a word association task according to three instructions, which triggered a more appropriate (ordinary), novel (random), or balanced (creative) response. Results revealed that children exhibited greater cognitive flexibility in the creative condition compared to the control conditions, as revealed by the structure and resiliency of the semantic networks. Moreover, responses' word embeddings extracted from pretrained deep neural networks showed that semantic distance and category switching index increased in the creative condition with respect to the ordinary condition and decreased compared to the random condition. Critically, we showed how children efficiently solved the exploration/exploitation trade-off to generate creative associations by fitting a computational reinforcement learning (RL) model that simulates semantic search strategies. Our findings provide compelling evidence that children balance novelty and appropriateness to generate creative associations by optimally regulating the level of exploration in the semantic search. This corroborates previous findings on the adult population and highlights the crucial contribution of both components to the overall creative process. In conclusion, these results shed light on the connections between theoretical concepts such as bottom-up/top-down modes of thinking in creativity research and the exploration/exploitation trade-off in human RL research.

Embodied and embedded ecological rationality: A common vertebrate mechanism for action selection underlies cognition and heuristic decision-making in humans

The last common ancestor shared by humans and other vertebrates lived over half a billion years ago. In the time since that ancestral line diverged, evolution by natural selection has produced an impressive diversity—from fish to birds to elephants—of vertebrate morphology; yet despite the great species-level differences that otherwise exist across the brains of many animals, the neural circuitry that underlies motor control features a functional architecture that is virtually unchanged in every living species of vertebrate. In this article, we review how that circuitry facilitates motor control, trial-and-error-based procedural learning, and habit formation; we then develop a model that describes how this circuitry (embodied in an agent) works to build and refine sequences of goal-directed actions that are molded to fit the structure of the environment (in which the agent is embedded). We subsequently review evidence suggesting that this same functional circuitry became further adapted to regulate cognitive control in humans as well as motor control; then, using examples of heuristic decision-making from the ecological rationality tradition, we show how the model can be used to understand how that circuitry operates analogously in both cognitive and motor domains. We conclude with a discussion of how the model encourages a shift in perspective regarding ecological rationality’s “adaptive toolbox”—namely, to one that views heuristic processes and other forms of goal-directed cognition as likely being implemented by the same neural circuitry (and in the same fashion) as goal-directed action in the motor domain—and how this change of perspective can be useful.

Habituation Reflects Optimal Exploration Over Noisy Perceptual Samples

From birth, humans constantly make decisions about what to look at and for how long. Yet, the mechanism behind such decision-making remains poorly understood. Here, we present the rational action, noisy choice for habituation (RANCH) model. RANCH is a rational learning model that takes noisy perceptual samples from stimuli and makes sampling decisions based on expected information gain (EIG). The model captures key patterns of looking time documented in developmental research: habituation and dishabituation. We evaluated the model with adult looking time collected from a paradigm analogous to the infant habituation paradigm. We compared RANCH with baseline models (no learning model, no perceptual noise model) and models with alternative linking hypotheses (Surprisal, KL divergence). We showed that (1) learning and perceptual noise are critical assumptions of the model, and (2) Surprisal and KL are good proxies for EIG under the current learning context.

The network science of collective intelligence

In the last few years, breakthroughs in computational and experimental techniques have produced several key discoveries in the science of networks and human collective intelligence. This review presents the latest scientific findings from two key fields of research: collective problem-solving and the wisdom of the crowd. I demonstrate the core theoretical tensions separating these research traditions and show how recent findings offer a new synthesis for understanding how network dynamics alter collective intelligence, both positively and negatively. I conclude by highlighting current theoretical problems at the forefront of research on networked collective intelligence, as well as vital public policy challenges that require new research efforts.