The central executive as a search process: priming exploration and exploitation across domains

Divergent and Convergent Creativity Are Different Kinds of Foraging

According to accounts of neural reuse and embodied cognition, higher-level cognitive abilities recycle evolutionarily ancient mechanisms for perception and action. Here, building on these accounts, we investigate whether creativity builds on our capacity to forage in space ("creativity as strategic foraging"). We report systematic connections between specific forms of creative thinking-divergent and convergent-and corresponding strategies for searching in space. U.S. American adults completed two tasks designed to measure creativity. Before each creativity trial, participants completed an unrelated search of a city map. Between subjects, we manipulated the search pattern, with some participants seeking multiple, dispersed spatial locations and others repeatedly converging on the same location. Participants who searched divergently in space were better at divergent thinking but worse at convergent thinking; this pattern reversed for participants who had converged repeatedly on a single location. These results demonstrate a targeted link between foraging and creativity, thus advancing our understanding of the origins and mechanisms of high-level cognition.

The Spread of Beliefs in Partially Modularized Communities

Many life-influencing social networks are characterized by considerable informational isolation. People within a community are far more likely to share beliefs than people who are part of different communities. The spread of useful information across communities is impeded by echo chambers (far greater connectivity within than between communities) and filter bubbles (more influence of beliefs by connected neighbors within than between communities). We apply the tools of network analysis to organize our understanding of the spread of beliefs across modularized communities and to predict the effect of individual and group parameters on the dynamics and distribution of beliefs. In our Spread of Beliefs in Modularized Communities (SBMC) framework, a stochastic block model generates social networks with variable degrees of modularity, beliefs have different observable utilities, individuals change their beliefs on the basis of summed or average evidence (or intermediate decision rules), and parameterized stochasticity introduces randomness into decisions. SBMC simulations show surprising patterns; for example, increasing out-group connectivity does not always improve group performance, adding randomness to decisions can promote performance, and decision rules that sum rather than average evidence can improve group performance, as measured by the average utility of beliefs that the agents adopt. Overall, the results suggest that intermediate degrees of belief exploration are beneficial for the spread of useful beliefs in a community, and so parameters that pull in opposite directions on an explore-exploit continuum are usefully paired.

Attention deficits linked with proclivity to explore while foraging

All mobile organisms forage for resources, choosing how and when to search for new opportunities by comparing current returns with the average for the environment. In humans, nomadic lifestyles favouring exploration have been associated with genetic mutations implicated in attention deficit hyperactivity disorder (ADHD), inviting the hypothesis that this condition may impact foraging decisions in the general population. Here we tested this pre-registered hypothesis by examining how human participants collected resources in an online foraging task. On every trial, participants chose either to continue to collect rewards from a depleting patch of resources or to replenish the patch. Participants also completed a well-validated ADHD self-report screening assessment at the end of sessions. Participants departed resource patches sooner when travel times between patches were shorter than when they were longer, as predicted by optimal foraging theory. Participants whose scores on the ADHD scale crossed the threshold for a positive screen departed patches significantly sooner than participants who did not meet this criterion. Participants meeting this threshold for ADHD also achieved higher reward rates than individuals who did not. Our findings suggest that ADHD attributes may confer foraging advantages in some environments and invite the possibility that this condition may reflect an adaptation favouring exploration over exploitation.

The journey within: mental navigation as a novel framework for understanding psychotherapeutic transformation

BACKGROUND

Despite the demonstrated efficacy of psychotherapy, the precise mechanisms that drive therapeutic transformations have posed a challenge and still remain unresolved. Here, we suggest a potential solution to this problem by introducing a framework based on the concept of mental navigation. It refers to our ability to navigate our cognitive space of thoughts, ideas, concepts, and memories, similar to how we navigate physical space. We start by analyzing the neural, cognitive, and experiential constituents intrinsic to mental navigation. Subsequently, we posit that the metaphoric spatial language we employ to articulate introspective experiences (e.g., "unexplored territory" or "going in circles") serves as a robust marker of mental navigation.

METHODS

Using large text corpora, we compared the utilization of spatial language between transcripts of psychotherapy sessions (≈ 12 M. words), casual everyday conversations (≈ 12 M. words), and fictional dialogues in movies (≈ 14 M. words). We also examined 110 psychotherapy transcripts qualitatively to discern patterns and dynamics associated with mental navigation.

RESULTS

We found a notable increase in the utilization of spatial metaphors during psychotherapy compared to casual everyday dialogues (U = 192.0, p = .001, d = 0.549) and fictional conversations (U = 211, p < .001, d = 0.792). In turn, analyzing the usage of non-spatial metaphors, we did not find significant differences between the three datasets (H = 0.682, p = 0.710). The qualitative analysis highlighted specific examples of mental navigation at play.

CONCLUSION

Mental navigation might underlie the psychotherapy process and serve as a robust framework for understanding the transformative changes it brings about.

Understanding patch foraging strategies across development

Patch foraging is a near-ubiquitous behaviour across the animal kingdom and characterises many decision-making domains encountered by humans. We review how a disposition to explore in adolescence may reflect the evolutionary conditions under which hunter-gatherers foraged for resources. We propose that neurocomputational mechanisms responsible for reward processing, learning, and cognitive control facilitate the transition from exploratory strategies in adolescence to exploitative strategies in adulthood - where individuals capitalise on known resources. This developmental transition may be disrupted by psychopathology, as there is emerging evidence of biases in explore/exploit choices in mental health problems. Explore/exploit choices may be an informative marker for mental health across development and future research should consider this feature of decision-making as a target for clinical intervention.

Information gathering explains decision dynamics during human and monkey reward foraging

Foraging in humans and other animals requires a delicate balance between exploitation of current resources and exploration for new ones. The tendency to overharvest-lingering too long in depleting patches-is a routine behavioral deviation from predictions of optimal foraging theories. To characterize the computational mechanisms driving these deviations, we modeled foraging behavior using a virtual patch-leaving task with human participants and validated our findings in an analogous foraging task in two monkeys. Both humans and monkeys overharvested and stayed longer in patches with longer travel times compared to shorter ones. Critically, patch residence times in both species declined over the course of sessions, enhancing reward rates in humans. These decisions were best explained by a logistic transformation that integrated both current rewards and information about declining rewards. This parsimonious model demystifies both the occurrence and dynamics of overharvesting, highlighting the role of information gathering in foraging. Our findings provide insight into computational mechanisms shaped by ubiquitous foraging dilemmas, underscoring how behavioral modeling can reveal underlying motivations of seemingly irrational decisions.

How Discrimination Gets Under the Skin: Biological Determinants of Discrimination Associated With Dysregulation of the Brain-Gut Microbiome System and Psychological Symptoms

BACKGROUND

Discrimination is associated with negative health outcomes as mediated in part by chronic stress, but a full understanding of the biological pathways is lacking. Here we investigate the effects of discrimination involved in dysregulating the brain-gut microbiome (BGM) system.

METHODS

A total of 154 participants underwent brain magnetic resonance imaging to measure functional connectivity. Fecal samples were obtained for 16S ribosomal RNA profiling and fecal metabolites and serum for inflammatory markers, along with questionnaires. The Everyday Discrimination Scale was administered to measure chronic and routine experiences of unfair treatment. A sparse partial least squares-discriminant analysis was conducted to predict BGM alterations as a function of discrimination, controlling for sex, age, body mass index, and diet. Associations between discrimination-related BGM alterations and psychological variables were assessed using a tripartite analysis.

RESULTS

Discrimination was associated with anxiety, depression, and visceral sensitivity. Discrimination was associated with alterations of brain networks related to emotion, cognition and self-perception, and structural and functional changes in the gut microbiome. BGM discrimination-related associations varied by race/ethnicity. Among Black and Hispanic individuals, discrimination led to brain network changes consistent with psychological coping and increased systemic inflammation. For White individuals, discrimination was related to anxiety but not inflammation, while for Asian individuals, the patterns suggest possible somatization and behavioral (e.g., dietary) responses to discrimination.

CONCLUSIONS

Discrimination is attributed to changes in the BGM system more skewed toward inflammation, threat response, emotional arousal, and psychological symptoms. By integrating diverse lines of research, our results demonstrate evidence that may explain how discrimination contributes to health inequalities.

Exploratory preferences explain the human fascination for imaginary worlds in fictional stories

Imaginary worlds are present and often central in many of the most culturally successful modern narrative fictions, be it in novels (e.g., Harry Potter), movies (e.g., Star Wars), video games (e.g., The Legend of Zelda), graphic novels (e.g., One Piece) and TV series (e.g., Game of Thrones). We propose that imaginary worlds are popular because they activate exploratory preferences that evolved to help us navigate the real world and find new fitness-relevant information. Therefore, we hypothesize that the attraction to imaginary worlds is intrinsically linked to the desire to explore novel environments and that both are influenced by the same underlying factors. Notably, the inter-individual and cross-cultural variability of the preference for imaginary worlds should follow the inter-individual and cross-cultural variability of exploratory preferences (with the personality trait Openness-to-experience, age, sex, and ecological conditions). We test these predictions with both experimental and computational methods. For experimental tests, we run a pre-registered online experiment about movie preferences (N = 230). For computational tests, we leverage two large cultural datasets, namely the Internet Movie Database (N = 9424 movies) and the Movie Personality Dataset (N = 3.5 million participants), and use machine-learning algorithms (i.e., random forest and topic modeling). In all, consistent with how the human preference for spatial exploration adaptively varies, we provide empirical evidence that imaginary worlds appeal more to more explorative people, people higher in Openness-to-experience, younger individuals, males, and individuals living in more affluent environments. We discuss the implications of these findings for our understanding of the cultural evolution of narrative fiction and, more broadly, the evolution of human exploratory preferences.

Visuospatial information foraging describes search behavior in learning latent environmental features

In the real world, making sequences of decisions to achieve goals often depends upon the ability to learn aspects of the environment that are not directly perceptible. Learning these so-called latent features requires seeking information about them. Prior efforts to study latent feature learning often used single decisions, used few features, and failed to distinguish between reward-seeking and information-seeking. To overcome this, we designed a task in which humans and monkeys made a series of choices to search for shapes hidden on a grid. On our task, the effects of reward and information outcomes from uncovering parts of shapes could be disentangled. Members of both species adeptly learned the shapes and preferred to select tiles expected to be informative earlier in trials than previously rewarding ones, searching a part of the grid until their outcomes dropped below the average information outcome-a pattern consistent with foraging behavior. In addition, how quickly humans learned the shapes was predicted by how well their choice sequences matched the foraging pattern, revealing an unexpected connection between foraging and learning. This adaptive search for information may underlie the ability in humans and monkeys to learn latent features to support goal-directed behavior in the long run.

Humans trade off search costs and accuracy in a combined visual search and perceptual task

To interact with one's environment, relevant objects have to be selected as targets for saccadic eye movements. Previous studies have demonstrated that factors such as visual saliency and reward influence saccade target selection, and that humans can dynamically trade off these factors to maximize expected value during visual search. However, expected value in everyday situations not only depends on saliency and reward, but also on the required time to find objects, and the likelihood of a successful object-interaction after search. Here we studied whether search costs and the accuracy to discriminate an object feature can be traded off to maximize expected value. We designed a combined visual search and perceptual discrimination task, where participants chose whether to search for an easy- or difficult-to-discriminate target in search displays populated by distractors that shared features with either the easy or the difficult target. Participants received a monetary reward for correct discriminations and were given limited time to complete as many trials as they could. We found that participants considered their discrimination performance and the search costs when choosing targets and, by this, maximized expected value. However, the accumulated reward was constrained by noise in both the choice of which target to search for, and which elements to fixate during search. We conclude that humans take into account the prospective search time and the likelihood of successful a object-interaction, when deciding what to search for. However, search performance is constrained by noise in decisions about what to search for and how to search for it.

Integrated world modeling theory expanded: Implications for the future of consciousness

Integrated world modeling theory (IWMT) is a synthetic theory of consciousness that uses the free energy principle and active inference (FEP-AI) framework to combine insights from integrated information theory (IIT) and global neuronal workspace theory (GNWT). Here, I first review philosophical principles and neural systems contributing to IWMT's integrative perspective. I then go on to describe predictive processing models of brains and their connections to machine learning architectures, with particular emphasis on autoencoders (perceptual and active inference), turbo-codes (establishment of shared latent spaces for multi-modal integration and inferential synergy), and graph neural networks (spatial and somatic modeling and control). Future directions for IIT and GNWT are considered by exploring ways in which modules and workspaces may be evaluated as both complexes of integrated information and arenas for iterated Bayesian model selection. Based on these considerations, I suggest novel ways in which integrated information might be estimated using concepts from probabilistic graphical models, flow networks, and game theory. Mechanistic and computational principles are also considered with respect to the ongoing debate between IIT and GNWT regarding the physical substrates of different kinds of conscious and unconscious phenomena. I further explore how these ideas might relate to the "Bayesian blur problem," or how it is that a seemingly discrete experience can be generated from probabilistic modeling, with some consideration of analogies from quantum mechanics as potentially revealing different varieties of inferential dynamics. I go on to describe potential means of addressing critiques of causal structure theories based on network unfolding, and the seeming absurdity of conscious expander graphs (without cybernetic symbol grounding). Finally, I discuss future directions for work centered on attentional selection and the evolutionary origins of consciousness as facilitated "unlimited associative learning." While not quite solving the Hard problem, this article expands on IWMT as a unifying model of consciousness and the potential future evolution of minds.

Generalized Simultaneous Localization and Mapping (G-SLAM) as unification framework for natural and artificial intelligences: towards reverse engineering the hippocampal/entorhinal system and principles of high-level cognition

Simultaneous localization and mapping (SLAM) represents a fundamental problem for autonomous embodied systems, for which the hippocampal/entorhinal system (H/E-S) has been optimized over the course of evolution. We have developed a biologically-inspired SLAM architecture based on latent variable generative modeling within the Free Energy Principle and Active Inference (FEP-AI) framework, which affords flexible navigation and planning in mobile robots. We have primarily focused on attempting to reverse engineer H/E-S "design" properties, but here we consider ways in which SLAM principles from robotics may help us better understand nervous systems and emergent minds. After reviewing LatentSLAM and notable features of this control architecture, we consider how the H/E-S may realize these functional properties not only for physical navigation, but also with respect to high-level cognition understood as generalized simultaneous localization and mapping (G-SLAM). We focus on loop-closure, graph-relaxation, and node duplication as particularly impactful architectural features, suggesting these computational phenomena may contribute to understanding cognitive insight (as proto-causal-inference), accommodation (as integration into existing schemas), and assimilation (as category formation). All these operations can similarly be describable in terms of structure/category learning on multiple levels of abstraction. However, here we adopt an ecological rationality perspective, framing H/E-S functions as orchestrating SLAM processes within both concrete and abstract hypothesis spaces. In this navigation/search process, adaptive cognitive equilibration between assimilation and accommodation involves balancing tradeoffs between exploration and exploitation; this dynamic equilibrium may be near optimally realized in FEP-AI, wherein control systems governed by expected free energy objective functions naturally balance model simplicity and accuracy. With respect to structure learning, such a balance would involve constructing models and categories that are neither too inclusive nor exclusive. We propose these (generalized) SLAM phenomena may represent some of the most impactful sources of variation in cognition both within and between individuals, suggesting that modulators of H/E-S functioning may potentially illuminate their adaptive significances as fundamental cybernetic control parameters. Finally, we discuss how understanding H/E-S contributions to G-SLAM may provide a unifying framework for high-level cognition and its potential realization in artificial intelligences.

An instance-based model account of the benefits of varied practice in visuomotor skill

Exposing learners to variability during training has been demonstrated to improve performance in subsequent transfer testing. Such variability benefits are often accounted for by assuming that learners are developing some general task schema or structure. However much of this research has neglected to account for differences in similarity between varied and constant training conditions. In a between-groups manipulation, we trained participants on a simple projectile launching task, with either varied or constant conditions. We replicate previous findings showing a transfer advantage of varied over constant training. Furthermore, we show that a standard similarity model is insufficient to account for the benefits of variation, but, if the model is adjusted to assume that varied learners are tuned towards a broader generalization gradient, then a similarity-based model is sufficient to explain the observed benefits of variation. Our results therefore suggest that some variability benefits can be accommodated within instance-based models without positing the learning of some schemata or structure.

Developmental Dyslexia: Disorder or Specialization in Exploration?

We raise the new possibility that people diagnosed with developmental dyslexia (DD) are specialized in explorative cognitive search, and rather than having a neurocognitive disorder, play an essential role in human adaptation. Most DD research has studied educational difficulties, with theories framing differences in neurocognitive processes as deficits. However, people with DD are also often proposed to have certain strengths - particularly in realms like discovery, invention, and creativity - that deficit-centered theories cannot explain. We investigate whether these strengths reflect an underlying explorative specialization. We re-examine experimental studies in psychology and neuroscience using the framework of cognitive search, whereby many psychological processes involve a trade-off between exploration and exploitation. We report evidence of an explorative bias in DD-associated cognitive strategies. High DD prevalence and an attendant explorative bias across multiple areas of cognition suggest the existence of explorative specialization. An evolutionary perspective explains the combination of findings and challenges the view that individuals with DD have a disorder. In cooperating groups, individual specialization is favored when features that confer fitness benefits are functionally incompatible. Evidence for search specialization suggests that, as with some other social organisms, humans mediate the exploration-exploitation trade-off by specializing in complementary strategies. The existence of a system of collective cognitive search that emerges through collaboration would help to explain our species' exceptional adaptiveness. It also aligns with evidence for substantial variability during our evolutionary history and the notion that humans are adapted not to a particular habitat but to variability itself. Specialization creates interdependence and necessitates balancing complementary strategies. Reframing DD therefore underscores the urgency of changing certain cultural practices to ensure we do not inhibit adaptation. Key improvements would remove cultural barriers to exploration and nurture explorative learning in education, academia, and the workplace, as well as emphasize collaboration over competition. Specialization in complementary search abilities represents a meta-adaptation; through collaboration, this likely enables human groups (as a species and as cultural systems) to successfully adapt. Cultural change to support this system of collaborative search may therefore be essential in confronting the challenges humanity now faces.

Effects of theta transcranial alternating current stimulation (tACS) on exploration and exploitation during uncertain decision-making

Exploring ones surroundings may yield unexpected rewards, but is associated with uncertainty and risk. Alternatively, exploitation of certain outcomes is related to low risk, yet potentially better outcomes remain unexamined. As such, risk-taking behavior depends on perceived uncertainty and a trade-off between exploration-exploitation. Previously, it has been suggested that risk-taking may relate to theta activity in the prefrontal cortex. Furthermore, previous studies hinted at a relationship between a right-hemispheric bias in frontal theta asymmetry and risky behavior. In the present double-blind sham-controlled within-subject study, we applied bifrontal transcranial alternating current stimulation (tACS) at the theta frequency (5 Hz) on eighteen healthy volunteers during a gambling task. Two tACS montages with either left-right or posterior-anterior current flow were employed at an intensity of 1 mA. Results showed that, compared to sham, theta tACS increased perceived uncertainty irrespective of current flow direction. Despite this observation, no direct effect of tACS on exploration behavior and general risk-taking was observed. Furthermore, frontal theta asymmetry was more right-hemispherically biased after posterior-anterior tACS, compared to sham. Finally, we used electric field simulation to identify which regions were targeted by the tACS montages as an overlap in regions may explain why the two montages resulted in comparable outcomes. Our findings provide a first step towards understanding the relationship between frontal theta oscillations and different features of risk-taking.

Foraging tempo: Human run patterns in multiple-target search are constrained by the rate of successive responses

Human foraging tasks are beginning to provide new insights into the roles of vision, attention, and working memory during complex, multiple-target search. Here, we test the idea that "foraging tempo"-the rate of successive target selections-helps determine patterns of behaviour in these tasks. Previously, we established that the majority of target selections during unconstrained foraging happen at regular, rapid intervals, forming the "cruise phase" of a foraging trial. Furthermore, we noted that when the temporal interval between cruise phase responses was longer, the tendency to switch between target categories increased. To directly explore this relationship, we modified our standard iPad foraging task so that observers had to synchronise each response with an auditory metronome signal. Across trials, we increased the tempo and examined how this changed patterns of foraging when targets were defined either by a single feature or by a conjunction of features. The results were very clear. Increasing tempo systematically decreased the tendency for participants to switch between target categories. Although this was true for both feature and conjunction trials, there was also evidence that time constraints and target complexity interacted. As in our previous work, we also observed clear individual differences in how participants responded to changes in task difficulty. Overall, our results show that foraging tempo does influence the way participants respond, and we suggest this parameter may prove to be useful in further explorations of group and individual strategies during multiple-target search.

From exploration to exploitation: a shifting mental mode in late life development

Changes in cognition, affect, and brain function combine to promote a shift in the nature of mentation in older adulthood, favoring exploitation of prior knowledge over exploratory search as the starting point for thought and action. Age-related exploitation biases result from the accumulation of prior knowledge, reduced cognitive control, and a shift toward affective goals. These are accompanied by changes in cortical networks, as well as attention and reward circuits. By incorporating these factors into a unified account, the exploration-to-exploitation shift offers an integrative model of cognitive, affective, and brain aging. Here, we review evidence for this model, identify determinants and consequences, and survey the challenges and opportunities posed by an exploitation-biased mental mode in later life.

Sources of variation in search and foraging: A theoretical perspective

Search-the problem of exploring a space of alternatives to identify target goals-is a fundamental behaviour for many species. Although its foundation lies in foraging, most studies of human search behaviour have been directed towards understanding the attentional mechanisms that underlie the efficient visual exploration of two-dimensional (2D) scenes. With this review, we aim to characterise how search behaviour can be explained across a wide range of contexts, environments, spatial scales, and populations, both typical and atypical. We first consider the generality of search processes across psychological domains. We then review studies of interspecies differences in search. Finally, we explore in detail the individual and contextual variables that affect visual search and related behaviours in established experimental psychology paradigms. Despite the heterogeneity of the findings discussed, we identify that variations in control processes, along with the ability to regulate behaviour as a function of the structure of search space and the sampling processes adopted, to be central to explanations of variations in search behaviour. We propose a tentative theoretical model aimed at integrating these notions and close by exploring questions that remain unaddressed.

Priming exploration across domains: does search in a spatial environment influence search in a cognitive environment?

Is there a general tendency to explore that connects search behaviour across different domains? Although the experimental evidence collected so far suggests an affirmative answer, this fundamental question about human behaviour remains open. A feasible way to test the domain-generality hypothesis is that of testing the so-called priming hypothesis: priming explorative behaviour in one domain should subsequently influence explorative behaviour in another domain. However, only a limited number of studies have experimentally tested this priming hypothesis, and the evidence is mixed. We tested the priming hypothesis in a registered report. We manipulated explorative behaviour in a spatial search task by randomly allocating people to search environments with resources that were either clustered together or dispersedly distributed. We hypothesized that, in a subsequent anagram task, participants who searched in clustered spatial environments would search for words in a more clustered way than participants who searched in the dispersed spatial environments. The pre-registered hypothesis was not supported. An equivalence test showed that the difference between conditions was smaller than the smallest effect size of interest (d = 0.36). Out of several exploratory analyses, we found only one inferential result in favour of priming. We discuss implications of these findings for the theory and propose future tests of the hypothesis.

The Radically Embodied Conscious Cybernetic Bayesian Brain: From Free Energy to Free Will and Back Again

Drawing from both enactivist and cognitivist perspectives on mind, I propose that explaining teleological phenomena may require reappraising both "Cartesian theaters" and mental homunculi in terms of embodied self-models (ESMs), understood as body maps with agentic properties, functioning as predictive-memory systems and cybernetic controllers. Quasi-homuncular ESMs are suggested to constitute a major organizing principle for neural architectures due to their initial and ongoing significance for solutions to inference problems in cognitive (and affective) development. Embodied experiences provide foundational lessons in learning curriculums in which agents explore increasingly challenging problem spaces, so answering an unresolved question in Bayesian cognitive science: what are biologically plausible mechanisms for equipping learners with sufficiently powerful inductive biases to adequately constrain inference spaces? Drawing on models from neurophysiology, psychology, and developmental robotics, I describe how embodiment provides fundamental sources of empirical priors (as reliably learnable posterior expectations). If ESMs play this kind of foundational role in cognitive development, then bidirectional linkages will be found between all sensory modalities and frontal-parietal control hierarchies, so infusing all senses with somatic-motoric properties, thereby structuring all perception by relevant affordances, so solving frame problems for embodied agents. Drawing upon the Free Energy Principle and Active Inference framework, I describe a particular mechanism for intentional action selection via consciously imagined (and explicitly represented) goal realization, where contrasts between desired and present states influence ongoing policy selection via predictive coding mechanisms and backward-chained imaginings (as self-realizing predictions). This embodied developmental legacy suggests a mechanism by which imaginings can be intentionally shaped by (internalized) partially-expressed motor acts, so providing means of agentic control for attention, working memory, imagination, and behavior. I further describe the nature(s) of mental causation and self-control, and also provide an account of readiness potentials in Libet paradigms wherein conscious intentions shape causal streams leading to enaction. Finally, I provide neurophenomenological handlings of prototypical qualia including pleasure, pain, and desire in terms of self-annihilating free energy gradients via quasi-synesthetic interoceptive active inference. In brief, this manuscript is intended to illustrate how radically embodied minds may create foundations for intelligence (as capacity for learning and inference), consciousness (as somatically-grounded self-world modeling), and will (as deployment of predictive models for enacting valued goals).

Gotcha: Working memory prioritization from automatic attentional biases

Attention is an important resource for prioritizing information in working memory (WM), and it can be deployed both strategically and automatically. Most research investigating the relationship between WM and attention has focused on strategic efforts to deploy attentional resources toward remembering relevant information. However, such voluntary attentional control represents a mere subset of the attentional processes that select information to be encoded and maintained in WM (Theeuwes, Journal of Cognition, 1[1]: 29, 1-15, 2018). Here, we discuss three ways in which information becomes prioritized automatically in WM-physical salience, statistical learning, and reward learning. This review integrates findings from perception and working memory studies to propose a more sophisticated understanding of the relationship between attention and working memory.

The exploration-exploitation trade-off in a foraging task is affected by mood-related arousal and valence

The exploration-exploitation trade-off shows conceptual, functional, and neural analogies with the persistence-flexibility trade-off. We investigated whether mood, which is known to modulate the persistence-flexibility balance, would similarly affect the exploration-exploitation trade-off in a foraging task. More specifically, we tested whether interindividual differences in foraging behavior can be predicted by mood-related arousal and valence. In 119 participants, we assessed mood-related interindividual differences in exploration-exploitation using a foraging task that included minimal task constraints to reduce paradigm-induced biases of individual control tendencies. We adopted the marginal value theorem as a model-based analysis approach, which approximates optimal foraging behavior by tackling the patch-leaving problem. To assess influences of mood on foraging, participants underwent either a positive or negative mood induction. Throughout the experiment, we assessed arousal and valence levels as predictors for explorative/exploitative behavior. Our mood manipulation affected participants' arousal and valence ratings as expected. Moreover, mood-related arousal was found to predict exploration while valence predicted exploitation, which only partly matched our expectations and thereby the proposed conceptual overlap with flexibility and persistence, respectively. The current study provides a first insight into how processes related to arousal and valence differentially modulate foraging behavior. Our results imply that the relationship between exploration-exploitation and flexibility-persistence is more complicated than the semantic overlap between these terms might suggest, thereby calling for further research on the functional, neural, and neurochemical underpinnings of both trade-offs.

Investigating information seeking in physical and online environments with escape room and web search

Searching and interacting with information is one of the most fundamental behaviours of human beings – something that takes place in both online and physical environments. Yet, most studies of information interaction have focused on only one of these sides. This work aims to connect them by investigating one’s information interaction behaviours in different physical and online contexts as well as different types of tasks. During Web search (online searching) and Escape Room (physical searching), 31 participants’ behavioural data during web search (online searching) and escape room (physical searching) were collected through eye-tracker, web browser logs, and wearable video recorder. Analysis of the behavioural data suggests that individuals have a preferred search strategy that they adopt across different tasks and environments. The behavioural pattern, however, was found to be affected by the task type (e.g. problem searching vs exploratory search) and the way information is structured within the environments.

Similarities and differences in spatial and non-spatial cognitive maps

Learning and generalization in spatial domains is often thought to rely on a "cognitive map", representing relationships between spatial locations. Recent research suggests that this same neural machinery is also recruited for reasoning about more abstract, conceptual forms of knowledge. Yet, to what extent do spatial and conceptual reasoning share common computational principles, and what are the implications for behavior? Using a within-subject design we studied how participants used spatial or conceptual distances to generalize and search for correlated rewards in successive multi-armed bandit tasks. Participant behavior indicated sensitivity to both spatial and conceptual distance, and was best captured using a Bayesian model of generalization that formalized distance-dependent generalization and uncertainty-guided exploration as a Gaussian Process regression with a radial basis function kernel. The same Gaussian Process model best captured human search decisions and judgments in both domains, and could simulate realistic learning curves, where we found equivalent levels of generalization in spatial and conceptual tasks. At the same time, we also find characteristic differences between domains. Relative to the spatial domain, participants showed reduced levels of uncertainty-directed exploration and increased levels of random exploration in the conceptual domain. Participants also displayed a one-directional transfer effect, where experience in the spatial task boosted performance in the conceptual task, but not vice versa. While confidence judgments indicated that participants were sensitive to the uncertainty of their knowledge in both tasks, they did not or could not leverage their estimates of uncertainty to guide exploration in the conceptual task. These results support the notion that value-guided learning and generalization recruit cognitive-map dependent computational mechanisms in spatial and conceptual domains. Yet both behavioral and model-based analyses suggest domain specific differences in how these representations map onto actions.

Heuristics and optimal solutions to the breadth-depth dilemma

In multialternative risky choice, we are often faced with the opportunity to allocate our limited information-gathering capacity between several options before receiving feedback. In such cases, we face a natural trade-off between breadth-spreading our capacity across many options-and depth-gaining more information about a smaller number of options. Despite its broad relevance to daily life, including in many naturalistic foraging situations, the optimal strategy in the breadth-depth trade-off has not been delineated. Here, we formalize the breadth-depth dilemma through a finite-sample capacity model. We find that, if capacity is small (∼10 samples), it is optimal to draw one sample per alternative, favoring breadth. However, for larger capacities, a sharp transition is observed, and it becomes best to deeply sample a very small fraction of alternatives, which roughly decreases with the square root of capacity. Thus, ignoring most options, even when capacity is large enough to shallowly sample all of them, is a signature of optimal behavior. Our results also provide a rich casuistic for metareasoning in multialternative decisions with bounded capacity using close-to-optimal heuristics.

Multiscale Computation and Dynamic Attention in Biological and Artificial Intelligence

Biological and artificial intelligence (AI) are often defined by their capacity to achieve a hierarchy of short-term and long-term goals that require incorporating information over time and space at both local and global scales. More advanced forms of this capacity involve the adaptive modulation of integration across scales, which resolve computational inefficiency and explore-exploit dilemmas at the same time. Research in neuroscience and AI have both made progress towards understanding architectures that achieve this. Insight into biological computations come from phenomena such as decision inertia, habit formation, information search, risky choices and foraging. Across these domains, the brain is equipped with mechanisms (such as the dorsal anterior cingulate and dorsolateral prefrontal cortex) that can represent and modulate across scales, both with top-down control processes and by local to global consolidation as information progresses from sensory to prefrontal areas. Paralleling these biological architectures, progress in AI is marked by innovations in dynamic multiscale modulation, moving from recurrent and convolutional neural networks-with fixed scalings-to attention, transformers, dynamic convolutions, and consciousness priors-which modulate scale to input and increase scale breadth. The use and development of these multiscale innovations in robotic agents, game AI, and natural language processing (NLP) are pushing the boundaries of AI achievements. By juxtaposing biological and artificial intelligence, the present work underscores the critical importance of multiscale processing to general intelligence, as well as highlighting innovations and differences between the future of biological and artificial intelligence.

When working memory mechanisms compete: Predicting cognitive flexibility versus mental set

Cognitive flexibility is a hallmark of individuals with higher working memory capacity (WMC). Yet, individuals with higher WMC sometimes demonstrate greater rigidity in problem solving. The present research examines a novel account for these contradictory findings-that different WMC mechanisms support versus constrain cognitive flexibility. Across three studies, participants completed the water jug task-a problem-solving task requiring them to first establish and then break mental set. Predictor measures targeted three WMC mechanisms: attention control, primary memory, and secondary memory. In Study 1, primary and secondary memory predicted breaking mental set in opposite directions. Higher primary memory facilitated breaking mental set, whereas higher secondary memory hindered it. Study 2 demonstrated that attention control moderates these effects. Study 3 replicated these results using a less restrictive sampling procedure (i.e., participants were provided the strategy needed to establish mental set). The present research supports the proposed theory of functional opponency in WMC.

Foraging in Mind

People and other animals can search for information inside their heads. Where does this ability come from, and what does it enable cognitive systems to do? In this article, we address the behavioral and cognitive similarities between search in external environments and internal environments (e.g., memory). These require both maplike representations and the means to navigate them, and the latter involves modulation between exploitation and exploration analogous to a foraging process called area-restricted search. These findings have implications for understanding a number of cognitive abilities commonly considered to be hallmarks of the human species, such as well-developed executive control and goal-directed cognition, autonoetic consciousness (i.e., self-awareness), deliberation, and free will. Moreover, this research extends our conception of what organisms may share these abilities and how they evolved.

Neurocognitive free will

Free will is an apparent paradox because it requires a historical identity to escape its history in a self-guided fashion. Philosophers have itemized design features necessary for this escape, scaling from action to agency and vice versa. These can be organized into a coherent framework that neurocognitive capacities provide and that form a basis for neurocognitive free will. These capacities include (1) adaptive access to unpredictability, (2) tuning of this unpredictability in the service of hierarchical goal structures, (3) goal-directed deliberation via search over internal cognitive representations, and (4) a role for conscious construction of the self in the generation and choice of alternatives. This frames free will as a process of generative self-construction, by which an iterative search process samples from experience in an adaptively exploratory fashion, allowing the agent to explore itself in the construction of alternative futures. This provides an explanation of how effortful conscious control modulates adaptive access to unpredictability and resolves one of free will's key conceptual problems: how randomness is used in the service of the will. The implications provide a contemporary neurocognitive grounding to compatibilist and libertarian positions on free will, and demonstrate how neurocognitive understanding can contribute to this debate by presenting free will as an interaction between our freedom and our will.

Sources of suboptimality in a minimalistic explore-exploit task

People often choose between sticking with an available good option (exploitation) and trying out a new option that is uncertain but potentially more rewarding (exploration)^1,2. Laboratory studies on explore-exploit decisions often contain real-world complexities such as non-stationary environments, stochasticity under exploitation and unknown reward distributions^3-7. However, such factors might limit the researcher's ability to understand the essence of people's explore-exploit decisions. For this reason, we introduce a minimalistic task in which the optimal policy is to start off exploring and to switch to exploitation at most once in each sequence of decisions. The behaviour of 49 laboratory and 143 online participants deviated both qualitatively and quantitatively from the optimal policy, even when allowing for bias and decision noise. Instead, people seem to follow a suboptimal rule in which they switch from exploration to exploitation when the highest reward so far exceeds a certain threshold. Moreover, we show that this threshold decreases approximately linearly with the proportion of the sequence that remains, suggesting a temporal ratio law. Finally, we find evidence for 'sequence-level' variability that is shared across all decisions in the same sequence. Our results emphasize the importance of examining sequence-level strategies and their variability when studying sequential decision-making.

Local field potentials in dorsal anterior cingulate sulcus reflect rewards but not travel time costs during foraging

To maximise long-term reward rates, foragers deciding when to leave a patch must compute a decision variable that reflects both the immediately available reward and the time costs associated with travelling to the next patch. Identifying the mechanisms that mediate this computation is central to understanding how brains implement foraging decisions. We previously showed that firing rates of dorsal anterior cingulate sulcus neurons incorporate both variables. This result does not provide information about whether integration of information reflected in dorsal anterior cingulate sulcus spiking activity arises locally or whether it is inherited from upstream structures. Here, we examined local field potentials gathered simultaneously with our earlier recordings. In the majority of recording sites, local field potential spectral bands - specifically theta, beta, and gamma frequency ranges - encoded immediately available rewards but not time costs. The disjunction between information contained in spiking and local field potentials can constrain models of foraging-related processing. In particular, given the proposed link between local field potentials and inputs to a brain area, it raises the possibility that local processing within dorsal anterior cingulate sulcus serves to more fully bind immediate reward and time costs into a single decision variable.

More expertise for a better perspective: Task and strategy-driven adaptive neurofunctional reorganization for word production in high-performing older adults

The suggestion that neurofunctional reorganization may contribute to preserved language abilities is still emerging in aging studies. Some of these abilities, such as verbal fluency (VF), are not unitary but instead rely on different strategic processes that are differentially changed with age. Younger (n = 13) and older adults (n = 13) carried out an overt self-paced semantic and orthographic VF tasks within mixed fMRI design. Our results suggest that patterns of brain activation sustaining equivalent performances could be underpinned by different strategies facing brain changes during healthy aging. These main findings suggest that temporally mediated semantic clustering and frontally mediated orthographic switching were driven by evolutive neurofunctional resources in high-performing older adults. These age-related activation changes can appear to be compatible with the idea that unique neural patterns expressing distinctive cognitive strategies are necessary to support older adults' performance on VF tasks.

Posterior Cingulate Neurons Dynamically Signal Decisions to Disengage during Foraging

Foraging for resources is a fundamental behavior balancing systematic search and strategic disengagement. The foraging behavior of primates is especially complex and requires long-term memory, value comparison, strategic planning, and decision-making. Here we provide evidence from two different foraging tasks that neurons in primate posterior cingulate cortex (PCC) signal decision salience during foraging to motivate disengagement from the current strategy. In our foraging tasks, salience refers to the difference between decision thresholds and the net harvested reward. Salience signals were stronger in poor foraging contexts than rich ones, suggesting low harvest rates recruit mechanisms in PCC that regulate strategic disengagement and exploration during foraging.

Creative foraging: An experimental paradigm for studying exploration and discovery

Creative exploration is central to science, art and cognitive development. However, research on creative exploration is limited by a lack of high-resolution automated paradigms. To address this, we present such an automated paradigm, the creative foraging game, in which people search for novel and valuable solutions in a large and well-defined space made of all possible shapes made of ten connected squares. Players discovered shape categories such as digits, letters, and airplanes as well as more abstract categories. They exploited each category, then dropped it to explore once again, and so on. Aligned with a prediction of optimal foraging theory (OFT), during exploration phases, people moved along meandering paths that are about three times longer than the shortest paths between shapes; when exploiting a category of related shapes, they moved along the shortest paths. The moment of discovery of a new category was usually done at a non-prototypical and ambiguous shape, which can serve as an experimental proxy for creative leaps. People showed individual differences in their search patterns, along a continuum between two strategies: a mercurial quick-to-discover/quick-to-drop strategy and a thorough slow-to-discover/slow-to-drop strategy. Contrary to optimal foraging theory, players leave exploitation to explore again far before categories are depleted. This paradigm opens the way for automated high-resolution study of creative exploration.

Semantic search during divergent thinking

Divergent thinking, as a method of examining creative cognition, has not been adequately analyzed in the context of modern cognitive theories. This article casts divergent thinking responding in the context of theories of memory search. First, it was argued that divergent thinking tasks are similar to semantic fluency tasks, but are more constrained, and less well structured. Next, response time distributions from 54 participants were analyzed for temporal and semantic clustering. Participants responded to two prompts from the alternative uses test: uses for a brick and uses for a bottle, for two minutes each. Participants' cumulative response curves were negatively accelerating, in line with theories of search of associative memory. However, results of analyses of semantic and temporal clustering suggested that clustering is less evident in alternative uses responding compared to semantic fluency tasks. This suggests either that divergent thinking responding does not involve an exhaustive search through a clustered memory trace, but rather that the process is more exploratory, yielding fewer overall responses that tend to drift away from close associates of the divergent thinking prompt.

The social transmission of metacontrol policies: Mechanisms underlying the interpersonal transfer of persistence and flexibility

Humans often face binary cognitive-control dilemmas, with the choice between persistence and flexibility being a crucial one. Tackling these dilemmas requires metacontrol, i.e., the control of the current cognitive-control policy. As predicted from functional, psychometric, neuroscientific, and modeling approaches, interindividual variability in metacontrol biases towards persistence or flexibility could be demonstrated in metacontrol-sensitive tasks. These biases covary systematically with genetic predispositions regarding mesofrontal and nigrostriatal dopaminergic functioning and the individualistic or collectivistic nature of the cultural background. However, there is also evidence for mood- and meditation-induced intraindividual variability (with negative mood and focused-attention meditation being associated with a bias towards persistence, and positive mood and open-monitoring meditation being associated with a bias towards flexibility), suggesting that genetic and cultural factors do not determine metacontrol settings entirely. We suggest a theoretical framework that explains how genetic predisposition and cultural learning can lead to the implementation of metacontrol defaults, which however can be shifted towards persistence or flexibility by situational factors.

Understanding visual attention in childhood: Insights from a new visual foraging task

A recently developed visual foraging task, involving multiple targets of different types, can provide a rich and dynamic picture of visual attention performance. We measured the foraging performance of 66 children aged 4–7 years, along with measures of two conceptually related constructs, self-regulation and verbal working memory. Our results show that foraging patterns of young children differ from adult patterns. Children have difficulty with foraging for two target types, not only when they are defined by a conjunction of features but, unlike adults, also when they forage simultaneously for two target types that are distinguished from distractors by a single feature. Importantly, such feature/conjunction differences between adults and children are not seen in more traditional single-target visual search tasks. Interestingly, the foraging patterns of the youngest children were slightly more adult-like than of the oldest ones, which may suggest that older children attempt to use strategies that they have not yet fully mastered. The older children were, however, able to complete more trials, during both feature and conjunction foraging. Self-regulation and verbal working memory did not seem to affect foraging strategies, but both were connected with faster and more efficient foraging. We propose that our visual foraging paradigm is a promising avenue for studying the development of visual cognitive abilities.

Is semantic verbal fluency impairment explained by executive function deficits in schizophrenia?

OBJECTIVE

To investigate if verbal fluency impairment in schizophrenia reflects executive function deficits or results from degraded semantic store or inefficient search and retrieval strategies.

METHOD

Two groups were compared: 141 individuals with schizophrenia and 119 healthy age and education-matched controls. Both groups performed semantic and phonetic verbal fluency tasks. Performance was evaluated using three scores, based on 1) number of words generated; 2) number of clustered/related words; and 3) switching score. A fourth performance score based on the number of clusters was also measured.

RESULTS

SZ individuals produced fewer words than controls. After controlling for the total number of words produced, a difference was observed between the groups in the number of cluster-related words generated in the semantic task. In both groups, the number of words generated in the semantic task was higher than that generated in the phonemic task, although a significant group vs. fluency type interaction showed that subjects with schizophrenia had disproportionate semantic fluency impairment. Working memory was positively associated with increased production of words within clusters and inversely correlated with switching.

CONCLUSION

Semantic fluency impairment may be attributed to an inability (resulting from reduced cognitive control) to distinguish target signal from competing noise and to maintain cues for production of memory probes.

Evolution of protolinguistic abilities as a by-product of learning to forage in structured environments

The skills required for the learning and use of language are the focus of extensive research, and their evolutionary origins are widely debated. Using agent-based simulations in a range of virtual environments, we demonstrate that challenges of foraging for food can select for cognitive mechanisms supporting complex, hierarchical, sequential learning, the need for which arises in language acquisition. Building on previous work, where we explored the conditions under which reinforcement learning is out-competed by seldom-reinforced continuous learning that constructs a network model of the environment, we now show that realistic features of the foraging environment can select for two critical advances: (i) chunking of meaningful sequences found in the data, leading to representations composed of units that better fit the prevalent statistical patterns in the environment; and (ii) generalization across units based on their contextual similarity. Importantly, these learning processes, which in our framework evolved for making better foraging decisions, had been earlier shown to reproduce a range of findings in language learning in humans. Thus, our results suggest a possible evolutionary trajectory that may have led from basic learning mechanisms to complex hierarchical sequential learning that can support advanced cognitive abilities of the kind needed for language acquisition.

Hybrid foraging search: Searching for multiple instances of multiple types of target

This paper introduces the "hybrid foraging" paradigm. In typical visual search tasks, observers search for one instance of one target among distractors. In hybrid search, observers search through visual displays for one instance of any of several types of target held in memory. In foraging search, observers collect multiple instances of a single target type from visual displays. Combining these paradigms, in hybrid foraging tasks observers search visual displays for multiple instances of any of several types of target (as might be the case in searching the kitchen for dinner ingredients or an X-ray for different pathologies). In the present experiment, observers held 8-64 target objects in memory. They viewed displays of 60-105 randomly moving photographs of objects and used the computer mouse to collect multiple targets before choosing to move to the next display. Rather than selecting at random among available targets, observers tended to collect items in runs of one target type. Reaction time (RT) data indicate searching again for the same item is more efficient than searching for any other targets, held in memory. Observers were trying to maximize collection rate. As a result, and consistent with optimal foraging theory, they tended to leave 25-33% of targets uncollected when moving to the next screen/patch. The pattern of RTs shows that while observers were collecting a target item, they had already begun searching memory and the visual display for additional targets, making the hybrid foraging task a useful way to investigate the interaction of visual and memory search.

Integration of Social Information by Human Groups

We consider a situation in which individuals search for accurate decisions without direct feedback on their accuracy, but with information about the decisions made by peers in their group. The "wisdom of crowds" hypothesis states that the average judgment of many individuals can give a good estimate of, for example, the outcomes of sporting events and the answers to trivia questions. Two conditions for the application of wisdom of crowds are that estimates should be independent and unbiased. Here, we study how individuals integrate social information when answering trivia questions with answers that range between 0% and 100% (e.g., "What percentage of Americans are left-handed?"). We find that, consistent with the wisdom of crowds hypothesis, average performance improves with group size. However, individuals show a consistent bias to produce estimates that are insufficiently extreme. We find that social information provides significant, albeit small, improvement to group performance. Outliers with answers far from the correct answer move toward the position of the group mean. Given that these outliers also tend to be nearer to 50% than do the answers of other group members, this move creates group polarization away from 50%. By looking at individual performance over different questions we find that some people are more likely to be affected by social influence than others. There is also evidence that people differ in their competence in answering questions, but lack of competence is not significantly correlated with willingness to change guesses. We develop a mathematical model based on these results that postulates a cognitive process in which people first decide whether to take into account peer guesses, and if so, to move in the direction of these guesses. The size of the move is proportional to the distance between their own guess and the average guess of the group. This model closely approximates the distribution of guess movements and shows how outlying incorrect opinions can be systematically removed from a group resulting, in some situations, in improved group performance. However, improvement is only predicted for cases in which the initial guesses of individuals in the group are biased.

Commentary on Mata and von Helversen: Foraging Theory as a Paradigm Shift for Cognitive Aging

Mata and von Helversen's integrative review of adult age differences in search performance makes a good case that cognitive control may impact certain aspects of self-regulation of search. However, information foraging as a framework also offers an avenue to consider how adults of different ages adapt to age-related changes in cognition, such as in cognitive control.

Search and the Aging Mind: The Promise and Limits of the Cognitive Control Hypothesis of Age Differences in Search

Search is a prerequisite for successful performance in a broad range of tasks ranging from making decisions between consumer goods to memory retrieval. How does aging impact search processes in such disparate situations? Aging is associated with structural and neuromodulatory brain changes that underlie cognitive control processes, which in turn have been proposed as a domain-general mechanism controlling search in external environments as well as memory. We review the aging literature to evaluate the cognitive control hypothesis that suggests that age-related change in cognitive control underlies age differences in both external and internal search. We also consider the limits of the cognitive control hypothesis and propose additional mechanisms such as changes in strategy use and affect that may be necessary to understand how aging affects search.

The Use of Canonical Correlation Analysis to Assess the Relationship Between Executive Functioning and Verbal Memory in Older Adults

Executive functioning (EF), which is considered to govern complex cognition, and verbal memory (VM) are constructs assumed to be related. However, it is not known the magnitude of the association between EF and VM, and how sociodemographic and psychological factors may affect this relationship, including in normal aging. In this study, we assessed different EF and VM parameters, via a battery of neurocognitive/psychological tests, and performed a Canonical Correlation Analysis (CCA) to explore the connection between these constructs, in a sample of middle-aged and older healthy individuals without cognitive impairment (N = 563, 50+ years of age). The analysis revealed a positive and moderate association between EF and VM independently of gender, age, education, global cognitive performance level, and mood. These results confirm that EF presents a significant association with VM performance.

Exploration versus exploitation in space, mind, and society

Search is a ubiquitous property of life. Although diverse domains have worked on search problems largely in isolation, recent trends across disciplines indicate that the formal properties of these problems share similar structures and, often, similar solutions. Moreover, internal search (e.g., memory search) shows similar characteristics to external search (e.g., spatial foraging), including shared neural mechanisms consistent with a common evolutionary origin across species. Search problems and their solutions also scale from individuals to societies, underlying and constraining problem solving, memory, information search, and scientific and cultural innovation. In summary, search represents a core feature of cognition, with a vast influence on its evolution and processes across contexts and requiring input from multiple domains to understand its implications and scope.