Exploration versus exploitation in space, mind, and society

Foraging in Mind

People and other animals can search for information inside their heads. Where does this ability come from, and what does it enable cognitive systems to do? In this article, we address the behavioral and cognitive similarities between search in external environments and internal environments (e.g., memory). These require both maplike representations and the means to navigate them, and the latter involves modulation between exploitation and exploration analogous to a foraging process called area-restricted search. These findings have implications for understanding a number of cognitive abilities commonly considered to be hallmarks of the human species, such as well-developed executive control and goal-directed cognition, autonoetic consciousness (i.e., self-awareness), deliberation, and free will. Moreover, this research extends our conception of what organisms may share these abilities and how they evolved.

Fifty years of Theoretical Population Biology

The year 2020 marks the 50th anniversary of Theoretical Population Biology. This special issue examines the past and continuing contributions of the journal. We identify some of the most important developments that have taken place in the pages of TPB, connecting them to current research and to the numerous forms of significance achieved by theory in population biology.

Mouse Navigation Strategies for Odor Source Localization

Navigating an odor landscape is a critical behavior for the survival of many species, including mice. An ethologically relevant mouse behavior is locating food using information about odor concentration. To approximate this behavior, we used an open field odor-based spot-finding task indoors with little wind, examining navigation strategies as mice search for and approach an odor source. After mice were trained to navigate to odor sources paired with food reward, we observed behavioral changes consistent with localization 10-45 cm away from the source. These behaviors included orientation toward the source, decreased velocity, and increased exploration time. We also found that the amplitude of 'casting,' which we define as lateral back and forth movement of the nose, increased with proximity to the source. Based on these observations, we created a concentration-sensitive agent-based model to simulate mouse behavior. This model provided evidence for a binaral-sniffing strategy (inter-nostril comparison of concentration in a single sniff) and a serial-sniffing strategy (sampling concentration, moving in space, and then sampling again). Serial-sniffing may be accomplished at farther distances by moving the body and at closer distances by moving the head (casting). Together, these results elucidate components of behavioral strategies for odor-based navigation.

Attentional selection is biased towards controllable stimuli

While the factors that contribute to individuals feeling a sense agency over a stimulus have been extensively studied, the cognitive effects of a sense of agency over a stimulus are little known. Here, we conducted three experiments examining whether attentional selection is biased towards controllable stimuli. In all three experiments, participants moved four circle stimuli, one of which was under their control. A search target then appeared on one of the stimuli. In Experiment 1, the target was always on the controlled stimulus, but we manipulated the degree of control the participant had. In Experiment 2, the controlled stimulus was the target on 50% of the trials. In Experiment 3, we used a central arrow cue to tell participants which arrow key to press (rather than using a free choice task) and made the controlled stimulus the target on 25% of the trials, making it nonpredictive of the target's location. Across the three experiments we found that visual selection was biased towards controllable stimuli. This attentional bias was larger when participants had full, rather than partial, control over the stimulus, indicating that sense of agency leads one to prioritize objects under their control. The fact that agency influenced attention when the controlled object contained the target in 100%, 50%, and 25% of trials, and occurred even when participants needed to monitor the center of the display in order to know which arrow key to press, suggests that its influence does not depend on task relevance or volitional decision-making.

Decisions from experience: Competitive search and choice in kind and wicked environments

Information search is key to making decision from experience: exploration permits people to learn about the statistical properties of choice options and thus to become aware of rare but potentially momentous decision consequences. This registered report investigates whether and how people differ when making decisions from experience in isolation versus under competitive pressure, which may have important implications for choice performance in different types of choice environments: in “kind” environments without any rare and extreme events, frugal search is sufficient to identify advantageous options. Conversely, in “wicked” environments with skewed outcome distributions, rare but important events will tend to be missed in frugal search. One theoretical view is that competitive pressure encourages efficiency and may thereby boost adaptive search in different environments. An alternative and more pessimistic view is that competitive pressure triggers agency-related concerns, leading to minimal search irrespective of the choice environment, and hence to inferior choice performance. Using a sampling game, the present study (N = 277) found that solitary search was not adaptive to different choice environments (M = 14 samples), leading to a high choice performance in a kind and in a moderately wicked environment, but somewhat lower performance in an extremely wicked environment. Competitive pressure substantially reduced search irrespective of the choice environment (M = 4 samples), thus negatively affecting overall choice performance. Yet, from the perspective of a cost-benefit framework, frugal search may be efficient under competitive pressure. In sum, this report extends research on decisions from experience by adopting an ecological perspective (i.e., systematically varying different choice environments) and by introducing a cost-benefit framework to evaluate solitary and competitive search — with the latter constituting a challenging problem for people in an increasingly connected world.

Association between Behavioral Ambidexterity and Brain Health

Appropriately handling and switching exploration of novel knowledge and exploitation of existing knowledge is a fundamental element of genuine innovation in society. Moreover, a mounting number of studies have suggested that such “ambidexterity” is associated not only with organizational performance but also with the human brain. Among these reports, however, there have not been any definitive MRI-based parameters that objectively and easily evaluate such ambidexterity. Therefore, an MRI-based index derived from gray matter volume, called the gray-matter brain healthcare quotient (GM-BHQ), was used to measure the association between ambidexterity and the entire human brain. For this purpose, 200 healthy adults were recruited as subjects to undergo structural T1-weighted imaging and to answer multiple psychological questionnaires. Ambidexterity was evaluated using two scales: the Curiosity and Exploration Inventory II and the Short Grit Scale, as exploration–exploitation indicators of curiosity and grit, respectively. Additionally, to enrich the understanding of these associations, three additional positive thinking scales were used—the General Self-Efficacy Scale, the Rosenberg Self-Esteem Scale, and the Life Orientation Test—to evaluate self-efficacy, self-esteem, and optimism, respectively. The authors discovered the GM-BHQ was weakly associated with curiosity, grit, and self-efficacy individually after controlling for age and sex. Furthermore, the GM-BHQ was directly associated with curiosity but indirectly associated with grit in the path model. However, no significant association was found between the GM-BHQ and the other outcome indicators (i.e., self-esteem and optimism). These results suggest that brain health is weakly associated with ambidexterity evaluated using psychological tests.

Integrating models of cognition and culture will require a bit more math

We support the goal to integrate models of culture and cognition. However, we are not convinced that the free energy principle and Thinking Through Other Minds will be useful in achieving it. There are long traditions of modeling both cultural evolution and cognition. Demonstrating that FEP or TTOM can integrate these models will require a bit more math.

Altered verbal fluency processes in older adults with age-related hearing loss

Epidemiological studies have linked age-related hearing loss (ARHL) with an increased risk of neurocognitive decline. Difficulties in speech perception with subsequent changes in brain morphometry, including regions important for lexical-semantic memory, are thought to be a possible mechanism for this relationship. This study investigated differences in automatic and executive lexical-semantic processes on verbal fluency tasks in individuals with acquired hearing loss. The primary outcomes were indices of automatic (clustering/word retrieval at start of task) and executive (switching/word retrieval after start of the task) processes from semantic and phonemic fluency tasks. To extract indices of clustering and switching, we used both manual and computerised methods. There were no differences between groups on indices of executive fluency processes or on any indices from the semantic fluency task. The hearing loss group demonstrated weaker automatic processes on the phonemic fluency task. Further research into differences in lexical-semantic processes with ARHL is warranted.

Reward learning biases the direction of saccades

The role of associative reward learning in guiding feature-based attention and spatial attention is well established. However, no studies have looked at the extent to which reward learning can modulate the direction of saccades during visual search. Here, we introduced a novel reward learning paradigm to examine whether reward-associated directions of eye movements can modulate performance in different visual search tasks. Participants had to fixate a peripheral target before fixating one of four disks that subsequently appeared in each cardinal position. This was followed by reward feedback contingent upon the direction chosen, where one direction consistently yielded a high reward. Thus, reward was tied to the direction of saccades rather than the absolute location of the stimulus fixated. Participants selected the target in the high-value direction on the majority of trials, demonstrating robust learning of the task contingencies. In an untimed visual foraging task that followed, which was performed in extinction, initial saccades were reliably biased in the previously rewarded-associated direction. In a second experiment, following the same training procedure, eye movements in the previously high-value direction were facilitated in a saccade-to-target task. Our findings suggest that rewarding directional eye movements biases oculomotor search patterns in a manner that is robust to extinction and generalizes across stimuli and task.

Six clarifications for behaviour systems

The precursors of contemporary behaviour systems theory were hotly debated, and yet a similar critical fervour has not followed the second generation of behaviour systems research. I raise six items of potential or extant misunderstanding concerning behaviour systems perspectives, and attempt to set straight some of the assumptions and what motivated them, with attention to historical and theoretical context. The six challenges in focus are: 1) variety of conceptualisation of consummation; 2) potential misapprehensions about the role of general search; 3) ambiguity of predictions concerning response form; 4) ambiguity concerning what aspects are modelled as hierarchical; 5) assumptions of directedness; and 6) the relevance of spontaneous activity. For each of these six issues, some clarification is offered.

The Neurobiology of Mammalian Navigation

Mammals have evolved specialized brain systems to support efficient navigation within diverse habitats and over varied distances, but while navigational strategies and sensory mechanisms vary across species, core spatial components appear to be widely shared. This review presents common elements found in mammalian spatial mapping systems, focusing on the cells in the hippocampal formation representing orientational and locational spatial information, and 'core' mammalian hippocampal circuitry. Mammalian spatial mapping systems make use of both allothetic cues (space-defining cues in the external environment) and idiothetic cues (cues derived from self-motion). As examples of each cue type, we discuss: environmental boundaries, which control both orientational and locational neuronal activity and behaviour; and 'path integration', a process that allows the estimation of linear translation from velocity signals, thought to depend upon grid cells in the entorhinal cortex. Building cognitive maps entails sampling environments: we consider how the mapping system controls exploration to acquire spatial information, and how exploratory strategies may integrate idiothetic with allothetic information. We discuss how 'replay' may act to consolidate spatial maps, and simulate trajectories to aid navigational planning. Finally, we discuss grid cell models of vector navigation.

Resynthesizing behavior through phylogenetic refinement

This article proposes that biologically plausible theories of behavior can be constructed by following a method of "phylogenetic refinement," whereby they are progressively elaborated from simple to complex according to phylogenetic data on the sequence of changes that occurred over the course of evolution. It is argued that sufficient data exist to make this approach possible, and that the result can more effectively delineate the true biological categories of neurophysiological mechanisms than do approaches based on definitions of putative functions inherited from psychological traditions. As an example, the approach is used to sketch a theoretical framework of how basic feedback control of interaction with the world was elaborated during vertebrate evolution, to give rise to the functional architecture of the mammalian brain. The results provide a conceptual taxonomy of mechanisms that naturally map to neurophysiological and neuroanatomical data and that offer a context for defining putative functions that, it is argued, are better grounded in biology than are some of the traditional concepts of cognitive science.

Hunger increases delay discounting of food and non-food rewards

How do our valuation systems change to homeostatically correct undesirable psychological or physiological states, such as those caused by hunger? There is evidence that hunger increases discounting for food rewards, biasing choices towards smaller but sooner food reward over larger but later reward. However, it is not understood how hunger modulates delay discounting for non-food items. We outline and quantitatively evaluate six possible models of how our valuation systems modulate discounting of various commodities in the face of the undesirable state of being hungry. With a repeated-measures design, an experimental hunger manipulation, and quantitative modeling, we find strong evidence that hunger causes large increases in delay discounting for food, with an approximately 25% spillover effect to non-food commodities. The results provide evidence that in the face of hunger, our valuation systems increase discounting for commodities, which cannot achieve a desired state change as well as for those commodities that can. Given that strong delay discounting can cause negative outcomes in many non-food (consumer, investment, medical, or inter-personal) domains, the present findings suggest caution may be necessary when making decisions involving non-food outcomes while hungry.

Search as a simple take-the-best heuristic

Humans commonly engage in a variety of search behaviours, for example when looking for an object, a partner, information or a solution to a complex problem. The success or failure of a search strategy crucially depends on the structure of the environment and the constraints it imposes on the individuals. Here, we focus on environments in which individuals have to explore the solution space gradually and where their reward is determined by one unique solution they choose to exploit. This type of environment has been relatively overlooked in the past despite being relevant to numerous real-life situations, such as spatial search and various problem-solving tasks. By means of a dedicated experimental design, we show that the search behaviour of experimental participants can be well described by a simple heuristic model. Both in rich and poor solution spaces, a take-the-best procedure that ignores all but one cue at a time is capable of reproducing a diversity of observed behavioural patterns. Our approach, therefore, sheds lights on the possible cognitive mechanisms involved in human search.

Search Strategies in the Perceptual-Motor Workspace and the Acquisition of Coordination, Control, and Skill

In this paper we re-visit and elaborate-on the theoretical framework of learning as searching within the perceptual-motor workspace for a solution to the task. The central focus is the nature of search strategies to locate and create stable equilibrium regions in the perceptual-motor workspace and how these strategies relate to the emergent movement forms in the acquisition of coordination, control, and skill. In the ecological theory of perception and action, the enhanced stability of performance occurs through the attunement of the perceptual systems to the task dynamics together with modifications of action as task and intrinsic dynamics cooperate and/or compete. Thus, through practice in this search process, individuals adapt to the pick-up of task relevant perceptual variables and change their movement form according to the stability of the performed action and its outcome in relation to the task demands. Contemporary experimental findings have revealed features of the search process given the interaction of individual intrinsic dynamics in the context of task requirements and principles that drive the change – e.g., exploitation of more tolerant task-space solutions and emergence of compensatory mechanisms. Finally, we outline how the search strategy framework relates to traditional learning-related phenomena: including the dynamical pathways of learning, learning curves, factors of learning, individuality, motor development, and sport and rehabilitation interventions.

Addiction beyond pharmacological effects: The role of environment complexity and bounded rationality

Several decision-making vulnerabilities have been identified as underlying causes for addictive behaviours, or the repeated execution of stereotyped actions despite their adverse consequences. These vulnerabilities are mostly associated with brain alterations caused by the consumption of substances of abuse. However, addiction can also happen in the absence of a pharmacological component, such as seen in pathological gambling and videogaming. We use a new reinforcement learning model to highlight a previously neglected vulnerability that we suggest interacts with those already identified, whilst playing a prominent role in non-pharmacological forms of addiction. Specifically, we show that a dual-learning system (i.e. combining model-based and model-free) can be vulnerable to highly rewarding, but suboptimal actions, that are followed by a complex ramification of stochastic adverse effects. This phenomenon is caused by the overload of the capabilities of an agent, as time and cognitive resources required for exploration, deliberation, situation recognition, and habit formation, all increase as a function of the depth and richness of detail of an environment. Furthermore, the cognitive overload can be aggravated due to alterations (e.g. caused by stress) in the bounded rationality, i.e. the limited amount of resources available for the model-based component, in turn increasing the agent's chances to develop or maintain addictive behaviours. Our study demonstrates that, independent of drug consumption, addictive behaviours can arise in the interaction between the environmental complexity and the biologically finite resources available to explore and represent it.

Neurocognitive free will

Free will is an apparent paradox because it requires a historical identity to escape its history in a self-guided fashion. Philosophers have itemized design features necessary for this escape, scaling from action to agency and vice versa. These can be organized into a coherent framework that neurocognitive capacities provide and that form a basis for neurocognitive free will. These capacities include (1) adaptive access to unpredictability, (2) tuning of this unpredictability in the service of hierarchical goal structures, (3) goal-directed deliberation via search over internal cognitive representations, and (4) a role for conscious construction of the self in the generation and choice of alternatives. This frames free will as a process of generative self-construction, by which an iterative search process samples from experience in an adaptively exploratory fashion, allowing the agent to explore itself in the construction of alternative futures. This provides an explanation of how effortful conscious control modulates adaptive access to unpredictability and resolves one of free will's key conceptual problems: how randomness is used in the service of the will. The implications provide a contemporary neurocognitive grounding to compatibilist and libertarian positions on free will, and demonstrate how neurocognitive understanding can contribute to this debate by presenting free will as an interaction between our freedom and our will.

Bumblebees learn foraging routes through exploitation-exploration cycles

How animals explore and acquire knowledge from the environment is a key question in movement ecology. For pollinators that feed on multiple small replenishing nectar resources, the challenge is to learn efficient foraging routes while dynamically acquiring spatial information about new resource locations. Here, we use the behavioural mapping t-Stochastic Neighbouring Embedding algorithm and Shannon entropy to statistically analyse previously published sampling patterns of bumblebees feeding on artificial flowers in the field. We show that bumblebees modulate foraging excursions into distinctive behavioural strategies, characterizing the trade-off dynamics between (i) visiting and exploiting flowers close to the nest, (ii) searching for new routes and resources, and (iii) exploiting learned flower visitation sequences. Experienced bees combine these behavioural strategies even after they find an optimal route minimizing travel distances between flowers. This behavioural variability may help balancing energy costs-benefits and facilitate rapid adaptation to changing environments and the integration of more profitable resources in their routes.

New Perspectives on the Aging Lexicon

The field of cognitive aging has seen considerable advances in describing the linguistic and semantic changes that happen during the adult life span to uncover the structure of the mental lexicon (i.e., the mental repository of lexical and conceptual representations). Nevertheless, there is still debate concerning the sources of these changes, including the role of environmental exposure and several cognitive mechanisms associated with learning, representation, and retrieval of information. We review the current status of research in this field and outline a framework that promises to assess the contribution of both ecological and psychological aspects to the aging lexicon.

Ventromedial Prefrontal Cortex Tracks Multiple Environmental Variables during Search

To make efficient foraging decisions, we must combine information about the values of available options with nonvalue information. Some accounts of ventromedial PFC (vmPFC) suggest that it has a narrow role limited to evaluating immediately available options. We examined responses of neurons in area 14 (a putative macaque homolog of human vmPFC) as 2 male macaques performed a novel foraging search task. Although many neurons encoded the values of immediately available offers, they also independently encoded several other variables that influence choice, but that are conceptually distinct from offer value. These variables include average reward rate, number of offers viewed per trial, previous offer values, previous outcome sizes, and the locations of the currently attended offer. We conclude that, rather than serving as specialized economic value center, vmPFC plays a broad role in integrating relevant environmental information to drive foraging decisions.SIGNIFICANCE STATEMENT Decision makers must often choose whether to take an immediately available option or continue to search for a better one. We hypothesized that this process, which is integral to foraging theory, leaves neural signatures in the brain region ventromedial PFC. Subjects performed a novel foraging task in which they searched through differently valued options and attempted to balance their reward threshold with various time costs. We found that neurons not only encode the values of immediately available offers, but multiplexed these with environmental variables, including reward rate, number of offers viewed, previous offer values, and spatial information. We conclude that vmPFC plays a rich role in encoding and integrating multiple foraging-related variables during economic decisions.

Creative Outcome as Implausible Utility

Two perspectives are used to reframe Simonton’s recent three-factor definition of creative outcome. The first perspective is functional: that creative ideas are those that add significantly to knowledge by providing both utility and learning. The second perspective is calculational: that learning can be estimated by the change in probabilistic beliefs about an idea’s utility before and after it has played out in its environment. The results of the reframing are proposed conceptual and mathematical definitions of (a) creative outcome as the product of two overarching factors (utility and learning) and (b) learning as a function of two subsidiary factors (blindness reduction and surprise). Learning will be shown to depend much more strongly on surprise than on blindness reduction, so creative outcome may then also be defined as “implausible utility.”

Explore or reset? Pupil diameter transiently increases in self-chosen switches between cognitive labor and leisure in either direction

When people invest effort in cognitive work, they often keep an eye open for rewarding alternative activities. Previous research suggests that the norepinephrine (NE) system regulates such trade-offs between exploitation of the current task and exploration of alternative possibilities. We examined the possibility that the NE system is involved in another trade-off, i.e., the trade-off between cognitive labor and leisure. We conducted two pre-registered studies (total N = 62) in which participants freely chose to perform either a paid 2-back task (labor) versus a non-paid task (leisure), while we tracked their pupil diameter—which is an indicator of the state of the NE system. In both studies, consistent with prior work, we found (a) increases in pupil baseline and (b) decreases in pupil dilation when participants switched from labor to leisure. Unexpectedly, we found the same pattern when participants switched from leisure back to labor. Both increases in pupil baseline and decreases in pupil dilation were short-lived. Collectively, these results are more consistent with a role of norepinephrine in reorienting attention and task switching, as suggested by network reset theory, than with a role in motivation, as suggested by adaptive gain theory.

The Shifting Architecture of Cognition and Brain Function in Older Adulthood

Cognitive aging is often described in the context of loss or decline. Emerging research suggests that the story is more complex, with older adults showing both losses and gains in cognitive ability. With increasing age, declines in controlled, or fluid, cognition occur in the context of gains in crystallized knowledge of oneself and the world. This inversion in cognitive capacities, from greater reliance on fluid abilities in young adulthood to increasingly crystallized or semanticized cognition in older adulthood, has profound implications for cognitive and real-world functioning in later life. The shift in cognitive architecture parallels changes in the functional network architecture of the brain. Observations of greater functional connectivity between lateral prefrontal brain regions, implicated in cognitive control, and the default network, implicated in memory and semantic processing, led us to propose the default-executive coupling hypothesis of aging. In this review we provide evidence that these changes in the functional architecture of the brain serve as a neural mechanism underlying the shifting cognitive architecture from younger to older adulthood. We incorporate findings spanning cognitive aging and cognitive neuroscience to present an integrative model of cognitive and brain aging, describing its antecedents, determinants, and implications for real-world functioning.

Guidance and selection history in hybrid foraging visual search

In Hybrid Foraging tasks, observers search for multiple instances of several types of target. Collecting all the dirty laundry and kitchenware out of a child's room would be a real-world example. How are such foraging episodes structured? A series of four experiments shows that selection of one item from the display makes it more likely that the next item will be of the same type. This pattern holds if the targets are defined by basic features like color and shape but not if they are defined by their identity (e.g., the letters p & d). Additionally, switching between target types during search is expensive in time, with longer response times between successive selections if the target type changes than if they are the same. Finally, the decision to leave a screen/patch for the next screen in these foraging tasks is imperfectly consistent with the predictions of optimal foraging theory. The results of these hybrid foraging studies cast new light on the ways in which prior selection history guides subsequent visual search in general.

Diverse stochasticity leads a colony of ants to optimal foraging

A mathematical model of garden ants (Lasius japonicus) is introduced herein to investigate the relationship between the distribution of the degree of stochasticity in following pheromone trails and the group foraging efficiency. Numerical simulations of the model indicate that depending on the systematic change of the feeding environment, the optimal distribution of stochasticity shifts from a mixture of almost deterministic and mildly stochastic ants to a contrasted mixture of almost deterministic ants and highly stochastic ants. In addition, the interaction between the stochasticity and the pheromone path regulates the dynamics of the foraging efficiency optimization. Stochasticity could strengthen the collective efficiency when the variance in the sensitivity to pheromone for ants is introduced in the model.

Inflammation Predicts Decision-Making Characterized by Impulsivity, Present Focus, and an Inability to Delay Gratification

Here, we propose a novel theoretical model linking present-focused decision-making to the activities of the immune system. We tested our model by examining the relationship between inflammatory activity – in vivo and in vitro – and decision-making characterized by impulsivity, present focus, and an inability to delay gratification. Results support our model, revealing that inflammation predicts these outcomes even after controlling for factors that may contribute to a spurious linkage between them. Moreover, subsequent analyses revealed that our model was a better fit for the data than alternative models using present-focused decision-making and its health-harming behavioural sequelae (e.g., smoking, risky sexual behaviour) to predict inflammation, lending support for the proposed directionality of this relationship. Together, these results suggest that inflammation may contribute to decision-making patterns that can result in undesirable personal and societal outcomes.

CAM-ADX: A New Genetic Algorithm with Increased Intensification and Diversification for Design Optimization Problems with Real Variables

This paper presents a modified genetic algorithm (GA) using a new crossover operator (ADX) and a novel statistic correlation mutation algorithm (CAM). Both ADX and CAM work with population information to improve existing individuals of the GA and increase the exploration potential via the correlation mutation. Solution-based methods offer better local improvement of already known solutions while lacking at exploring the whole search space; in contrast, evolutionary algorithms provide better global search in exchange of exploitation power. Hybrid methods are widely used for constrained optimization problems due to increased global and local search capabilities. The modified GA improves results of constrained problems by balancing the exploitation and exploration potential of the algorithm. The conducted tests present average performance for various CEC’2015 benchmark problems, while offering better reliability and superior results on path planning problem for redundant manipulator and most of the constrained engineering design problems tested compared with current works in the literature and classic optimization algorithms.

Attentional fluctuations in preschoolers: Direct and indirect relations with task accuracy, academic readiness, and school performance

Attentional control fluctuates in the presence of internal and external distractors, wandering on and off a given task. The current study investigated individual differences in attentional fluctuations in 250 preschoolers. Attentional fluctuations were assessed via intra-individual variability in response time in a Go/No-Go task. Greater fluctuations in attentional control were linked to lower task accuracy. In addition, greater attentional fluctuations predicted lower performance in a task of cognitive flexibility, the Dimensional Change Card Sort task. Attentional fluctuations were also associated with laboratory measures of academic readiness in preschool, as assessed by the Applied Problems and Letter-Word Identification subscales of the Woodcock-Johnson III Tests of Achievement, which in turn predicted teacher reports of academic performance in first grade. Attentional fluctuations also had indirect associations with emergent math skills in preschool, via cognitive flexibility, as well as indirect associations with first-grade teacher reports of academic performance, via the relations between cognitive flexibility and emergent math skills in preschool. These results suggest that consistency is an important aspect of attentional control during early childhood.

Individual differences in learning and biogenic amine levels influence the behavioural division between foraging honeybee scouts and recruits

Animals must effectively balance the time they spend exploring the environment for new resources and exploiting them. One way that social animals accomplish this balance is by allocating these two tasks to different individuals. In honeybees, foraging is divided between scouts, which tend to explore the landscape for novel resources, and recruits, which tend to exploit these resources. Exploring the variation in cognitive and physiological mechanisms of foraging behaviour will provide a deeper understanding of how the division of labour is regulated in social insect societies. Here, we uncover how honeybee foraging behaviour may be shaped by predispositions in performance of latent inhibition (LI), which is a form of non-associative learning by which individuals learn to ignore familiar information. We compared LI between scouts and recruits, hypothesizing that differences in learning would correlate with differences in foraging behaviour. Scouts seek out and encounter many new odours while locating novel resources, while recruits continuously forage from the same resource, even as its quality degrades. We found that scouts show stronger LI than recruits, possibly reflecting their need to discriminate forage quality. We also found that scouts have significantly elevated tyramine compared to recruits. Furthermore, after associative odour training, recruits have significantly diminished octopamine in their brains compared to scouts. These results suggest that individual variation in learning behaviour shapes the phenotypic behavioural differences between different types of honeybee foragers. These differences in turn have important consequences for how honeybee colonies interact with their environment. Uncovering the proximate mechanisms that influence individual variation in foraging behaviour is crucial for understanding the ecological context in which societies evolve.

Costly curiosity: People pay a price to resolve an uncertain gamble early

Humans are inherently curious creatures, continuously seeking out information about future outcomes. Such advance information is often valuable, potentially allowing people to select better courses of action. In non-human animals, this drive for information can be so strong that they forego food or water to find out a few seconds earlier whether an uncertain option will provide a reward. Here, we assess whether people will exhibit a similar sub-optimal preference for advance information. Participants played a card-flipping task where they were probabilistically rewarded based on the pattern of 3 cards that were revealed after a 5-s delay. During this delay, participants could instead pay a cost to find out the next card's identity immediately. This choice to find out early did not influence the eventual outcome. Participants preferred to find out early about 80% of the time when the information was free; they were even willing to incur an expense to get advance information about the eventual outcome. The expected magnitude of the outcome, however, had little impact on the likelihood of finding out early. These results suggest that humans, like animals, value non-instrumental information and will pay a price for such information, independent of its utility.

Visuospatial Integration and Hand-Tool Interaction in Cognitive Archaeology

Testing cognitive hypotheses in extinct species can be challenging, but it can be done through the integration of independent sources of information (e.g., anatomy, archaeology, neurobiology, psychology), and validated with quantitative and experimental approaches. The parietal cortex has undergone changes and specializations in humans, probably in regions involved in visuospatial integration. Visual imagery and hand-eye coordination are crucial for a species with a remarkable technological and symbolic capacity. Hand-tool relationships are not only a matter of spatial planning but involve deeper cognitive levels that concern body cognition, self-awareness, and the ability to integrate tools into body schemes, extending the body's functional and structural range. Therefore, a co-evolution between body and technology is to be expected not only in terms of anatomical correspondence but also in terms of cognitive integration. In prehistory, lithic tools are crucial in the interpretation of the cognitive abilities of extinct human species. The shape of tools and the grasping patterns associated with the corresponding haptic experience can supply some basic quantitative approaches to evaluate changes in the archaeological record. At the physiological level, electrodermal activity can be used as proxy to investigate the cognitive response during haptic experiences, revealing differences between tools and between subjects. These approaches can be also useful to evaluate whether and to what extent our complex cognitive resources are based on the capacity to export and delegate functions to external technological components.

Creative exploration as a scale-invariant search on a meaning landscape

Can knowledge accumulated in systems biology on mechanisms governing cell behavior help us to elucidate cognitive processes, such as human creative search? To address this, we focus on the property of scale invariance, which allows sensory systems to adapt to environmental signals spanning orders of magnitude. For example, bacteria search for nutrients, by responding to relative changes in nutrient concentration rather than absolute levels, via a sensory mechanism termed fold-change detection (FCD). Scale invariance is prevalent in cognition, yet the specific mechanisms are mostly unknown. Here, we screen many possible dynamic equation topologies, to find that an FCD model best describes creative search dynamics. The model further predicts robustness to variations in meaning perception, in agreement with behavioral data. We thus suggest FCD as a specific mechanism for scale invariant search, connecting sensory processes of cells and cognitive processes in human.

The Emergence of Social Norms and Conventions

The utility of our actions frequently depends upon the beliefs and behavior of other agents. Thankfully, through experience, we learn norms and conventions that provide stable expectations for navigating our social world. Here, we review several distinct influences on their content and distribution. At the level of individuals locally interacting in dyads, success depends on rapidly adapting pre-existing norms to the local context. Hence, norms are shaped by complex cognitive processes involved in learning and social reasoning. At the population level, norms are influenced by intergenerational transmission and the structure of the social network. As human social connectivity continues to increase, understanding and predicting how these levels and time scales interact to produce new norms will be crucial for improving communities.

Psychological Distance Promotes Exploration in Search of a Global Maximum

Agents must sometimes decide whether to exploit a known resource or search for potentially more profitable options. Here, we investigate the role of psychological distancing in promoting exploratory behavior. We argue that exploration dilemmas pit the value of a reward (“desirability”) against the difficulty or uncertainty of obtaining it (“feasibility”). Based on construal level theory, which suggests that psychological distance increases the importance of rewards’ desirability (vs. feasibility), we expect that psychological distance will increase exploration. Four experiments support this prediction. In Experiment 1, participants who were prompted to consider an exploration game from a physically distanced perspective were more likely to leave a local maximum in search of a global maximum. Experiments 2 and 3 show that social distance has similar results. Experiment 4 finds evidence of a direct association between construal mind-set and exploration. Overall, this research highlights how psychological distancing strategies can promote exploration.

A Neuroeconomic Framework for Creative Cognition

Neuroeconomics is the study of the neurobiological bases of subjective preferences and choices. We present a novel framework that synthesizes findings from the literatures on neuroeconomics and creativity to provide a neurobiological description of creative cognition. We propose that value-based decision-making processes and activity in the locus ceruleus-norepinephrine (LC-NE) neuromodulatory system underlie creative cognition, as well as the large-scale brain network dynamics shown to be associated with creativity. This reconceptualization leads to several falsifiable hypotheses that can further understanding of creativity, decision making, and brain network dynamics.

Uncertainty and Exploration

In order to discover the most rewarding actions, agents must collect information about their environment, potentially foregoing reward. The optimal solution to this "explore-exploit" dilemma is often computationally challenging, but principled algorithmic approximations exist. These approximations utilize uncertainty about action values in different ways. Some random exploration algorithms scale the level of choice stochasticity with the level of uncertainty. Other directed exploration algorithms add a "bonus" to action values with high uncertainty. Random exploration algorithms are sensitive to total uncertainty across actions, whereas directed exploration algorithms are sensitive to relative uncertainty. This paper reports a multi-armed bandit experiment in which total and relative uncertainty were orthogonally manipulated. We found that humans employ both exploration strategies, and that these strategies are independently controlled by different uncertainty computations.

Relationship Foraging: Does time spent searching predict relationship length?

Animals foraging for resources often need to alternate between searching for and benefiting from patches of those resources. Here we explore whether such patterns of behavior can usefully be applied to the human search for romantic relationships. Optimal foraging theory suggests that foragers should alter their time spent in patches based on how long they typically spend searching between patches. We test whether human relationship search can be described as a foraging task that fits this OFT prediction. By analyzing a large, demographically representative dataset on marriage and cohabitation timing using survival analysis, we find that the likelihood of a relationship ending per unit time goes down with increased duration of search before that relationship, in accord with the foraging prediction. We consider the possible applications and limits of a foraging perspective on mate search and suggest further directions for study.