The glass is half-full: overestimating the quality of a novel environment is advantageous

Exploration versus exploitation decisions in the human brain: A systematic review of functional neuroimaging and neuropsychological studies

Thoughts and actions are often driven by a decision to either explore new avenues with unknown outcomes, or to exploit known options with predictable outcomes. Yet, the neural mechanisms underlying this exploration-exploitation trade-off in humans remain poorly understood. This is attributable to variability in the operationalization of exploration and exploitation as psychological constructs, as well as the heterogeneity of experimental protocols and paradigms used to study these choice behaviours. To address this gap, here we present a comprehensive review of the literature to investigate the neural basis of explore-exploit decision-making in humans. We first conducted a systematic review of functional magnetic resonance imaging (fMRI) studies of exploration-versus exploitation-based decision-making in healthy adult humans during foraging, reinforcement learning, and information search. Eleven fMRI studies met inclusion criterion for this review. Adopting a network neuroscience framework, synthesis of the findings across these studies revealed that exploration-based choice was associated with the engagement of attentional, control, and salience networks. In contrast, exploitation-based choice was associated with engagement of default network brain regions. We interpret these results in the context of a network architecture that supports the flexible switching between externally and internally directed cognitive processes, necessary for adaptive, goal-directed behaviour. To further investigate potential neural mechanisms underlying the exploration-exploitation trade-off we next surveyed studies involving neurodevelopmental, neuropsychological, and neuropsychiatric disorders, as well as lifespan development, and neurodegenerative diseases. We observed striking differences in patterns of explore-exploit decision-making across these populations, again suggesting that these two decision-making modes are supported by independent neural circuits. Taken together, our review highlights the need for precision-mapping of the neural circuitry and behavioural correlates associated with exploration and exploitation in humans. Characterizing exploration versus exploitation decision-making biases may offer a novel, trans-diagnostic approach to assessment, surveillance, and intervention for cognitive decline and dysfunction in normal development and clinical populations.

Behavioral-dependent recursive movements and implications for resource selection

Within home ranges, animals repeatedly visit certain areas. Recursive movement patterns are widespread throughout the animal kingdom, but are rarely considered when developing resource selection models. We examined how behavioral state-dependent recursive movements influenced reource selection of eastern wild turkey (Meleagris gallopavo silvestris) broods as they aged from day 1 to 28. Because broods become more plastic in behaviors once they begin roosting off the ground, we separated data into broods that were ground roosting (1-13 days) and tree roosting (14-28 days). We used Hidden Markov Models to identify 2 behavioral states (restricted and mobile). We extracted state-specific recursive movements based on states and specific step lengths, which we integrated into a step selection analysis to evaluate resource selection. We found that in a restricted state, ground roosting broods spent less time in areas of mixed pine-hardwoods and more time in areas with greater vegetation density. Tree roosting broods revisited areas closer to shrub/scrub landcover types, and areas with greater vegetation density. Tree roosting broods also spent less time near mixed pine-hardwoods, while spending more time in areas with greater vegetation density. We found that in a mobile state, ground roosting broods revisited areas closer to secondary roads and mixed pine-hardwoods, but farther from hardwoods. Tree roosting broods revisited areas farther from secondary roads and with greater vegetation density. Tree roosting broods also spent more time in areas closer to pine. Resource selection varied depending on behavioral state and recursive movements. However, revisitation and residence time impacted selection in both ground and tree roosting broods. Our findings highlight the need to consider how behaviors can influence movement decisions and ultimately resource selection.

Resource limitations: A taxonomy

Decision making within the context of resource limitations requires balancing the short-term benefits of obtaining a resource and the long-term consequences of depleting those resources. The present manuscript focuses on four types of tasks that share this tradeoff to develop a taxonomy that will encourage a deeper understanding of the psychological processes at play. The four types considered are foraging, common pool traps, deterioration traps, and a novel designation referred to as resource cliffs. All four will be shown to include two opposite processes - depletion of the resource and its replenishment over time. By considering the unique and shared features of these tasks, a taxonomy of features emerges that can be combined to not only create novel tasks but also to shift the research focus to task features rather than specific tasks. The paper closes with a consideration of current theoretical frameworks previously applied to one or more of these resource-limitation tasks as well as the promise of reinforcement learning as a unifying theory.

Recursive use of home ranges and seasonal shifts in foraging behavior by a generalist carnivore

Coyotes (Canis latrans) colonized the southeastern United States over the last century as large predators, including the red wolf (Canis rufus) and eastern cougar (Puma concolor), were extirpated from the region. As a generalist carnivore, the coyote preys on white-tailed deer (Odocoileus virginianus) and various smaller mammals, birds, and vegetation. While resource selection by coyotes has been well documented at the home-range scale, little is known about their foraging behavior, which is an important factor in thoroughly understanding influences of coyotes on prey and sympatric carnivores. We assessed third-order resource selection of coyotes at sites across Alabama, Georgia, and South Carolina during 2015-2016. Using GPS collars, we tracked 41 resident coyotes across four calendar seasons and identified suspected foraging areas using recursive analysis where individuals repeatedly returned to known locations. We found that resident coyotes selected for open landcover types throughout the year, while avoiding primary and secondary roads. Additionally, resident coyotes avoided forested landcover types while selecting for forest edges except from April to June when they foraged within interior forest away from edges. Previous studies have documented substantive predation rates on white-tailed deer neonates by coyotes, and that fawn mortality may increase in forested landscapes away from forest edge. Our findings indicate that foraging coyotes may select forest cover types during spring where fawns are more vulnerable to predation. Additionally, there has been debate in the literature as to how coyotes obtain consistent levels of deer in their diets outside of fawning and fall hunting seasons. Our study indicates that use of road-kill carcasses by coyotes was an unlikely explanation for the presence of deer in coyote diets throughout the year, as coyotes in our study were not observed using roads during foraging excursions.

Biased Learning as a Simple Adaptive Foraging Mechanism

Adaptive cognitive biases, such as “optimism,” may have evolved as heuristic rules for computationally efficient decision-making, or as error-management tools when error payoff is asymmetrical. Ecologists typically use the term “optimism” to describe unrealistically positive expectations from the future that are driven by positively biased initial belief. Cognitive psychologists on the other hand, focus on valence-dependent optimism bias, an asymmetric learning process where information about undesirable outcomes is discounted (sometimes also termed “positivity biased learning”). These two perspectives are not mutually exclusive, and both may lead to similar emerging space-use patterns, such as increased exploration. The distinction between these two biases may becomes important, however, when considering the adaptive value of balancing the exploitation of known resources with the exploration of an ever-changing environment. Deepening our theoretical understanding of the adaptive value of valence-dependent learning, as well as its emerging space-use and foraging patterns, may be crucial for understanding whether, when and where might species withstand rapid environmental change. We present the results of an optimal-foraging model implemented as an individual-based simulation in continuous time and discrete space. Our forager, equipped with partial knowledge of average patch quality and inter-patch travel time, iteratively decides whether to stay in the current patch, return to previously exploited patches, or explore new ones. Every time the forager explores a new patch, it updates its prior belief using a simple single-parameter model of valence-dependent learning. We find that valence-dependent optimism results in the maintenance of positively biased expectations (prior-based optimism), which, depending on the spatiotemporal variability of the environment, often leads to greater fitness gains. These results provide insights into the potential ecological and evolutionary significance of valence-dependent optimism and its interplay with prior-based optimism.

Movement of Rehabilitated African Elephant Calves Following Soft Release Into a Wildlife Sanctuary

The ability to locate essential resources is a critical step for wildlife translocated into novel environments. Understanding this process of exploration is highly desirable for management that seeks to resettle wildlife, particularly as translocation projects tend to be expensive and have a high potential for failure. African savannah elephants (Loxodonta africana) are very mobile and rely on large areas especially in arid environments, and are translocated for differing management and conservation objectives. Thus, research into how translocated elephants use the landscape when released may both guide elephant managers and be useful for translocations of other species that adjust their movement to social and ecological conditions. In this study, we investigated the movement of eight GPS tracked calves (translocated in three cohorts) following their soft release into a 107 km2 fenced wildlife sanctuary in northern Kenya and compared their movement with that of five tracked wild elephants in the sanctuary. We describe their exploration of the sanctuary, discovery of water points, and activity budgets during the first seven, 14, and 20 months after release. We explored how patterns are affected by time since release, ecological conditions, and social factors. We found that calves visited new areas of the sanctuary and water points during greener periods and earlier post-release. Social context was associated with exploration, with later release and association with wild elephants predictive of visits to new areas. Wild elephants tended to use a greater number of sites per 14-day period than the released calves. Activity budgets determined from hidden Markov models (including the states directed walk, encamped, and meandering) suggested that released calves differed from wild elephants. The first two cohorts of calves spent a significantly greater proportion of time in the directed walk state and a significantly lower proportion of time in the encamped state relative to the wild elephants. Our results represent a step forward in describing the movements of elephant orphan calves released to the wild following a period of profound social disruption when they lost their natal family and were rehabilitated with other orphan calves under human care. We discuss the implications of the elephant behavior we observed for improving release procedures and for defining success benchmarks for translocation projects.

Control over patch encounters changes foraging behavior

Foraging is a common decision problem in natural environments. When new exploitable sites are always available, a simple optimal strategy is to leave a current site when its return falls below a single average reward rate. Here, we examined foraging in a more structured environment, with a limited number of sites that replenished at different rates and had to be revisited. When participants could choose sites, they visited fast-replenishing sites more often, left sites at higher levels of reward, and achieved a higher net reward rate. Decisions to exploit-or-leave a site were best explained with a computational model that included both the average reward rate for the environment and reward information about the unattended sites. This suggests that unattended sites influence leave decisions, in foraging environments where sites can be revisited.

•

Being able to select sites during foraging increased visits to high value sites

•

This visitation pattern was efficient, producing higher average reward rates

•

Decisions to leave a site were influenced by information about alternative sites

Hierarchical, Memory-Based Movement Models for Translocated Elk (Cervus canadensis)

The use of spatial memory is well-documented in many animal species and has been shown to be critical for the emergence of spatial learning. Adaptive behaviors based on learning can emerge thanks to an interdependence between the acquisition of information over time and movement decisions. The study of how spatio-ecological knowledge is constructed throughout the life of an individual has not been carried out in a quantitative and comprehensive way, hindered by the lack of knowledge of the information an animal already has of its environment at the time monitoring begins. Identifying how animals use memory to make beneficial decisions is fundamental to developing a general theory of animal movement and space use. Here we propose several mobility models based on memory and perform hierarchical Bayesian inference on 11-month trajectories of 21 elk after they were released in a completely new environment. Almost all the observed animals exhibited preferential returns to previously visited patches, such that memory and random exploration phases occurred. Memory decay was mild or negligible over the study period. The fact that individual elk rapidly become used to a relatively small number of patches was consistent with the hypothesis that they seek places with predictable resources and reduced mortality risks such as predation.

Individual-Level Memory Is Sufficient to Create Spatial Segregation among Neighboring Colonies of Central Place Foragers

AbstractCentral place foragers often segregate in space, even without signs of direct agonistic interactions. Using parsimonious individual-based simulations, we show that for species with spatial cognitive abilities, individual-level memory of resource availability can be sufficient to cause spatial segregation in the foraging ranges of colonial animals. The shapes of the foraging distributions are governed by commuting costs, the emerging distribution of depleted resources, and the fidelity of foragers to their colonies. When colony fidelity is weak and foragers can easily switch to colonies located closer to favorable foraging grounds, this leads to space partitioning with equidistant borders between neighboring colonies. In contrast, when colony fidelity is strong-for example, because larger colonies provide safety in numbers or individuals are unable to leave-it can create a regional imbalance between resource requirements and resource availability. This leads to nontrivial space-use patterns that propagate through the landscape. Interestingly, while better spatial memory creates more defined boundaries between neighboring colonies, it can lower the average intake rate of the population, suggesting a potential trade-off between an individual's attempt for increased intake and population growth rates.

Learning and Animal Movement

Integrating diverse concepts from animal behavior, movement ecology, and machine learning, we develop an overview of the ecology of learning and animal movement. Learning-based movement is clearly relevant to ecological problems, but the subject is rooted firmly in psychology, including a distinct terminology. We contrast this psychological origin of learning with the task-oriented perspective on learning that has emerged from the field of machine learning. We review conceptual frameworks that characterize the role of learning in movement, discuss emerging trends, and summarize recent developments in the analysis of movement data. We also discuss the relative advantages of different modeling approaches for exploring the learning-movement interface. We explore in depth how individual and social modalities of learning can matter to the ecology of animal movement, and highlight how diverse kinds of field studies, ranging from translocation efforts to manipulative experiments, can provide critical insight into the learning process in animal movement.

The Rules of Attraction: The Necessary Role of Animal Cognition in Explaining Conservation Failures and Successes

Integrating knowledge and principles of animal behavior into wildlife conservation and management has led to some concrete successes but has failed to improve conservation outcomes in other cases. Many conservation interventions involve attempts to either attract or repel animals, which we refer to as approach/avoidance issues. These attempts can be reframed as issues of manipulating the decisions animals make, which are driven by their perceptual abilities and attentional biases, as well as the value animals attribute to current stimuli and past learned experiences. These processes all fall under the umbrella of animal cognition. Here, we highlight rules that emerge when considering approach/avoidance conservation issues through the lens of cognitive-based management. For each rule, we review relevant conservation successes and failures to better predict the conditions in which behavior can be manipulated, and we suggest how to avoid future failures.

Patterns of ties in problem-solving networks and their dynamic properties

Understanding the functions carried out by network subgraphs is important to revealing the organizing principles of diverse complex networks. Here, we study this question in the context of collaborative problem-solving, which is central to a variety of domains from engineering and medicine to economics and social planning. We analyze the frequency of all three- and four-node subgraphs in diverse real problem-solving networks. The results reveal a strong association between a dynamic property of network subgraphs-synchronizability-and the frequency and significance of these subgraphs in problem-solving networks. In particular, we show that highly-synchronizable subgraphs are overrepresented in the networks, while poorly-synchronizable subgraphs are underrepresented, suggesting that dynamical properties affect their prevalence, and thus the global structure of networks. We propose the possibility that selective pressures that favor more synchronizable subgraphs could account for their abundance in problem-solving networks. The empirical results also show that unrelated problem-solving networks display very similar local network structure, implying that network subgraphs could represent organizational routines that enable better coordination and control of problem-solving activities. The findings could also have potential implications in understanding the functionality of network subgraphs in other information-processing networks, including biological and social networks.

Post-Release Monitoring of Western Grey Kangaroos (Macropus fuliginosus) Relocated from an Urban Development Site

Simple Summary

As a result of urban development, 122 western grey kangaroos (Macropus fuliginosus) were relocated from the outskirts of Perth, Western Australia, to a nearby forest. Tracking collars were fitted to 67 of the kangaroos to monitor survival rates and movement patterns over 12 months. Spotlighting and camera traps were used as a secondary monitoring technique particularly for those kangaroos without collars. The survival rate of kangaroos was poor, with an estimated 80% dying within the first month following relocation and only six collared kangaroos surviving for up to 12 months. This result implicates stress associated with the capture, handling, and transport of animals as the likely cause. The unexpected rapid rate of mortality emphasises the importance of minimising stress when undertaking animal relocations.

Abstract

The expansion of urban areas and associated clearing of habitat can have severe consequences for native wildlife. One option for managing wildlife in these situations is to relocate them. While there is a general perception that relocation is humane, transparency of outcomes is lacking. Here, we document the outcome of 122 western grey kangaroos (Macropus fuliginosus) relocated from an urban development site on the edge of Perth, Western Australia. Global Positioning System (GPS) or Very High Frequency (VHF) collars were fitted to 67 kangaroos, and their survival and movement were monitored over 12 months using telemetry, camera traps and spotlighting. Only six collared animals survived for the duration of the study with most dying within a week of the relocation, indicating stress associated with capture as the likely cause. By the completion of the study, 111 kangaroos were predicted to have died based on the proportion of individuals known to have died. Movement patterns of surviving GPS collared kangaroos changed over time from largely exploratory forays, to more repeated movements between focus areas within home ranges. The poor outcome here raises concerns around the viability of relocating a relatively large number of kangaroos as a management option. It also highlights the need for careful planning to limit the stress associated with capture and transport if relocations are to be used for managing kangaroos in urban areas.

Wood mouse feeding effort and decision-making when encountering a restricted unknown food source

Animals making foraging decisions must balance the energy gained, the time invested, and the influence of key environmental factors. In our work, we examined the effect of predation risk cues and experience on feeding efforts when a novel food resource was made available. To achieve this, we live-trapped wood mouse Apodemus sylvaticus in Monte de Valdelatas (Madrid), where 80 Sherman traps were set in four plots. Traps were subjected to two food-access difficulties in treatments consisting of three consecutive nights: open plastic bottles (easy) and closed bottles (difficult), both using corn as bait. To simulate predation risk, we set fox faeces in half of the traps in each plot. We also considered moonlight (medium/low) as an indirect predation risk cue. We analysed whether bottles had been bitten by mice and the gnawed area of each bottle was measured. Our results indicated that food access difficulty, experience, and predation risk determined mice feeding decisions and efforts. The ability of mice to adapt feeding effort when a new food source is available was demonstrated because a higher proportion of closed bottles exhibited bite marks and the gnawed area was bigger. Moreover, mouse experience was determinant in the use of this new resource since recaptured mice gnawed broader orifices in the bottles and the gnawed area increased each time an individual was recaptured. Additionally, direct predation risk cues prompted mice to bite the bottles whereas the effect of different moon phases varied among the food access treatments. This study provides direct evidence of formidable efficacy of wild mice to exploit a new nutrient resource while considering crucial environmental factors that shape the decision-making procedure.

Updating Beliefs under Perceived Threat

Humans are better at integrating desirable information into their beliefs than undesirable information. This asymmetry poses an evolutionary puzzle, as it can lead to an underestimation of risk and thus failure to take precautionary action. Here, we suggest a mechanism that can speak to this conundrum. In particular, we show that the bias vanishes in response to perceived threat in the environment. We report that an improvement in participants' tendency to incorporate bad news into their beliefs is associated with physiological arousal in response to threat indexed by galvanic skin response and self-reported anxiety. This pattern of results was observed in a controlled laboratory setting (Experiment I), where perceived threat was manipulated, and in firefighters on duty (Experiment II), where it naturally varied. Such flexibility in how individuals integrate information may enhance the likelihood of responding to warnings with caution in environments rife with threat, while maintaining a positivity bias otherwise, a strategy that can increase well-being.SIGNIFICANCE STATEMENT The human tendency to be overly optimistic has mystified scholars and lay people for decades: How could biased beliefs have been selected over unbiased beliefs? Scholars have suggested that although the optimism bias can lead to negative outcomes, including financial collapse and war, it can also facilitate health and productivity. Here, we demonstrate that a mechanism generating the optimism bias, namely asymmetric information integration, evaporates under threat. Such flexibility could result in enhanced caution in dangerous environments while supporting an optimism bias otherwise, potentially increasing well-being.

Energy state affects exploratory behavior of tree sparrows in a group context under differential food-patch distributions

BACKGROUND

When facing a novel situation, animals can retreat or leave to avoid risks, but will miss potential resources and opportunities. Alternatively they may reduce environmental uncertainty by exploration, while risking no energy rewards and exposure to hazards, and use the information retrieved for subsequent decision making. When exploring, however, animals may adopt different tactics according to individual states.

RESULTS

We tested that energy states will affect exploratory behavior by experimenting with wild-caught untrained Eurasian tree sparrows (Passer montanus) in fasted or fed states exploring in a novel space with hidden food supply in different patch distribution patterns. Our data revealed that fasted sparrows risked being earlier explorers more often, initiated more exploratory bouts before patches were found, and stayed longer on the ground under both patch patterns. Fasted sparrows discovered more patches and consumed more food than fed sparrows in dispersed, but not necessary so in clumped, patch patterns; whereas fed birds also increased patch finding to a certain level in dispersed patterns. Sparrows of both energy states, however, did not differ in feeding rates in either patch pattern.

CONCLUSIONS

Exploratory behavior of tree sparrows is state-dependent, which supports our prediction that birds with an energy shortage will be risk-prone and explore more readily. Our study also indicates a game nature of tree sparrow exploratory behavior in a group context when explorers are in different energy states and are exposed to different patch distributions. Birds of lower energy state adopting an active exploring tactic may be favored by obtaining higher energy gains in dispersed patch patterns with lower patch richness. More satiated birds, however, achieved a similar feeding rate by lowered exposure time.

The exploration-exploitation dilemma: a multidisciplinary framework

The trade-off between the need to obtain new knowledge and the need to use that knowledge to improve performance is one of the most basic trade-offs in nature, and optimal performance usually requires some balance between exploratory and exploitative behaviors. Researchers in many disciplines have been searching for the optimal solution to this dilemma. Here we present a novel model in which the exploration strategy itself is dynamic and varies with time in order to optimize a definite goal, such as the acquisition of energy, money, or prestige. Our model produced four very distinct phases: Knowledge establishment, Knowledge accumulation, Knowledge maintenance, and Knowledge exploitation, giving rise to a multidisciplinary framework that applies equally to humans, animals, and organizations. The framework can be used to explain a multitude of phenomena in various disciplines, such as the movement of animals in novel landscapes, the most efficient resource allocation for a start-up company, or the effects of old age on knowledge acquisition in humans.