Reliability, uncertainty, and costs in the evolution of animal learning

Behavioral selection in structured populations

The multilevel model of behavioral selection (MLBS) by Borgstede and Eggert (Behav Process 186:104370. 10.1016/j.beproc.2021.104370 , 2021) provides a formal framework that integrates reinforcement learning with natural selection using an extended Price equation. However, the MLBS is so far only formulated for homogeneous populations, thereby excluding all sources of variation between individuals. This limitation is of primary theoretical concern because any application of the MLBS to real data requires to account for variation between individuals. In this paper, I extend the MLBS to account for inter-individual variation by dividing the population into homogeneous sub-populations and including class-specific reproductive values as weighting factors for an individual's evolutionary fitness. The resulting formalism closes the gap between the theoretical underpinnings of behavioral selection and the application of the theory to empirical data, which naturally includes inter-individual variation. Furthermore, the extended MLBS is used to establish an explicit connection between the dynamics of learning and the maximization of individual fitness. These results expand the scope of the MLBS as a general theoretical framework for the quantitative analysis of learning and evolution.

A neural network model for the evolution of learning in changing environments

Learning from past experience is an important adaptation and theoretical models may help to understand its evolution. Many of the existing models study simple phenotypes and do not consider the mechanisms underlying learning while the more complex neural network models often make biologically unrealistic assumptions and rarely consider evolutionary questions. Here, we present a novel way of modelling learning using small neural networks and a simple, biology-inspired learning algorithm. Learning affects only part of the network, and it is governed by the difference between expectations and reality. We use this model to study the evolution of learning under various environmental conditions and different scenarios for the trade-off between exploration (learning) and exploitation (foraging). Efficient learning readily evolves in our individual-based simulations. However, in line with previous studies, the evolution of learning is less likely in relatively constant environments, where genetic adaptation alone can lead to efficient foraging, or in short-lived organisms that cannot afford to spend much of their lifetime on exploration. Once learning does evolve, the characteristics of the learning strategy (i.e. the duration of the learning period and the learning rate) and the average performance after learning are surprisingly little affected by the frequency and/or magnitude of environmental change. In contrast, an organism's lifespan and the distribution of resources in the environment have a clear effect on the evolved learning strategy: a shorter lifespan or a broader resource distribution lead to fewer learning episodes and larger learning rates. Interestingly, a longer learning period does not always lead to better performance, indicating that the evolved neural networks differ in the effectiveness of learning. Overall, however, we show that a biologically inspired, yet relatively simple, learning mechanism can evolve to lead to an efficient adaptation in a changing environment.

Information Ecology: an integrative framework for studying animal behavior

Information is simultaneously a valuable resource for animals and a tractable variable for researchers. We propose the name Information Ecology to describe research focused on how individual animals use information to enhance fitness. An explicit focus on information in animal behavior is far from novel - we simply build on these ideas and promote a unified approach to how and why animals use information. The value of information to animals favors the theoretically rich adaptive approach of field-based research. Simultaneously, our ability to manipulate information lends itself to the strong methods of laboratory-based research. Information Ecology asks three questions: What information is available? How is it used (or not)? And, why is it used (or not)?

Microhabitat conditions drive uncertainty of risk and shape neophobic responses in Trinidadian guppies, Poecilia reticulata

In response to uncertain risks, prey may rely on neophobic phenotypes to reduce the costs associated with the lack of information regarding local conditions. Neophobia has been shown to be driven by information reliability, ambient risk and predator diversity, all of which shape uncertainty of risk. We similarly expect environmental conditions to shape uncertainty by interfering with information availability. In order to test how environmental variables might shape neophobic responses in Trinidadian guppies (Poecilia reticulata), we conducted an in situ field experiment of two high-predation risk guppy populations designed to determine how the 'average' and 'variance' of several environmental factors might influence the neophobic response to novel predator models and/or novel foraging patches. Our results suggest neophobia is shaped by water velocity, microhabitat complexity, pool width and depth, as well as substrate diversity and heterogeneity. Moreover, we found differential effects of the 'average' and 'variance' environmental variables on food- and predator-related neophobia. Our study highlights that assessment of neophobic drivers should consider predation risk, various microhabitat conditions and neophobia being tested. Neophobic phenotypes are expected to increase the probability of prey survival and reproductive success (i.e. fitness), and are therefore likely linked to population health and species survival. Understanding the drivers and consequences of uncertainty of risk is an increasingly pressing issue, as ecological uncertainty increases with the combined effects of climate change, anthropogenic disturbances and invasive species.

Age-dependent winner-loser effects in a mangrove rivulus fish, Kryptolebias marmoratus

The outcomes of recent fights can provide individuals information about their relative fighting ability and affect their contest decisions (winner-loser effects). Most studies investigate the presence/absence of the effects in populations/species, but here we examine how they vary between individuals of a species in response to age-dependent growth rate. Many animals' fighting ability is highly dependent on body size, so rapid growth makes information from previous fights unreliable. Furthermore, fast-growing individuals are often at earlier developmental stages and are relatively smaller and weaker than most other individuals but are growing larger and stronger quickly. We therefore predicted winner-loser effects to be less detectable in individuals with high than low growth rates and to decay more quickly. Fast-growing individuals should also display stronger winner than loser effects, because a victory when small indicates a strength which will grow, whereas a loss might soon become irrelevant. We tested these predictions using naïve individuals of a mangrove killifish, Kryptolebias marmoratus, in different growth stages. Measures of contest intensity revealed winner/loser effects only for slow-growth individuals. Both fast- and slow-growth fish with a winning experience won more of the subsequent non-escalated contests than those with a losing experience; in fast-growth individuals this effect disappeared in 3 days, but in slow-growth fish it did not. Fast-growth individuals also displayed winner effects but not loser effects. The fish therefore responded to their contest experiences in a way which reflected value of the information from these experiences to them, consistent with our predictions.

A minimal model of learning: quantifying the cost and benefit of learning in changing environments

Many organisms have the ability to learn, but the costs and benefits of learning are difficult to quantify. We construct a minimal mathematical model of learning in which a forager attempts to maximize the amount of resources (food) it collects in a changing environment. Our model has two learning parameters: α, corresponding to the duration of the forager's memory, and [Formula: see text], corresponding to how much the forager explores the environment to learn more about it. We analyse the effect of different regimes of environmental change on the optimal memory and exploration parameters [Formula: see text]. By comparing the fitness outcomes from learning foragers to the outcomes from foragers following fixed strategies, we explicitly quantify the fitness benefit (or cost) of learning as a function of environmental change. We find that in many environments, the marginal benefit of learning is surprisingly small. In every environment, it is possible to implement learning in such a way that performance is as bad or worse than following a fixed strategy. In some environments, even the best implementations of our minimal model of learning perform worse than the best fixed strategy. Finally, we find that variance in resource values negatively biases foragers' estimates for those values, potentially explaining experimental results showing that animals prefer less variable resources.

The effect of experimental hybridization on cognition and brain anatomy: Limited phenotypic variation and transgression in Poeciliidae

Hybridization can promote phenotypic variation and often produces trait combinations distinct from the parental species. This increase in available variation can lead to the manifestation of functional novelty when new phenotypes bear adaptive value under the environmental conditions in which they occur. Although the role of hybridization as a driver of variation and novelty in traits linked to fitness is well recognized, it remains largely unknown whether hybridization can fuel behavioral novelty by promoting phenotypic variation in brain morphology and/or cognitive traits. To address this question, we investigated the effect of hybridization on brain anatomy, learning ability, and cognitive flexibility in first- and second-generation hybrids of two closely related fish species (Poecilia reticulata and Poecilia wingei). Overall, we found that F1 and F2 hybrids showed intermediate brain morphology and cognitive traits compared to parental groups. Moreover, as phenotypic dispersion and transgression were low for both brain and cognitive traits, we suggest that hybridization is not a strong driver of brain anatomical and cognitive diversification in these Poeciliidae. To determine the generality of this conclusion, hybridization experiments with cognitive tests need to be repeated in other families.

Problems with using comparative analyses of avian brain size to test hypotheses of cognitive evolution

There are multiple hypotheses for the evolution of cognition. The most prominent hypotheses are the Social Intelligence Hypothesis (SIH) and the Ecological Intelligence Hypothesis (EIH), which are often pitted against one another. These hypotheses tend to be tested using broad-scale comparative studies of brain size, where brain size is used as a proxy of cognitive ability, and various social and/or ecological variables are included as predictors. Here, we test how robust conclusions drawn from such analyses may be. First, we investigate variation in brain and body size measurements across >1000 bird species. We demonstrate that there is substantial variation in brain and body size estimates across datasets, indicating that conclusions drawn from comparative brain size models are likely to differ depending on the source of the data. Following this, we subset our data to the Corvides infraorder and interrogate how modelling decisions impact results. We show that model results change substantially depending on variable inclusion, source and classification. Indeed, we could have drawn multiple contradictory conclusions about the principal drivers of brain size evolution. These results reflect concerns from a growing number of researchers that conclusions drawn from comparative brain size studies may not be robust. We suggest that to interrogate hypotheses of cognitive evolution, a fruitful way forward is to focus on testing cognitive performance within and between closely related taxa, with an emphasis on understanding the relationship between informational uncertainty and cognitive evolution.

Individual experience influences reconstruction of division of labour under colony disturbance in a queenless ant species

BACKGROUND

Division of labour (DOL) is ubiquitous across biological hierarchies. In eusocial insects, DOL is often characterized by age-related task allocation, but workers can flexibly change their tasks, allowing for DOL reconstruction in fluctuating environments. Behavioural change driven by individual experience is regarded as a key to understanding this task flexibility. However, experimental evidence for the influence of individual experience is remains sparse. Here we tested the effect of individual experience on task choice in the queenless ponerine ant, Diacamma cf. indicum from Japan.

RESULTS

We confirmed that both nurses and foragers shifted to vacant tasks when the colony composition was biased to one or the other. We also found that nurses which are induced to forage readily revert to nursing when reintroduced into balanced colonies. In contrast, foragers which are induced to revert to nursing very rarely return to a foraging role, even 19 days post reintroduction to their original colony.

CONCLUSIONS

Taken together, our results suggest that individual experience decreases the response threshold of original foragers, as they continue to be specialist nurses in a disturbed colony. However, original nurses do not appear strongly affected by having forager experience and revert to being nurses. Therefore, while individual experience does have an effect, other factors, such as reproductive ability, are clearly required to understand DOL maintenance in fluctuating environments.

An evolutionary model of sensitive periods when the reliability of cues varies across ontogeny

Sensitive periods are widespread in nature, but their evolution is not well understood. Recent mathematical modeling has illuminated the conditions favoring the evolution of sensitive periods early in ontogeny. However, sensitive periods also exist at later stages of ontogeny, such as adolescence. Here, we present a mathematical model that explores the conditions that favor sensitive periods at later developmental stages. In our model, organisms use environmental cues to incrementally construct a phenotype that matches their environment. Unlike in previous models, the reliability of cues varies across ontogeny. We use stochastic dynamic programming to compute optimal policies for a range of evolutionary ecologies and then simulate developmental trajectories to obtain mature phenotypes. We measure changes in plasticity across ontogeny using study paradigms inspired by empirical research: adoption and cross-fostering. Our results show that sensitive periods only evolve later in ontogeny if the reliability of cues increases across ontogeny. The onset, duration, and offset of sensitive periods—and the magnitude of plasticity—depend on the specific parameter settings. If the reliability of cues decreases across ontogeny, sensitive periods are favored only early in ontogeny. These results are robust across different paradigms suggesting that empirical findings might be comparable despite different experimental designs.

Memory extinction and spontaneous recovery shaping parasitoid foraging behavior

Animals can alter their foraging behavior through associative learning, where an encounter with an essential resource (e.g., food or a reproductive opportunity) is associated with nearby environmental cues (e.g., volatiles). This can subsequently improve the animal's foraging efficiency. However, when these associated cues are encountered again, the anticipated resource is not always present. Such an unrewarding experience, also called a memory-extinction experience, can change an animal's response to the associated cues. Although some studies are available on the mechanisms of this process, they rarely focus on cues and rewards that are relevant in an animal's natural habitat. In this study, we tested the effect of different types of ecologically relevant memory-extinction experiences on the conditioned plant volatile preferences of the parasitic wasp Cotesia glomerata that uses these cues to locate its caterpillar hosts. These extinction experiences consisted of contact with only host traces (frass and silk), contact with nonhost traces, or oviposition in a nonhost near host traces, on the conditioned plant species. Our results show that the lack of oviposition, after contacting host traces, led to the temporary alteration of the conditioned plant volatile preference in C. glomerata, but this effect was plant species-specific. These results provide novel insights into how ecologically relevant memory-extinction experiences can fine-tune an animal's foraging behavior. This fine-tuning of learned behavior can be beneficial when the lack of finding a resource accurately predicts current, but not future foraging opportunities. Such continuous reevaluation of obtained information helps animals to prevent maladaptive foraging behavior.

Odour Learning Bees Have Longer Foraging Careers Than Non-learners in a Natural Environment

Individual animals allowed the opportunity to learn generally outperform those prevented from learning, yet, within a species the capacity for learning varies markedly. The evolutionary processes that maintain this variation in learning ability are not yet well understood. Several studies demonstrate links between fitness traits and visual learning, but the selection pressures operating on cognitive traits are likely influenced by multiple sensory modalities. In addition to vision, most animals will use a combination of hearing, olfaction (smell), gustation (taste), and touch to gain information about their environment. Some animals demonstrate individual preference for, or enhanced learning performance using certain senses in relation to particular aspects of their behaviour (e.g., foraging), whereas conspecific individuals may show different preferences. By assessing fitness traits in relation to different sensory modalities we will strengthen our understanding of factors driving observed variation in learning ability. We assessed the relationship between the olfactory learning ability of bumble bees (Bombus terrestris) and their foraging performance in their natural environment. We found that bees which failed to learn this odour-reward association had shorter foraging careers; foraging for fewer days and thus provisioning their colonies with fewer resources. This was not due to a reduced propensity to forage, but may have been due to a reduced ability to return to their colony. When comparing among only individuals that did learn, we found that the rate at which floral resources were collected was similar, regardless of how they performed in the olfactory learning task. Our results demonstrate that an ability to learn olfactory cues can have a positive impact of the foraging performance of B. terrestris in a natural environment, but echo findings of earlier studies on visual learning, which suggest that enhanced learning is not necessarily beneficial for bee foragers provisioning their colony.

Hard limits to cognitive flexibility: ants can learn to ignore but not avoid pheromone trails

Learning allows animals to respond to changes in their environment within their lifespan. However, many responses to the environment are innate, and need not be learned. Depending on the level of cognitive flexibility an animal shows, such responses can either be modified by learning or not. Many ants deposit pheromone trails to resources, and innately follow such trails. Here, we investigated cognitive flexibility in the ant Lasius niger by asking whether ants can overcome their innate tendency and learn to avoid conspecific pheromone trails when these predict a negative stimulus. Ants were allowed to repeatedly visit a Y-maze, one arm of which was marked with a strong but realistic pheromone trail and led to a punishment (electric shock and/or quinine solution), and the other arm of which was unmarked and led to a 1 mol l-1 sucrose reward. After ca. 10 trials, ants stopped relying on the pheromone trail, but even after 25 exposures they failed to improve beyond chance levels. However, the ants did not choose randomly: rather, most ants began to favour just one side of the Y-maze, a strategy which resulted in more efficient food retrieval over time, when compared with the first visits. Even when trained in a go/no-go paradigm which precludes side bias development, ants failed to learn to avoid a pheromone trail. These results show rapid learning flexibility towards an innate social signal, but also demonstrate a rarely seen hard limit to this flexibility.

Multi-camera field monitoring reveals costs of learning for parasitoid foraging behaviour

Dynamic conditions in nature have led to the evolution of behavioural traits that allow animals to use information on local circumstances and adjust their behaviour accordingly, for example through learning. Although learning can improve foraging efficiency, the learned information can become unreliable as the environment continues to change. This could lead to potential fitness costs when memories holding such unreliable information persist. Indeed, persistent unreliable memory was found to reduce the foraging efficiency of the parasitoid Cotesia glomerata under laboratory conditions.

Here, we evaluated the effect of such persistent unreliable memory on the foraging behaviour of C. glomerata in the field. This is a critical step in studies of foraging theory, since animal behaviour evolved under the complex conditions present in nature.

Existing methods provide little detail on how parasitoids interact with their environment in the field, therefore we developed a novel multi‐camera system that allowed us to trace parasitoid foraging behaviour in detail. With this multi‐camera system, we studied how persistent unreliable memory affected the foraging behaviour of C. glomerata when these memories led parasitoids to plants infested with non‐host caterpillars in a semi‐field set‐up.

Our results demonstrate that persistent unreliable memory can lead to maladaptive foraging behaviour in C. glomerata under field conditions and increased the likelihood of oviposition in the non‐host caterpillar Mamestra brassica. Furthermore, these time‐ and egg‐related costs can be context dependent, since they rely on the plant species used.

These results provide us with new insight on how animals use previously obtained information in naturally complex and dynamic foraging situations and confirm that costs and benefits of learning depend on the environment animals forage in. Although behavioural studies of small animals in natural habitats remain challenging, novel methods such as our multi‐camera system contribute to understanding the nuances of animal foraging behaviour.

The developmental support hypothesis: adaptive plasticity in neural development in response to cues of social support

Across mammals, cues of developmental support, such as touching, licking or attentiveness, stimulate neural development, behavioural exploration and even overall body growth. Why should such fitness-related traits be so sensitive to developmental conditions? Here, we review what we term the 'developmental support hypothesis', a potential adaptive explanation of this plasticity. Neural development can be a costly process, in terms of time, energy and exposure. However, environmental variability may sometimes compromise parental care during this costly developmental period. We propose this environmental variation has led to the evolution of adaptive plasticity of neural and behavioural development in response to cues of developmental support, where neural development is stimulated in conditions that support associated costs. When parental care is compromised, offspring grow less and adopt a more resilient and stress-responsive strategy, improving their chances of survival in difficult conditions, similar to existing ideas on the adaptive value of early-life programming of stress. The developmental support hypothesis suggests new research directions, such as testing the adaptive value of reduced neural growth and metabolism in stressful conditions, and expanding the range of potential cues animals may attend to as indicators of developmental support. Considering evolutionary and ecologically appropriate cues of social support also has implications for promoting healthy neural development in humans. This article is part of the theme issue 'Life history and learning: how childhood, caregiving and old age shape cognition and culture in humans and other animals'.

The impact of learning opportunities on the development of learning and decision-making: an experiment with passerine birds

Developmental context has been shown to influence learning abilities later in life, namely through experiments with nutritional and/or environmental constraints (i.e. lack of enrichment). However, little is known about the extent to which opportunities for learning affect the development of animal cognition, even though such opportunities are known to influence human cognitive development. We exposed young zebra finches (Taenopygia guttata) (n = 26) to one of three experimental conditions, i.e. an environment where (i) colour cues reliably predicted the presence of food (associative learning), (ii) a combination of two-colour cues reliably predicted the presence of food (conditional learning), or (iii) colour cues were non-informative (control). After conducting two different discrimination tasks, our results showed that experience with predictive cues can cause increased choice accuracy and decision-making speed. Our first learning task showed that individuals in the associative learning treatment outperformed the control treatment, while task 2 showed that individuals in the conditional learning treatment had shorter latencies when making choices compared with the control treatment. We found no support for a speed-accuracy trade-off. This dataset provides a rare longitudinal and experimental examination of the effect of predictive versus non-predictive cues during development on the cognition of adult animals. This article is part of the theme issue 'Life history and learning: how childhood, caregiving and old age shape cognition and culture in humans and other animals'.

The lag-time constraint for behavioural plasticity

Environmental instability (i.e. environments changing often) can select fixed phenotypes because of the lag time of plastically adapting to environmental changes, known as the lag-time constraint. Because behaviour can change rapidly (e.g. switching between foraging strategies), the lag-time constraint is not considered important for behavioural plasticity. Instead, it is often argued that responsive behaviour (i.e. behaviour that changes according to the environment) evolves to cope with unstable environments. But proficiently performing certain behaviours may require time for learning, for practising or, in social animals, for the group to adjust to one's behaviour. Conversely, not using certain behaviours for a period of time can reduce their level of performance. Here, using individual-based evolutionary simulations, we show that environmental instability selects for fixed behaviour when the ratio between the rates of increase and reduction in behavioural performance is below a certain threshold; only above this threshold does responsive behaviour evolve in unstable environments. Thus, the lag-time constraint can apply to behaviours that attain high performance either slowly or rapidly, depending on the relative rate with which their performance decreases when not used. We discuss these results in the context of the evolution of reduced behavioural plasticity, as seen in fixed personality differences.

A case for environmental statistics of early-life effects

There is enduring debate over the question of which early-life effects are adaptive and which ones are not. Mathematical modelling shows that early-life effects can be adaptive in environments that have particular statistical properties, such as reliable cues to current conditions and high autocorrelation of environmental states. However, few empirical studies have measured these properties, leading to an impasse. Progress, therefore, depends on research that quantifies cue reliability and autocorrelation of environmental parameters in real environments. These statistics may be different for social and non-social aspects of the environment. In this paper, we summarize evolutionary models of early-life effects. Then, we discuss empirical data on environmental statistics from a range of disciplines. We highlight cases where data on environmental statistics have been used to test competing explanations of early-life effects. We conclude by providing guidelines for new data collection and reflections on future directions.

This article is part of the theme issue ‘Developing differences: early-life effects and evolutionary medicine'.

Modeling the evolution of sensitive periods

In the past decade, there has been monumental progress in our understanding of the neurobiological basis of sensitive periods. Little is known, however, about the evolution of sensitive periods. Recent studies have started to address this gap. Biologists have built mathematical models exploring the environmental conditions in which sensitive periods are likely to evolve. These models investigate how mechanisms of plasticity can respond optimally to experience during an individual's lifetime. This paper discusses the central tenets, insights, and predictions of these models, in relation to empirical work on humans and other animals. We also discuss which future models are needed to improve the bridge between theory and data, advancing their synergy. Our paper is written in an accessible manner and for a broad audience. We hope our work will contribute to recently emerging connections between the fields of developmental neuroscience and evolutionary biology.

Cuttlefish show flexible and future-dependent foraging cognition

Some animals optimize their foraging activity by learning and memorizing food availability, in terms of quantity and quality, and adapt their feeding behaviour accordingly. Here, we investigated whether cuttlefish flexibly adapt their foraging behaviour according to the availability of their preferred prey. In Experiment 1, cuttlefish switched from a selective to an opportunistic foraging strategy (or vice versa) when the availability of their preferred prey at night was predictable versus unpredictable. In Experiment 2, cuttlefish exhibited day-to-day foraging flexibility, in response to experiencing changes in the proximate future (i.e. preferred prey available on alternate nights). In Experiment 1, the number of crabs eaten during the day decreased when shrimp (i.e. preferred food) were predictably available at night, while the consumption of crabs during the day was maintained when shrimp availability was unpredictable. Cuttlefish quickly shifted from one strategy to the other, when experimental conditions were reversed. In Experiment 2, cuttlefish only reduced their consumption of crabs during the daytime when shrimps were predictably available the following night. Their daytime foraging behaviour appeared dependent on shrimps' future availability. Overall, cuttlefish can adopt dynamic and flexible foraging behaviours including selective, opportunistic and future-dependent strategies, in response to changing foraging conditions.

What are the costs of learning? Modest trade-offs and constitutive costs do not set the price of fast associative learning ability in a parasitoid wasp

Learning ability has been associated with energetic costs that typically become apparent through trade-offs in a wide range of developmental, physiological, and life-history traits. Costs associated with learning ability can be either constitutive or induced, depending on whether they are always incurred or only when information is actively learned and memorized. Using lines of the parasitoid wasp Nasonia vitripennis that were selected for fast associative learning ability, we assessed a range of traits that have previously been identified as potential costs associated with learning. No difference in longevity, lipid reserves, tibia length, egg load, or fecundity was observed between the selected and control lines. All of these traits are considered to potentially lead to constitutive costs in the setup of this study. A gradual reversal to baseline learning after two forms of relaxed selection was indicative of a small constitutive cost of learning ability. We also tested for a trade-off with other memory types formed at later stages, but found no evidence that the mid-term memory that was selected for caused a decrease in performance of other memory types. In conclusion, we observe only one minor effect of a constitutive cost and none of the other costs and trade-offs that are reported in the literature to be of significant value in this case. We, therefore, argue for better inclusion of ecological and economic costs in studies on costs and benefits of learning ability.

Measuring and understanding individual differences in cognition

Individuals vary in their cognitive performance. While this variation forms the foundation of the study of human psychometrics, its broader importance is only recently being recognized. Explicitly acknowledging this individual variation found in both humans and non-human animals provides a novel opportunity to understand the mechanisms, development and evolution of cognition. The papers in this special issue highlight the growing emphasis on individual cognitive differences from fields as diverse as neurobiology, experimental psychology and evolutionary biology. Here, we synthesize this body of work. We consider the distinct challenges in quantifying individual differences in cognition and provide concrete methodological recommendations. In particular, future studies would benefit from using multiple task variants to ensure they target specific, clearly defined cognitive traits and from conducting repeated testing to assess individual consistency. We then consider how neural, genetic, developmental and behavioural factors may generate individual differences in cognition. Finally, we discuss the potential fitness consequences of individual cognitive variation and place these into an evolutionary framework with testable hypotheses. We intend for this special issue to stimulate researchers to position individual variation at the centre of the cognitive sciences.This article is part of the theme issue 'Causes and consequences of individual differences in cognitive abilities'.

The repeatability of cognitive performance: a meta-analysis

Behavioural and cognitive processes play important roles in mediating an individual's interactions with its environment. Yet, while there is a vast literature on repeatable individual differences in behaviour, relatively little is known about the repeatability of cognitive performance. To further our understanding of the evolution of cognition, we gathered 44 studies on individual performance of 25 species across six animal classes and used meta-analysis to assess whether cognitive performance is repeatable. We compared repeatability (R) in performance (1) on the same task presented at different times (temporal repeatability), and (2) on different tasks that measured the same putative cognitive ability (contextual repeatability). We also addressed whether R estimates were influenced by seven extrinsic factors (moderators): type of cognitive performance measurement, type of cognitive task, delay between tests, origin of the subjects, experimental context, taxonomic class and publication status. We found support for both temporal and contextual repeatability of cognitive performance, with mean R estimates ranging between 0.15 and 0.28. Repeatability estimates were mostly influenced by the type of cognitive performance measures and publication status. Our findings highlight the widespread occurrence of consistent inter-individual variation in cognition across a range of taxa which, like behaviour, may be associated with fitness outcomes.This article is part of the theme issue 'Causes and consequences of individual differences in cognitive abilities'.

Adaptive learning in non-social insects: from theory to field work, and back

We review the evidence that learning affects fitness in non-social insects. Early accounts date back from the 1970s and were based on field-based observational and experimental work, yet exploration of the ways in which various forms of learning increase fitness remains limited in non-social insects. We highlight the concerns that arise when artificial laboratory settings, which do not take the ecology of the species into account, are used to estimate fitness benefits of learning. We argue that ecologically-relevant experimental designs are most useful to provide fitness estimates of learning, that is, designs that include: firstly, offspring of wild-caught animals producing newly established stocks under relevant breeding conditions, combined with common-garden and reciprocal transplant experiments; secondly, the spatio-temporal dynamics of key ecological resources; and thirdly, the natural behaviours of the animals while searching for, and probing, resources. Finally, we provide guidelines for the study of fitness-learning relationships in an eco-evolutionary framework.

Enriching behavioral ecology with reinforcement learning methods

This article focuses on the division of labor between evolution and development in solving sequential, state-dependent decision problems. Currently, behavioral ecologists tend to use dynamic programming methods to study such problems. These methods are successful at predicting animal behavior in a variety of contexts. However, they depend on a distinct set of assumptions. Here, we argue that behavioral ecology will benefit from drawing more than it currently does on a complementary collection of tools, called reinforcement learning methods. These methods allow for the study of behavior in highly complex environments, which conventional dynamic programming methods do not feasibly address. In addition, reinforcement learning methods are well-suited to studying how biological mechanisms solve developmental and learning problems. For instance, we can use them to study simple rules that perform well in complex environments. Or to investigate under what conditions natural selection favors fixed, non-plastic traits (which do not vary across individuals), cue-driven-switch plasticity (innate instructions for adaptive behavioral development based on experience), or developmental selection (the incremental acquisition of adaptive behavior based on experience). If natural selection favors developmental selection, which includes learning from environmental feedback, we can also make predictions about the design of reward systems. Our paper is written in an accessible manner and for a broad audience, though we believe some novel insights can be drawn from our discussion. We hope our paper will help advance the emerging bridge connecting the fields of behavioral ecology and reinforcement learning.

Evolving autonomous learning in cognitive networks

There are two common approaches for optimizing the performance of a machine: genetic algorithms and machine learning. A genetic algorithm is applied over many generations whereas machine learning works by applying feedback until the system meets a performance threshold. These methods have been previously combined, particularly in artificial neural networks using an external objective feedback mechanism. We adapt this approach to Markov Brains, which are evolvable networks of probabilistic and deterministic logic gates. Prior to this work MB could only adapt from one generation to the other, so we introduce feedback gates which augment their ability to learn during their lifetime. We show that Markov Brains can incorporate these feedback gates in such a way that they do not rely on an external objective feedback signal, but instead can generate internal feedback that is then used to learn. This results in a more biologically accurate model of the evolution of learning, which will enable us to study the interplay between evolution and learning and could be another step towards autonomously learning machines.

Sampling and tracking a changing environment: persistence and reward in the foraging decisions of bumblebees

The question of when to collect new information and how to apply that information is central to much of behaviour. Theory suggests that the value of collecting information, or sampling, depends on environmental persistence and on the relative costs of making wrong decisions. However, empirical tests of how these variables interact are lacking. We tested whether bumblebee foraging decisions are indeed influenced by these two factors. We gave bees repeated choices between a resource providing a steady, mediocre reward and a resource fluctuating between a low reward and a high reward. In this paradigm, we manipulated environmental persistence by changing how long the quality of a fluctuating resource remained stable at one reward level. We manipulated the costs of decision errors by changing the relative values of the available rewards. Bees sampled the fluctuating resource more frequently when it changed quality more frequently, indicating that they measured environmental persistence and reacted to it as predicted by theory. Bees showed surprisingly suboptimal tracking, not reliably choosing the currently best resource except when the fluctuating resource was very persistent and the potential rewards high. While bees modify their choices in response to different levels of change and potential rewards, they do not always do so according to optimality predictions.