Individual exploration and selective social learning: balancing exploration-exploitation trade-offs in collective foraging

Search requires balancing exploring for more options and exploiting the ones previously found. Individuals foraging in a group face another trade-off: whether to engage in social learning to exploit the solutions found by others or to solitarily search for unexplored solutions. Social learning can better exploit learned information and decrease the costs of finding new resources, but excessive social learning can lead to over-exploitation and too little exploration for new solutions. We study how these two trade-offs interact to influence search efficiency in a model of collective foraging under conditions of varying resource abundance, resource density and group size. We modelled individual search strategies as Lévy walks, where a power-law exponent (μ) controlled the trade-off between exploitative and explorative movements in individual search. We modulated the trade-off between individual search and social learning using a selectivity parameter that determined how agents responded to social cues in terms of distance and likely opportunity costs. Our results show that social learning is favoured in rich and clustered environments, but also that the benefits of exploiting social information are maximized by engaging in high levels of individual exploration. We show that selective use of social information can modulate the disadvantages of excessive social learning, especially in larger groups and when individual exploration is limited. Finally, we found that the optimal combination of individual exploration and social learning gave rise to trajectories with μ ≈ 2 and provide support for the general optimality of such patterns in search. Our work sheds light on the interplay between individual search and social learning, and has broader implications for collective search and problem-solving.

Efficient quantitative assessment of robot swarms: coverage and targeting Lévy strategies

Biologically inspiredstrategieshave long been adapted to swarm robotic systems, including biased random walks, reaction to chemotactic cues and long-range coordination. In this paper we applyanalysis toolsdeveloped for modeling biological systems, such as continuum descriptions, to the efficient quantitative characterization of robot swarms. As an illustration, both Brownian and Lévy strategies with a characteristic long-range movement are discussed. As a result we obtain computationally fast methods for the optimization of robot movement laws to achieve a prescribed collective behavior. We show how to compute performance metrics like coverage and hitting times, and illustrate the accuracy and efficiency of our approach for area coverage and search problems. Comparisons between the continuum model and robotic simulations confirm the quantitative agreement and speed up by a factor of over 100 of our approach. Results confirm and quantify the advantage of Lévy strategies over Brownian motion for search and area coverage problems in swarm robotics.

Scale-free movement patterns in termites emerge from social interactions and preferential attachments

As the number or density of interacting individuals in a social group increases, a transition can develop from uncorrelated and disordered behavior of the individuals to a collective coherent pattern. We expand this observation by exploring the fine details of termite movement patterns to demonstrate that the value of the scaling exponent μ of a power law describing the Lévy walk of an individual is modified collectively as the density of animals in the group changes. This effect is absent when termites interact with inert obstacles. We also show that the network of encounters and interactions among specific individuals is selective, resembling a preferential attachment mechanism that is important for social networking. Our data strongly suggest that preferential attachments, a phenomenon not reported previously, and favorite interactions with a limited number of acquaintances are responsible for the generation of Lévy movement patterns in these social insects.

Current status and future directions of Lévy walk research

Lévy walks are a mathematical construction useful for describing random patterns of movement with bizarre fractal properties that seem to have no place in biology. Nonetheless, movement patterns resembling Lévy walks have been observed at scales ranging from the microscopic to the ecological. They have been seen in the molecular machinery operating within cells during intracellular trafficking, in the movement patterns of T cells within the brain, in DNA, in some molluscs, insects, fish, birds and mammals, in the airborne flights of spores and seeds, and in the collective movements of some animal groups. Lévy walks are also evident in trace fossils (ichnofossils) – the preserved form of tracks made by organisms that occupied ancient sea beds about 252-66 million years ago. And they are utilised by algae that originated around two billion years ago, and still exist today. In September of 2017, leading researchers from across the life sciences, along with mathematicians and physicists, got together at a Company of Biologists' Workshop to discuss the origins and biological significance of these movement patterns. In this Review the essence of the technical and sometimes heated discussions is distilled and made accessible for all. In just a few pages, the reader is taken from a gentle introduction to the frontiers of a very active field of scientific enquiry. What emerges is a fascinating story of a truly inter-disciplinary scientific endeavour that is seeking to better understand movement patterns occurring across all biological scales.

Summary: Movement patterns resembling Lévy walks are found in a wide variety of organisms, from cells to humans. In this Review the latest research into their origins and biological significance is discussed.

The topography of the environment alters the optimal search strategy for active particles

In environments with scarce resources, adopting the right search strategy can make the difference between succeeding and failing, even between life and death. At different scales, this applies to molecular encounters in the cell cytoplasm, to animals looking for food or mates in natural landscapes, to rescuers during search and rescue operations in disaster zones, and to genetic computer algorithms exploring parameter spaces. When looking for sparse targets in a homogeneous environment, a combination of ballistic and diffusive steps is considered optimal; in particular, more ballistic Lévy flights with exponent [Formula: see text] are generally believed to optimize the search process. However, most search spaces present complex topographies. What is the best search strategy in these more realistic scenarios? Here, we show that the topography of the environment significantly alters the optimal search strategy toward less ballistic and more Brownian strategies. We consider an active particle performing a blind cruise search for nonregenerating sparse targets in a 2D space with steps drawn from a Lévy distribution with the exponent varying from [Formula: see text] to [Formula: see text] (Brownian). We show that, when boundaries, barriers, and obstacles are present, the optimal search strategy depends on the topography of the environment, with [Formula: see text] assuming intermediate values in the whole range under consideration. We interpret these findings using simple scaling arguments and discuss their robustness to varying searcher's size. Our results are relevant for search problems at different length scales from animal and human foraging to microswimmers' taxis to biochemical rates of reaction.

A Chemical Reaction Network to Generate Random, Power-Law-Distributed Time Intervals

In Lévy walks (LWs), particles move with a fixed speed along straight line segments and turn in new directions after random time intervals that are distributed according to a power law. Such LWs are thought to be an advantageous foraging and search strategy for organisms. While complex nervous systems are certainly capable of producing such behavior, it is not clear at present how single-cell organisms can generate the long-term correlated control signals required for a LW. Here, we construct a biochemical reaction system that generates long-time correlated concentration fluctuations of a signaling substance, with a tunable fractional exponent of the autocorrelation function. The network is based on well-known modules, and its basic function is highly robust with respect to the parameter settings.

Foraging success under uncertainty: search tradeoffs and optimal space use

Understanding the structural complexity and the main drivers of animal search behaviour is pivotal to foraging ecology. Yet, the role of uncertainty as a generative mechanism of movement patterns is poorly understood. Novel insights from search theory suggest that organisms should collect and assess new information from the environment by producing complex exploratory strategies. Based on an extension of the first passage time theory, and using simple equations and simulations, we unveil the elementary heuristics behind search behaviour. In particular, we show that normal diffusion is not enough for determining optimal exploratory behaviour but anomalous diffusion is required. Searching organisms go through two critical sequential phases (approach and detection) and experience fundamental search tradeoffs that may limit their encounter rates. Using experimental data, we show that biological search includes elements not fully considered in contemporary physical search theory. In particular, the need to consider search movement as a non-stationary process that brings the organism from one informational state to another. For example, the transition from remaining in an area to departing from it may occur through an exploratory state where cognitive search is challenged. Therefore, a more comprehensive view of foraging ecology requires including current perspectives about movement under uncertainty.

Rationalizing spatial exploration patterns of wild animals and humans through a temporal discounting framework

Understanding the exploration patterns of foragers in the wild provides fundamental insight into animal behavior. Recent experimental evidence has demonstrated that path lengths (distances between consecutive turns) taken by foragers are well fitted by a power law distribution. Numerous theoretical contributions have posited that "Lévy random walks"-which can produce power law path length distributions-are optimal for memoryless agents searching a sparse reward landscape. It is unclear, however, whether such a strategy is efficient for cognitively complex agents, from wild animals to humans. Here, we developed a model to explain the emergence of apparent power law path length distributions in animals that can learn about their environments. In our model, the agent's goal during search is to build an internal model of the distribution of rewards in space that takes into account the cost of time to reach distant locations (i.e., temporally discounting rewards). For an agent with such a goal, we find that an optimal model of exploration in fact produces hyperbolic path lengths, which are well approximated by power laws. We then provide support for our model by showing that humans in a laboratory spatial exploration task search space systematically and modify their search patterns under a cost of time. In addition, we find that path length distributions in a large dataset obtained from free-ranging marine vertebrates are well described by our hyperbolic model. Thus, we provide a general theoretical framework for understanding spatial exploration patterns of cognitively complex foragers.

Extending Lévy search theory from one to higher dimensions: Lévy walking favours the blind

A diverse range of organisms, including T cells, E. coli, honeybees, sharks, turtles, bony fish, jellyfish, wandering albatrosses and even human hunter-gatherers have movement patterns that can be approximated by Lévy walks (LW; sometimes called Lévy flights in the biological and ecological literature). These observations lend support to the 'Lévy flight foraging hypothesis' which asserts that natural selection should have led to adaptations for Lévy flight foraging, because Lévy flights can optimize search efficiencies. The hypothesis stems from a rigorous theory of one-dimensional searching and from simulation data for two-dimensional searching. The potential effectiveness of three-dimensional Lévy searches has not been examined but is central to a proper understanding of marine predators and T cells which have provided the most compelling empirical evidence for LW. Here I extend Lévy search theory from one to three dimensions. The new theory predicts that three-dimensional Lévy searching can be advantageous but only when targets are large compared with the perceptual range of the searchers, i.e. only when foragers are effectively blind and need to come into contact with a target to establish its presence. This may explain why effective blindness is a common factor among three-dimensional Lévy walkers.

Collective foraging in heterogeneous landscapes

Animals foraging alone are hypothesized to optimize the encounter rates with resources through Lévy walks. However, the issue of how the interactions between multiple foragers influence their search efficiency is still not completely understood. To address this, we consider a model to study the optimal strategy for a group of foragers searching for targets distributed heterogeneously. In our model, foragers move on a square lattice containing immobile but regenerative targets. At any instant, a forager is able to detect only those targets that happen to be in the same site. However, we allow the foragers to have information about the state of other foragers. A forager who has not detected any target walks towards the nearest location, where another forager has detected a target, with a probability exp(-αd), where d is the distance between the foragers and α is a parameter characterizing the propensity of the foragers to aggregate. The model reveals that neither overcrowding (α → 0) nor independent searching (α → ∞) is beneficial for the foragers. For a patchy distribution of targets, the efficiency is maximum for intermediate values of α. In addition, in the limit α → 0, the length of the walks can become scale-free.

Signatures of active and passive optimized Lévy searching in jellyfish

Some of the strongest empirical support for Lévy search theory has come from telemetry data for the dive patterns of marine predators (sharks, bony fishes, sea turtles and penguins). The dive patterns of the unusually large jellyfish Rhizostoma octopus do, however, sit outside of current Lévy search theory which predicts that a single search strategy is optimal. When searching the water column, the movement patterns of these jellyfish change over time. Movement bouts can be approximated by a variety of Lévy and Brownian (exponential) walks. The adaptive value of this variation is not known. On some occasions movement pattern data are consistent with the jellyfish prospecting away from a preferred depth, not finding an improvement in conditions elsewhere and so returning to their original depth. This 'bounce' behaviour also sits outside of current Lévy walk search theory. Here, it is shown that the jellyfish movement patterns are consistent with their using optimized 'fast simulated annealing'--a novel kind of Lévy walk search pattern--to locate the maximum prey concentration in the water column and/or to locate the strongest of many olfactory trails emanating from more distant prey. Fast simulated annealing is a powerful stochastic search algorithm for locating a global maximum that is hidden among many poorer local maxima in a large search space. This new finding shows that the notion of active optimized Lévy walk searching is not limited to the search for randomly and sparsely distributed resources, as previously thought, but can be extended to embrace other scenarios, including that of the jellyfish R. octopus. In the presence of convective currents, it could become energetically favourable to search the water column by riding the convective currents. Here, it is shown that these passive movements can be represented accurately by Lévy walks of the type occasionally seen in R. octopus. This result vividly illustrates that Lévy walks are not necessarily the result of selection pressures for advantageous searching behaviour but can instead arise freely and naturally from simple processes. It also shows that the family of Lévy walkers is vastly larger than previously thought and includes spores, pollens, seeds and minute wingless arthropods that on warm days disperse passively within the atmospheric boundary layer.

Anomalous diffusion and multifractality enhance mating encounters in the ocean

For millimeter-scale aquatic crustaceans such as copepods, ensuring reproductive success is a challenge as potential mates are often separated by hundreds of body lengths in a 3D environment. At the evolutionary scale, this led to the development of remote sensing abilities and behavioral strategies to locate, to track, and to capture a mate. Chemoreception plays a crucial role in increasing mate encounter rates through pheromone clouds and pheromone trails that can be followed over many body lengths. Empirical evidence of trail following behavior is, however, limited to laboratory experiments conducted in still water. An important open question concerns what happens in the turbulent waters of the surface ocean. We propose that copepods experience, and hence react to, a bulk-phase water pheromone concentration. Here we investigate the mating behavior of two key copepod species, Temora longicornis and Eurytemora affinis, to assess the role of background pheromone concentration and the relative roles played by males and females in mating encounters. We find that both males and females react to background pheromone concentration and exhibit both innate and acquired components in their mating strategies. The emerging swimming behaviors have stochastic properties that depend on pheromone concentration, sex, and species, are related to the level of reproductive experience of the individual tested, and significantly diverge from both the Lévy and Brownian models identified in predators searching for low- and high-density prey. Our results are consistent with an adaptation to increase mate encounter rates and hence to optimize reproductive fitness and success.

Selection pressures give composite correlated random walks Lévy walk characteristics

Composite correlated random walks have been posited as a strong alternative to Lévy walks as models of multi-scale forager movement patterns. Here it is shown that if plastic then intrinsic composite correlated random walks will, under selection pressures, evolve to resemble optimal Lévy walks when foraging is non-destructive. The fittest composite correlated random walkers are found to be those that come closest to being optimal Lévy walkers. This may explain why such a diverse range of foragers have movement patterns that can be approximated by optimal Lévy walks and shows that the 'Lévy-flight foraging' hypothesis has a broad hinterland. The new findings are consistent with recent observations of mussels Mytilus edulis and the Australian desert ant Melophorus bagoti which suggest that animals approximate a Lévy walk by adopting an intrinsic composite movement strategy with different modes.