When is it time to move to the next map? Optimal foraging in guided visual search

No matter what you do, travel is travel in visual foraging

In visual foraging, foragers collect multiple items from a series of visual displays (or "patches"). When the goal is to maximize the total or the rate of collection of target items, foragers must decide when to leave a depleted patch given that "traveling" from one patch to another incurs a temporal cost. In three experiments, we investigated whether the interposition of a secondary task during travel between patches in visual foraging altered patch-leaving behavior. Over the course of 10- or 30-minute experiments, participants foraged in simulated "berry patches" and traveled to the next patch at will. While they traveled, they either actively performed a secondary task or simply observed passing visual stimuli. Travel time was varied across conditions. The addition of a secondary task, regardless of its relevance to visual foraging, to traveling, or to both, did not impact patch-leaving times in the primary visual foraging task. In Experiment 1 and more weakly in Experiment 2, the patch-leaving decision was based on how long the travel took as predicted by the Marginal Value Theorem (MVT). In Experiment 3, however, patch-leaving did not depend on travel time. Participants 'overharvested' in a manner that suggests that they may have adopted rules different from those of MVT. Across all three experiments, patch-leaving did not depend on the nature of the travel.

Research on re-searching: interrupted foraging is not disrupted foraging

In classic visual search, observers typically search for the presence of a target in a scene or display. In foraging tasks, there may be multiple targets in the same display (or "patch"). Observers typically search for and collect these target items in one patch until they decide to leave that patch and move to the next one. This is a highly rule-governed behavior. The current study investigated whether these rules are disrupted when the foraging is interrupted in various manners. In Experiment 1, the foraging was briefly interrupted and then resumed in the same patch. In Experiments 2 and 3, the foraging in each patch either ended voluntarily or compulsorily after a fixed amount of time. In these cases, foraging resumed in a patch only after all patches were visited. Overall, the rules of foraging remained largely intact, though Experiment 2 shows that foraging rules can be overridden by the demand characteristics of the task. The results show that participants tended to perform approximately consistently despite interruptions. The results suggest that foraging behavior in a relatively simple foraging environment is resilient and not easily disrupted by interruption.

Toward viewing behavior for aerial scene categorization

Previous work has demonstrated similarities and differences between aerial and terrestrial image viewing. Aerial scene categorization, a pivotal visual processing task for gathering geoinformation, heavily depends on rotation-invariant information. Aerial image-centered research has revealed effects of low-level features on performance of various aerial image interpretation tasks. However, there are fewer studies of viewing behavior for aerial scene categorization and of higher-level factors that might influence that categorization. In this paper, experienced subjects' eye movements were recorded while they were asked to categorize aerial scenes. A typical viewing center bias was observed. Eye movement patterns varied among categories. We explored the relationship of nine image statistics to observers' eye movements. Results showed that if the images were less homogeneous, and/or if they contained fewer or no salient diagnostic objects, viewing behavior became more exploratory. Higher- and object-level image statistics were predictive at both the image and scene category levels. Scanpaths were generally organized and small differences in scanpath randomness could be roughly captured by critical object saliency. Participants tended to fixate on critical objects. Image statistics included in this study showed rotational invariance. The results supported our hypothesis that the availability of diagnostic objects strongly influences eye movements in this task. In addition, this study provides supporting evidence for Loschky et al.'s (Journal of Vision, 15(6), 11, 2015) speculation that aerial scenes are categorized on the basis of image parts and individual objects. The findings were discussed in relation to theories of scene perception and their implications for automation development.

Evidence for optimal semantic search throughout adulthood

As people age, they learn and store new knowledge in their semantic memory. Despite learning a tremendous amount of information, people can still recall information relevant to the current situation with ease. To accomplish this, the mind must efficiently organize and search a vast store of information. It also must continue to retrieve information effectively despite changes in cognitive mechanisms due to healthy aging, including a general slowing in information processing and a decline in executive functioning. How effectively does the mind of an individual adjust its search to account for changes due to aging? We tested 746 people ages 25 through 69 on a semantic fluency task (free listing animals) and found that, on average, retrieval follows an optimal path through semantic memory. Participants tended to list a sequence of semantically related animals (e.g., lion, tiger, puma) before switching to a semantically unrelated animal (e.g., whale). We found that the timing of these transitions to semantically unrelated animals was remarkably consistent with an optimal strategy for maximizing the overall rate of retrieval (i.e., the number of animals listed per unit time). Age did not affect an individual's deviation from the optimal strategy given their general performance, suggesting that people adapt and continue to search memory optimally throughout their lives. We argue that this result is more likely due to compensating for a general slowing than a decline in executive functioning.

Testing a relational account of search templates in visual foraging

Search templates guide human visual attention toward relevant targets. Templates are often seen as encoding exact target features, but recent studies suggest that templates rather contain "relational properties" (e.g., they facilitate "redder" stimuli instead of specific hues of red). Such relational guidance seems helpful in naturalistic searches where illumination or perspective renders exact feature values unreliable. So far relational guidance has only been demonstrated in rather artificial single-target search tasks with briefly flashed displays. Here, we investigate whether relational guidance also occurs when humans interact with the search environment for longer durations to collect multiple target elements. In a visual foraging task, participants searched for and collected multiple targets among distractors of different relationships to the target colour. Distractors whose colour differed from the environment in the same direction as the targets reduced foraging efficiency to the same amount as distractors whose colour matched the target colour. Distractors that differed by the same colour distance but in the opposite direction of the target colour did not reduce efficiency. These findings provide evidence that search templates encode relational target features in naturalistic search tasks and suggest that attention guidance based on relational features is a common mode in dynamic, real-world search environments.

Using Gaze for Behavioural Biometrics

A principled approach to the analysis of eye movements for behavioural biometrics is laid down. The approach grounds in foraging theory, which provides a sound basis to capture the uniqueness of individual eye movement behaviour. We propose a composite Ornstein-Uhlenbeck process for quantifying the exploration/exploitation signature characterising the foraging eye behaviour. The relevant parameters of the composite model, inferred from eye-tracking data via Bayesian analysis, are shown to yield a suitable feature set for biometric identification; the latter is eventually accomplished via a classical classification technique. A proof of concept of the method is provided by measuring its identification performance on a publicly available dataset. Data and code for reproducing the analyses are made available. Overall, we argue that the approach offers a fresh view on either the analyses of eye-tracking data and prospective applications in this field.

Humans trade off search costs and accuracy in a combined visual search and perceptual task

To interact with one's environment, relevant objects have to be selected as targets for saccadic eye movements. Previous studies have demonstrated that factors such as visual saliency and reward influence saccade target selection, and that humans can dynamically trade off these factors to maximize expected value during visual search. However, expected value in everyday situations not only depends on saliency and reward, but also on the required time to find objects, and the likelihood of a successful object-interaction after search. Here we studied whether search costs and the accuracy to discriminate an object feature can be traded off to maximize expected value. We designed a combined visual search and perceptual discrimination task, where participants chose whether to search for an easy- or difficult-to-discriminate target in search displays populated by distractors that shared features with either the easy or the difficult target. Participants received a monetary reward for correct discriminations and were given limited time to complete as many trials as they could. We found that participants considered their discrimination performance and the search costs when choosing targets and, by this, maximized expected value. However, the accumulated reward was constrained by noise in both the choice of which target to search for, and which elements to fixate during search. We conclude that humans take into account the prospective search time and the likelihood of successful a object-interaction, when deciding what to search for. However, search performance is constrained by noise in decisions about what to search for and how to search for it.

Into the Wild: Biological Timing in Natural Environments

Biological timing (including circadian and interval timing) has mainly focused on rigorously controlled laboratory experiments. There are relatively few studies looking into interval timing behaviors in the wild, which could be understandable due to the complexity of the experimental design but are definitely needed in order to comprehend the adaptive value of such behavior. In this opinion paper we review some of the literature regarding timing observations under field conditions, including reports from birds and mammals, and propose a call-to-action to think about the need of a more naturalistic interpretation of time production and perception, as well as the advantage of designing more ‘natural’ settings in the laboratory.

Sources of variation in search and foraging: A theoretical perspective

Search-the problem of exploring a space of alternatives to identify target goals-is a fundamental behaviour for many species. Although its foundation lies in foraging, most studies of human search behaviour have been directed towards understanding the attentional mechanisms that underlie the efficient visual exploration of two-dimensional (2D) scenes. With this review, we aim to characterise how search behaviour can be explained across a wide range of contexts, environments, spatial scales, and populations, both typical and atypical. We first consider the generality of search processes across psychological domains. We then review studies of interspecies differences in search. Finally, we explore in detail the individual and contextual variables that affect visual search and related behaviours in established experimental psychology paradigms. Despite the heterogeneity of the findings discussed, we identify that variations in control processes, along with the ability to regulate behaviour as a function of the structure of search space and the sampling processes adopted, to be central to explanations of variations in search behaviour. We propose a tentative theoretical model aimed at integrating these notions and close by exploring questions that remain unaddressed.

Individual and collective foraging in autonomous search agents with human intervention

Humans and other complex organisms exhibit intelligent behaviors as individual agents and as groups of coordinated agents. They can switch between independent and collective modes of behavior, and flexible switching can be advantageous for adapting to ongoing changes in conditions. In the present study, we investigated the flexibility between independent and collective modes of behavior in a simulated social foraging task designed to benefit from both modes: distancing among ten foraging agents promoted faster detection of resources, whereas flocking promoted faster consumption. There was a tradeoff between faster detection versus faster consumption, but both factors contributed to foraging success. Results showed that group foraging performance among simulated agents was enhanced by loose coupling that balanced distancing and flocking among agents and enabled them to fluidly switch among a variety of groupings. We also examined the effects of more sophisticated cognitive capacities by studying how human players improve performance when they control one of the search agents. Results showed that human intervention further enhanced group performance with loosely coupled agents, and human foragers performed better when coordinating with loosely coupled agents. Humans players adapted their balance of independent versus collective search modes in response to the dynamics of simulated agents, thereby demonstrating the importance of adaptive flexibility in social foraging.

Spatially and temporally distributed data foraging decisions in disciplinary field science

How do scientists generate and weight candidate queries for hypothesis testing, and how does learning from observations or experimental data impact query selection? Field sciences offer a compelling context to ask these questions because query selection and adaptation involves consideration of the spatiotemporal arrangement of data, and therefore closely parallels classic search and foraging behavior. Here we conduct a novel simulated data foraging study-and a complementary real-world case study-to determine how spatiotemporal data collection decisions are made in field sciences, and how search is adapted in response to in-situ data. Expert geoscientists evaluated a hypothesis by collecting environmental data using a mobile robot. At any point, participants were able to stop the robot and change their search strategy or make a conclusion about the hypothesis. We identified spatiotemporal reasoning heuristics, to which scientists strongly anchored, displaying limited adaptation to new data. We analyzed two key decision factors: variable-space coverage, and fitting error to the hypothesis. We found that, despite varied search strategies, the majority of scientists made a conclusion as the fitting error converged. Scientists who made premature conclusions, due to insufficient variable-space coverage or before the fitting error stabilized, were more prone to incorrect conclusions. We found that novice undergraduates used the same heuristics as expert geoscientists in a simplified version of the scenario. We believe the findings from this study could be used to improve field science training in data foraging, and aid in the development of technologies to support data collection decisions.

Gazing at Social Interactions Between Foraging and Decision Theory

Finding the underlying principles of social attention in humans seems to be essential for the design of the interaction between natural and artificial agents. Here, we focus on the computational modeling of gaze dynamics as exhibited by humans when perceiving socially relevant multimodal information. The audio-visual landscape of social interactions is distilled into a number of multimodal patches that convey different social value, and we work under the general frame of foraging as a tradeoff between local patch exploitation and landscape exploration. We show that the spatio-temporal dynamics of gaze shifts can be parsimoniously described by Langevin-type stochastic differential equations triggering a decision equation over time. In particular, value-based patch choice and handling is reduced to a simple multi-alternative perceptual decision making that relies on a race-to-threshold between independent continuous-time perceptual evidence integrators, each integrator being associated with a patch.

Foraging behavior in visual search: A review of theoretical and mathematical models in humans and animals

Visual search (VS) is a fundamental task in daily life widely studied for over half a century. A variant of the classic paradigm-searching one target among distractors-requires the observer to look for several (undetermined) instances of a target (so-called foraging) or several targets that may appear an undefined number of times (recently named as hybrid foraging). In these searches, besides looking for targets, the observer must decide how much time is needed to exploit the area, and when to quit the search to eventually explore new search options. In fact, visual foraging is a very common search task in the real world, probably involving additional cognitive functions than typical VS. It has been widely studied in natural animal environments, for which several mathematical models have been proposed, and just recently applied to humans: Lévy processes, composite and area-restricted search models, marginal value theorem, and Bayesian learning (among others). We conducted a systematic search in the literature to understand those mathematical models and study its applicability in human visual foraging. The review suggests that these models might be the first step, but they seem to be limited to fully comprehend foraging in visual search. There are essential variables involving human visual foraging still to be established and understood. Indeed, a jointly theoretical interpretation based on the different models reviewed could better account for its understanding. In addition, some other relevant variables, such as certain individual differences or time perception might be crucial to understanding visual foraging in humans.

Looking ahead: When do you find the next item in foraging visual search?

Many real-world visual tasks involve searching for multiple instances of a target (e.g., picking ripe berries). What strategies do observers use when collecting items in this type of search? Do they wait to finish collecting the current item before starting to look for the next target, or do they search ahead for future targets? We utilized behavioral and eye-tracking measures to distinguish between these two possibilities in foraging search. Experiment 1 used a color wheel technique in which observers searched for T shapes among L shapes while all items independently cycled through a set of colors. Trials were abruptly terminated, and observers reported both the color and location of the next target that they intended to click. Using observers’ color reports to infer target-finding times, we demonstrate that observers found the next item before the time of the click on the current target. We validated these results in Experiment 2 by recording fixation locations around the time of each click. Experiment 3 utilized a different procedure, in which all items were intermittently occluded during the trial. We then calculated a distribution of when targets were visible around the time of each click, allowing us to infer when they were most likely found. In a fourth and final experiment, observers indicated the locations of multiple future targets after the search was abruptly terminated. Together, our results provide converging evidence to demonstrate that observers can find the next target before collecting the current target and can typically forage one to two items ahead.

Guidance and selection history in hybrid foraging visual search

In Hybrid Foraging tasks, observers search for multiple instances of several types of target. Collecting all the dirty laundry and kitchenware out of a child's room would be a real-world example. How are such foraging episodes structured? A series of four experiments shows that selection of one item from the display makes it more likely that the next item will be of the same type. This pattern holds if the targets are defined by basic features like color and shape but not if they are defined by their identity (e.g., the letters p & d). Additionally, switching between target types during search is expensive in time, with longer response times between successive selections if the target type changes than if they are the same. Finally, the decision to leave a screen/patch for the next screen in these foraging tasks is imperfectly consistent with the predictions of optimal foraging theory. The results of these hybrid foraging studies cast new light on the ways in which prior selection history guides subsequent visual search in general.

Human Foragers: Searchers by Nature and Experience

Diverse studies of human foraging have revealed behavioral strategies that may have evolved as adaptations for foraging. Here, we used an outdoor experimental search task to explore the effect of three sources of information on participants' performance: (i) information obtained directly from performing a search, (ii) information obtained prior to testing in the form of a distilled snippet of knowledge intended to experimentally simulate information acquired culturally about the environment, and (iii) information obtained from experience of foraging for natural resources for economic gain. We found that (i) immediate searching experience improved performance from the beginning to the end of the short, 2-min task, (ii) information priming improved performance notably from the very beginning of the task, and (iii) natural resource foraging experience improved performance to a lesser extent. Our results highlight the role of culturally transmitted information as well as the presence of mechanisms to rapidly integrate and implement new information into searching choices, which ultimately influence performance in a foraging task.

A simple survey protocol for assessing terrestrial biodiversity in a broad range of ecosystems

Finding standard cost-effective methods for monitoring biodiversity is challenging due to trade-offs between survey costs (including expertise), specificity, and range of applicability. These trade-offs cause a lack of comparability among datasets collected by ecologists and conservationists, which is most regrettable in taxonomically demanding work on megadiverse inconspicuous taxon groups. We have developed a site-scale survey method for diverse sessile land organisms, which can be analyzed over multiple scales and linked with ecological insights and management. The core idea is that field experts can effectively allocate observation effort when the time, area, and priority sequence of tasks are fixed. We present the protocol, explain its specifications (taxon group; expert qualification; plot size; effort) and applications based on >800 original surveys of four taxon groups; and we analyze its effectiveness using data on polypores in hemiboreal and tropical forests. We demonstrate consistent effort-species richness curves and among-survey variation in contrasting ecosystems, and high effectiveness compared with casual observations both at local and regional scales. Bias related to observer experience appeared negligible compared with typical assemblage variation. Being flexible in terms of sampling design, the method has enabled us to compile data from various projects to assess conservation status and habitat requirements of most species (specifically rarities and including discovery of new species); also, when linked with site descriptions, to complete environmental assessments and select indicator species for management. We conclude that simple rules can significantly improve expert-based biodiversity surveys. Ideally, define (i) a common plot size that addresses multiple taxon groups and management goals; (ii) taxon groups based on field expertise and feasible number of species; (iii) sufficient and practical search time; (iv) a procedure for recording within-plot heterogeneity. Such a framework, combined with freedom to allocate effort on-site, helps utilizing full expertise of observers without losing technical rigor.

Getting satisfied with "satisfaction of search": How to measure errors during multiple-target visual search

Visual search studies are common in cognitive psychology, and the results generally focus upon accuracy, response times, or both. Most research has focused upon search scenarios where no more than 1 target will be present for any single trial. However, if multiple targets can be present on a single trial, it introduces an additional source of error because the found target can interfere with subsequent search performance. These errors have been studied thoroughly in radiology for decades, although their emphasis in cognitive psychology studies has been more recent. One particular issue with multiple-target search is that these subsequent search errors (i.e., specific errors which occur following a found target) are measured differently by different studies. There is currently no guidance as to which measurement method is best or what impact different measurement methods could have upon various results and conclusions. The current investigation provides two efforts to address these issues. First, the existing literature is reviewed to clarify the appropriate scenarios where subsequent search errors could be observed. Second, several different measurement methods are used with several existing datasets to contrast and compare how each method would have affected the results and conclusions of those studies. The evidence is then used to provide appropriate guidelines for measuring multiple-target search errors in future studies.

Use-inspired basic research in medical image perception

This journal is dedicated to "use-inspired basic research" where a problem in the world shapes the hypotheses for a study in the laboratory. This brief review presents several examples of "use-inspired basic research" in the area of medical image perception. These are cases where the field of radiology raises an interesting issue in visual cognition. Basic research on those issues may then lead to proposals to improve performance on clinical tasks in medical image perception. Of the six examples given here, the first three ask essentially perceptual questions: How can stereopsis improve medical image perception? How shall we assess the tradeoff between radiation dose and image quality? How does the choice of colors change the interpretation of medical images? The second three examples address attentional issues in those aspects of radiology that can be described as visual search problems: Can eye tracking help us understand errors in radiologic search? What happens if the number of targets in an image is unknown? What happens if, as in radiology screening programs, the target of search is very rare?

Making Sense of Real-World Scenes

To interact with the world, we have to make sense of the continuous sensory input conveying information about our environment. A recent surge of studies has investigated the processes enabling scene understanding, using increasingly complex stimuli and sophisticated analyses to highlight the visual features and brain regions involved. However, there are two major challenges to producing a comprehensive framework for scene understanding. First, scene perception is highly dynamic, subserving multiple behavioral goals. Second, a multitude of different visual properties co-occur across scenes and may be correlated or independent. We synthesize the recent literature and argue that for a complete view of scene understanding, it is necessary to account for both differing observer goals and the contribution of diverse scene properties.