Peck tracking: a method for localizing critical features within complex pictures for pigeons

Dynamically occluded action recognition by pigeons

Identifying the behaviors of organisms is essential for an animal's survival. This ability is particularly challenged when the "actors" are dynamically occluded by other objects and become fragmented as they move through an environment. Even when fragmented in time and across space, humans readily recognize the behavior of these dynamically occluded objects and actors. How animals process such fragmented information, especially when involving motion, remains uncertain. In three experiments, we investigated the ability of six pigeons to discriminate between the running and walking actions of digital animal models when dynamically occluded. The pigeons were tested in a go/no-go procedure using three models that transited behind multiple occluders in a semirealistic scene. Without ever seeing the entirety of the animal model at one time, all the pigeons learned to discriminate among these two behaviors. This discrimination transferred to an unfamiliar model, transit direction, transiting rates, camera perspectives, and occluders. Tests with different static and dynamic features indicated that the pigeons relied on motion features for the discrimination, especially articulated motion. These experiments demonstrate that pigeons, like humans, can discriminate actions even when their view of the actor is fragmented in time and space.

Visual categories and concepts in the avian brain

Birds are excellent model organisms to study perceptual categorization and concept formation. The renewed focus on avian neuroscience has sparked an explosion of new data in the field. At the same time, our understanding of sensory and particularly visual structures in the avian brain has shifted fundamentally. These recent discoveries have revealed how categorization is mediated in the avian brain and has generated a theoretical framework that goes beyond the realm of birds. We review the contribution of avian categorization research-at the methodical, behavioral, and neurobiological levels. To this end, we first introduce avian categorization from a behavioral perspective and the common elements model of categorization. Second, we describe the functional and structural organization of the avian visual system, followed by an overview of recent anatomical discoveries and the new perspective on the avian 'visual cortex'. Third, we focus on the neurocomputational basis of perceptual categorization in the bird's visual system. Fourth, an overview of the avian prefrontal cortex and the prefrontal contribution to perceptual categorization is provided. The fifth section outlines how asymmetries of the visual system contribute to categorization. Finally, we present a mechanistic view of the neural principles of avian visual categorization and its putative extension to concept learning.

Subsampling of cues in associative learning

Theories of learning distinguish between elemental and configural stimulus processing depending on whether stimuli are processed independently or as whole configurations. Evidence for elemental processing comes from findings of summation in animals where a compound of two dissimilar stimuli is deemed to be more predictive than each stimulus alone, whereas configural processing is supported by experiments using similar stimuli in which summation is not found. However, in humans the summation effect is robust and impervious to similarity manipulations. In three experiments in human predictive learning, we show that summation can be obliterated when partially reinforced cues are added to the summands in training and tests. This lack of summation only holds when the partially reinforced cues are similar to the reinforced cues (experiment 1) and seems to depend on participants sampling only the most salient cue in each trial (experiments 2a and 2b) in a sequential visual search process. Instead of attributing our and others' instances of lack of summation to the customary idea of configural processing, we offer a formal subsampling rule that might be applied to situations in which the stimuli are hard to parse from each other.

Digital embryos: a novel technical approach to investigate perceptual categorization in pigeons (Columba livia) using machine learning

Pigeons are classic model animals to study perceptual category learning. To achieve a deeper understanding of the cognitive mechanisms of categorization, a careful consideration of the employed stimulus material and a thorough analysis of the choice behavior is mandatory. In the present study, we combined the use of “virtual phylogenesis”, an evolutionary algorithm to generate artificial yet naturalistic stimuli termed digital embryos and a machine learning approach on the pigeons’ pecking responses to gain insight into the underlying categorization strategies of the animals. In a forced-choice procedure, pigeons learned to categorize these stimuli and transferred their knowledge successfully to novel exemplars. We used peck tracking to identify where on the stimulus the animals pecked and further investigated whether this behavior was indicative of the pigeon’s choice. Going beyond the classical analysis of the binary choice, we were able to predict the presented stimulus class based on pecking location using a k-nearest neighbor classifier, indicating that pecks are related to features of interest. By analyzing error trials with this approach, we further identified potential strategies of the pigeons to discriminate between stimulus classes. These strategies remained stable during category transfer, but differed between individuals indicating that categorization learning is not limited to a single learning strategy.

RUBubbles as a novel tool to study categorization learning

Grouping objects into discrete categories affects how we perceive the world and represents a crucial element of cognition. Categorization is a widespread phenomenon that has been thoroughly studied. However, investigating categorization learning poses several requirements on the stimulus set in order to control which stimulus feature is used and to prevent mere stimulus-response associations or rote learning. Previous studies have used a wide variety of both naturalistic and artificial categories, the latter having several advantages such as better control and more direct manipulation of stimulus features. We developed a novel stimulus type to study categorization learning, which allows a high degree of customization at low computational costs and can thus be used to generate large stimulus sets very quickly. 'RUBubbles' are designed as visual artificial category stimuli that consist of an arbitrary number of colored spheres arranged in 3D space. They are generated using custom MATLAB code in which several stimulus parameters can be adjusted and controlled separately, such as number of spheres, position in 3D-space, sphere size, and color. Various algorithms for RUBubble generation can be combined with distinct behavioral training protocols to investigate different characteristics and strategies of categorization learning, such as prototype- vs. exemplar-based learning, different abstraction levels, or the categorization of a sensory continuum and category exceptions. All necessary MATLAB code is freely available as open-source code and can be customized or expanded depending on individual needs. RUBubble stimuli can be controlled purely programmatically or via a graphical user interface without MATLAB license or programming experience.

Focusing and shifting attention in pigeon category learning

Adaptively and flexibly modifying one's behavior depending on the current demands of the situation is a hallmark of executive function. Here, we examined whether pigeons could flexibly shift their attention from one set of features that were relevant in one categorization task to another set of features that were relevant in a second categorization task. Critically, members of both sets of features were available on every training trial, thereby requiring that attention be adaptively deployed on a trial-by-trial basis based on contextual information. The pigeons not only learned to correctly categorize the stimuli but, as training progressed, they concentrated their pecks to the training stimuli (a proxy measure for attention) on those features that were relevant in a specific context. The pigeons selectively tracked the features that were relevant in Context 1-but were irrelevant in Context 2-and they selectively tracked the features that were relevant in Context 2-but were irrelevant in Context 1. This adept feature tracking requires disengaging attention from a previously relevant feature and shifting attention to a previously ignored feature on a trial-by-trial basis. Pigeons' adaptive and flexible performance provides strong empirical support for the involvement of focusing and shifting attention under exceptionally challenging training conditions. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Selective and distributed attention in human and pigeon category learning

Attention to relevant stimulus features in a categorization task helps to optimize performance. However, the relationship between attention and categorization is not fully understood. For example, even when human adults and young children exhibit comparable categorization behavior, adults tend to attend selectively during learning, whereas young children tend to attend diffusely (Deng & Sloutsky, 2016). Here, we used a comparative approach to investigate the link between attention and categorization in two different species. Given the noteworthy categorization ability of avian species, we compared the attentional profiles of pigeons and human adults. We gave human adults (Experiment 1) and pigeons (Experiment 2) a categorization task that could be learned on the basis of either one deterministic feature (encouraging selective attention) or multiple probabilistic features (encouraging distributed attention). Both humans and pigeons relied on the deterministic feature to categorize the stimuli, albeit humans did so to a much greater degree. Furthermore, computational modeling revealed that most of the adults exhibited maximal selectivity, whereas pigeons tended to distribute their attention among several features. Our findings indicate that human adults focus their attention on deterministic information and filter less predictive information, but pigeons do not. Implications for the underlying brain mechanisms of attention and categorization are discussed.

Pigeons process actor-action configurations more readily than bystander-action configurations

Behavior requires an actor. Two experiments using complex conditional action discriminations examined whether pigeons privilege information related to the digital actor who is engaged in behavior. In Experiment 1, each of two video displays contained a digital model, one an actor engaged in one of two behaviors (Indian dance or martial arts) and one a neutrally posed bystander. To correctly classify the display, the pigeons needed to conditionally process the action in conjunction with distinctive physical features of the actor or the bystander. Four actor-conditional pigeons learned to correctly discriminate the actions based on the identity of the actors, whereas four bystander-conditional birds failed to learn. Experiment 2 established that this failure was not due to the latter group's inability to spatially integrate information across the distance between the two models. Potentially, the colocalization of the relevant model identity and the action was critical due to a fundamental configural or integral representation of these properties. These findings contribute to our understanding of the evolution of action recognition, the recognition of social behavior, and forms of observational learning by animals.

Relative reinforcer rates determine pigeons' attention allocation when separately trained stimuli are presented together

Previous research suggests that organisms allocate more attention to stimuli associated with higher reinforcer rates. This finding has been replicated several times when stimuli are trained together as compounds, but not in other procedures. Thus, the generality of the relation between relative reinforcer rates and divided attention is not well established. Therefore, we investigated whether relative reinforcer rates determine attention allocation when stimuli are trained separately and then encountered together. Pigeons learned to associate two colors and two frequencies of key light on/off alternation with a left or right comparison key in a symbolic 0-s delayed matching-to-sample task. Across conditions, we varied the probability of reinforcement associated with each stimulus dimension during training. After training, we introduced test trials in which a color and flash-frequency stimulus were presented simultaneously. During sample-stimulus presentation in test trials, all pigeons preferred the stimulus associated with the higher reinforcer rate, suggesting that more attention was allocated to that stimulus. Interestingly, such attention allocation did not result in preference for the comparison that matched that stimulus. Instead, all pigeons preferred the comparison that was physically closer to the stimulus associated with the higher reinforcer rate, suggesting that comparison choice was controlled by the location of that stimulus. Nevertheless, overall, our results provide the first evidence that relative reinforcer rates determine divided attention between separately trained stimuli and thus demonstrate the generality of the relation between relative reinforcement and attention allocation. We suggest several avenues for future research to establish further the generality of this relation.

The role of category density in pigeons' tracking of relevant information

Prior categorization studies have shown that pigeons reliably track features that are relevant to category discrimination. In these studies, category exemplars contained two relevant and two irrelevant features; therefore, category density (specifically, the relevant to irrelevant information ratio) was relatively high. Here, we manipulated category density both between and within subjects by keeping constant the amount of relevant information (one feature) and varying the amount of irrelevant information (one or three features). One group of pigeons started with low-density training, then proceeded to high-density training, and finally returned to low-density training (Low-High-Low); a second group of pigeons started with high-density training and then proceeded to low-density training (High-Low). The statistical density of the category exemplars had a large effect on pigeons' performance. Training with high-density exemplars greatly benefitted category learning. Accuracy rose faster and to a higher level with high-density training than with low-density training; the percentage of relevant pecks showed a very similar pattern. In addition, high-density training (in the Low-High-Low group) led to an increase in performance on the more difficult low-density task, an observation reminiscent of the easy-to-hard effect. These results illuminate factors affecting pigeons' accuracy and tracking of relevant information in visual categorization.

Pigeons simultaneously attend to static and dynamic features of complex displays

The simultaneous processing and attention to temporally dynamic and static features remains an open and important question in theories of avian visual cognition. Here, four pigeons (Columba livia) learned to discriminate complex displays involving concurrently available static and dynamic features. These displays consisted of 20 elements built from combinations of two, binary-valued, static visual dimensions: red vs. green element color, large vs. small element size; and two binary-valued dynamic dimensions; fast vs slow element motion, right vs up motion direction. One combination of these four features was reinforced on a VI schedule. The remaining 15 combinations of element color, size, speed and direction were never reinforced. During acquisition, all four dimensions were simultaneously discriminated. Varying the number of elements revealed that a single element was sufficient to support discrimination of all four dimensions. The pigeons agreed on the relative discriminability of stimuli within and across the different dimensions, with the difference in motion direction being the hardest for all birds. Redundant facilitation suggested rapid, perhaps parallel, processing of both dynamic and static features. No attentional trade-offs between dynamic or static dimensions were observed. These results agree with theories of avian vision employing the notion of multiple independent channels for different types of information.

Feature predictiveness and selective attention in pigeons' categorization learning

Prior categorization studies have shown that pigeons reliably track features that are perfect predictors of category membership (Castro & Wasserman, 2014, 2016a). One might further ask whether pigeons would also track features that are relevant, but imperfect predictors of category membership. In our present project, pigeons had to categorize multiple exemplars from 2 different artificial categories, in which the exemplars were composed of 4 different features that were associated with 1 of 2 different report responses. Each exemplar contained 1 feature that perfectly predicted category membership; 1 feature that imperfectly predicted category membership; and, 2 irrelevant features that did not predict category membership. We monitored pigeons' choice accuracy as well as the location of their pecks to each of the 4 exemplar features to determine to which attributes the birds attended. As categorization accuracy rose, pecks to the perfect predictor of each category rose as well. Pigeons also showed evidence of attending more to the imperfect predictor than to the irrelevant features, but to a lesser degree. Overall, our results provide evidence of selective attention in pigeons' categorization behavior. (PsycINFO Database Record

Attentional shifts in categorization learning: Perseveration but not learned irrelevance

Once a categorization task has been mastered, if features that once were relevant become irrelevant and features that once were irrelevant become relevant, a decrement in performance-a shift cost-is typically observed. This shift cost may reflect the involvement of two distinguishable factors: the inability to release attention from a previously relevant feature (i.e., attentional perseveration) and/or the inability to re-engage attention to a previously irrelevant feature (i.e., learned irrelevance). Here, we examined the nature of this shift cost in pigeons. We gave four groups of pigeons a categorization task in which we monitored their choice accuracy; at the same time, we tracked the location of their pecks to the relevant and irrelevant attributes of the stimuli to determine to which attributes the birds were attending during the course of learning. After identical training in Phase 1, the roles of the relevant/irrelevant features were changed in Phase 2, so that one group could show only learned irrelevance, a second group could show only attentional perseverance, a third group could show both, and a fourth control group could show neither of these effects. Results disclosed evidence of attentional perseverance, but no evidence of learned irrelevance, either in accuracy or in relevant feature tracking. In addition, we determined that pigeons' allocation of attention to the relevant features followed rather than preceded an increase in choice accuracy. Overall, our findings are best explained by theories which propose that attention is learned and deployed to those features that prove to be reliable predictors of the correct categorization response (e.g., George and Pearce, 2012; Kruschke, 2001; Mackintosh, 1975).

Categorization of birds, mammals, and chimeras by pigeons

Identifying critical features that control categorization of complex polymorphous pictures by animals remains a challenging and important problem. Toward this goal, experiments were conducted to isolate the properties controlling the categorization of two pictorial categories by pigeons. Pigeons were trained in a go/no-go task to categorize black and white line drawings of birds and mammals. They were then tested with a variety of familiar and novel exemplars of these categories to examine the features controlling this categorization. These tests suggested the pigeons were segregating and using the principal axis of orientation of the animal figures as the primary means of discriminating each category, although other categorical and item-specific cues were likely involved. This perceptual/cognitive reduction of the categorical stimulus space to a few visual features or dimensions is likely a characteristic of this species' processing of complex pictorial discrimination problems and is a critical property for theoretical accounts of this behavior.

View-invariance learning in object recognition by pigeons depends on error-driven associative learning processes

A model hypothesizing that basic mechanisms of associative learning and generalization underlie object categorization in vertebrates can account for a large body of animal and human data. Here, we report two experiments which implicate error-driven associative learning in pigeons' recognition of objects across changes in viewpoint. Experiment 1 found that object recognition across changes in viewpoint depends on how well each view predicts reward. Analyses of generalization performance, spatial position of pecks to images, and learning curves all showed behavioral patterns analogous to those found in prior studies of relative validity in associative learning. In Experiment 2, pigeons were trained to recognize objects from multiple viewpoints, which usually promotes robust performance at novel views of the trained objects. However, when the objects possessed a salient, informative metric property for solving the task, the pigeons did not show view-invariant recognition of the training objects, a result analogous to the overshadowing effect in associative learning.

Overt attention toward oriented objects in free-viewing barn owls

Visual saliency based on orientation contrast is a perceptual product attributed to the functional organization of the mammalian brain. We examined this visual phenomenon in barn owls by mounting a wireless video microcamera on the owls' heads and confronting them with visual scenes that contained one differently oriented target among similarly oriented distracters. Without being confined by any particular task, the owls looked significantly longer, more often, and earlier at the target, thus exhibiting visual search strategies so far demonstrated in similar conditions only in primates. Given the considerable differences in phylogeny and the structure of visual pathways between owls and humans, these findings suggest that orientation saliency has computational optimality in a wide variety of ecological contexts, and thus constitutes a universal building block for efficient visual information processing in general.

Monitoring same/different discrimination behavior in time and space: finding differences and anticipatory discrimination behavior

Discrimination behavior in a standard, two-alternative forced choice same/different task is usually measured by the pigeon's pecking one or the other of two arbitrary report areas. We found that pigeons make anticipatory, discriminative responses to the visual display during the stimulus observing period prior to the availability of the report areas; the spatial distribution of these anticipatory discriminative responses strongly correlated with the upcoming choice response. These anticipatory pecks provide evidence that the process of discrimination occurs well before the moment of choice and that key aspects of this process can be revealed by looking at the distribution of observing responses. We also manipulated the variability of the displayed items to study the nature of these anticipatory responses; again, the spatial distribution of responding during the stimulus observing period strongly correlated with the upcoming choice response. The distribution of these prechoice pecks supports the theory that pigeons search for differences in the displayed items. If differences are found, then pigeons prepare to report "different"; if not, then they report "same."