Autoshaping the pigeon's gape response: acquisition and topography as a function of reinforcer type and magnitude

The Relevance of Operant Behavior in Conceptualizing the Psychological Well-Being of Captive Animals

The term "psychological well-being" is used in reference to husbandry with animals in human care settings such as research, agriculture, and zoos. This article seeks to clarify and conceptualize the term based upon two approaches that draw from several bodies of literature: the experimental analysis of behavior, experimental psychology, animal welfare and husbandry, farm animal behavior, zoo husbandry, and ethology. One approach focuses on the presence of problem behavior such as stereotypies, depressive-like behavior, and aggression, and emphasizes the conditions under which aberrant behavior in animals under human care occurs. The second approach examines what might be considered wellness by emphasizing opportunities to engage with its environment, or the absence of such opportunities, even if problematic behavior is not exhibited. Here, access to an interactive environment is relatively limited so opportunities for operant (voluntary) behavior could be considered. Designing for operant behavior provides opportunities for variability in both behavior and outcomes. Operant behavior also provides control over the environment, a characteristic that has been a core assumption of well-being. The importance of interactions with one's environment is especially evident in observations that animals prefer opportunities to work for items necessary for sustenance, such as food, over having them delivered freely. These considerations raise the importance of operant behavior to psychological well-being, especially as benefits to animals under human care.

Translations in Stimulus-Stimulus Pairing: Autoshaping of Learner Vocalizations

Stimulus-stimulus pairing (SSP) is a procedure used by behavior analysis practitioners that capitalizes on respondent conditioning processes to elicit vocalizations. These procedures usually are implemented only after other, more customary methods (e.g., standard echoic training via modeling) have been exhausted. Unfortunately, SSP itself has mixed research support, probably because certain as-yet-unidentified procedural variations are more effective than others. Even when SSP produces (or increases) vocalizations, its effects can be short-lived. Although specific features of SSP differ across published accounts, fundamental characteristics include presentation of a vocal stimulus proximal with presentation of a preferred item. In the present article, we draw parallels between SSP procedures and autoshaping, review factors shown to affect autoshaping, and interpret autoshaping research for suggested SSP tests and applications. We then call for extended use and reporting of SSP in behavior-analytic treatments. Finally, three bridges created by this article are identified: basic-applied, respondent-operant, and behavior analysis with other sciences.

Delay discounting: Pigeon, rat, human--does it matter?

Delay discounting refers to the decrease in subjective value of an outcome as the time to its receipt increases. Across species and situations, animals discount delayed rewards, and their discounting is well-described by a hyperboloid function. The current review begins with a comparison of discounting models and the procedures used to assess delay discounting in nonhuman animals. We next discuss the generality of discounting, reviewing the effects of different variables on the degree of discounting delayed reinforcers by nonhuman animals. Despite the many similarities in discounting observed between human and nonhuman animals, several differences have been proposed (e.g., the magnitude effect; nonhuman animals discount over a matter of seconds whereas humans report willing to wait months, if not years before receiving a reward), raising the possibility of fundamental species differences in intertemporal choice. After evaluating these differences, we discuss delay discounting from an adaptationist perspective. The pervasiveness of discounting across species and situations suggests it is a fundamental process underlying decision making.

Neurons in the pigeon nidopallium caudolaterale signal the selection and execution of perceptual decisions

Sensory systems provide organisms with information on the current status of the environment, thus enabling adaptive behavior. The neural mechanisms by which sensory information is exploited for action selection are typically studied with mammalian subjects performing perceptual decision-making tasks, and most of what is known about these mechanisms at the single-neuron level is derived from cortical recordings in behaving monkeys. To explore the generality of neural mechanisms underlying perceptual decision making across species, we recorded single-neuron activity in the pigeon nidopallium caudolaterale (NCL), a non-laminated associative forebrain structure thought to be functionally equivalent to mammalian prefrontal cortex, while subjects performed a visual categorisation task. We found that, whereas the majority of NCL neurons unspecifically upregulated or downregulated activity during stimulus presentation, ~20% of neurons exhibited differential activity for the sample stimuli and predicted upcoming choices. Moreover, neural activity in these neurons was ramping up during stimulus presentation and remained elevated until a choice was initiated, a response pattern similar to that found in monkey prefrontal and parietal cortices in saccadic choice tasks. In addition, many NCL neurons coded for movement direction during choice execution and differentiated between choice outcomes (reward and punishment). Taken together, our results implicate the NCL in the selection and execution of operant responses, an interpretation resonating well with the results of previous lesion studies. The resemblance of the response patterns of NCL neurons to those observed in mammalian cortex suggests that, despite differing neural architectures, mechanisms for perceptual decision making are similar across classes of vertebrates.

Assessing value representation in animals

Among all factors modulating our motivation to perform a given action, the ability to represent its outcome is clearly the most determining. Representation of outcomes, rewards in particular, and how they guide behavior, have sparked much research. Both practically and theoretically, understanding the relationship between the representation of outcome value and the organization of goal directed behavior implies that these two processes can be assessed independently. Most of animal studies essentially used instrumental actions as a proxy for the expected goal-value. The purpose of this article is to consider alternative measures of expected outcome value in animals, which are critical to understand the behavioral and neurobiological mechanisms relating the representation of the expected outcome to the organization of the behavior oriented towards its obtention. This would be critical in the field of decision making or social interactions, where the value of multiple items must often be compared and/or shared among individuals to determine the course of actions.

Species-specific response-topography of chickens' and pigeons' water-induced autoshaped responding

Four pigeons and eight chickens received autoshaping training where a keylight (conditioned stimulus) signaled response-independent deliveries of water (unconditioned stimulus). Pigeons drink while keeping their beaks submerged in water and moving their beaks to create suction ("mumbling"), whereas chickens drink by trapping a small amount of water in their mouths and then lifting their heads so the water trickles down. This experiment tested whether these and other species-specific differences in drinking and related behaviors of pigeons and chickens would be reflected in the form of conditioned (autoshaped) responding. Touchscreens and videotapes were used for data recording. Results showed that chickens moved their heads more than pigeons when drinking (unconditioned response). The birds also differed in conditioned responding in the presence of the keylight: Pigeons produced more keyswitch closures and mumbled at the keylight more than chickens whereas chickens scratched more than pigeons. In conclusion, with this unique comparative method that employed identical contingencies and comparable deprivation levels, species-specific differences in unconditioned responses and, more importantly, differences in their corresponding conditioned responses were observed.

From prediction error to incentive salience: mesolimbic computation of reward motivation

Reward contains separable psychological components of learning, incentive motivation and pleasure. Most computational models have focused only on the learning component of reward, but the motivational component is equally important in reward circuitry, and even more directly controls behavior. Modeling the motivational component requires recognition of additional control factors besides learning. Here I discuss how mesocorticolimbic mechanisms generate the motivation component of incentive salience. Incentive salience takes Pavlovian learning and memory as one input and as an equally important input takes neurobiological state factors (e.g. drug states, appetite states, satiety states) that can vary independently of learning. Neurobiological state changes can produce unlearned fluctuations or even reversals in the ability of a previously learned reward cue to trigger motivation. Such fluctuations in cue-triggered motivation can dramatically depart from all previously learned values about the associated reward outcome. Thus, one consequence of the difference between incentive salience and learning can be to decouple cue-triggered motivation of the moment from previously learned values of how good the associated reward has been in the past. Another consequence can be to produce irrationally strong motivation urges that are not justified by any memories of previous reward values (and without distorting associative predictions of future reward value). Such irrationally strong motivation may be especially problematic in addiction. To understand these phenomena, future models of mesocorticolimbic reward function should address the neurobiological state factors that participate to control generation of incentive salience.

Ontogeny has a phylogeny: background to adjunctive behaviors in pigeons and budgerigars

Animals coping with operant conditioning tasks often show behaviors that are not recorded by keys, levers and similar response transducers. Nevertheless, these adjunctive behaviors should not be disposed of by classifying them as incidental. Often they are found to be at least partially influenced by the experimentally programmed contingencies, and under certain conditions they can in turn influence conditioned behaviors. Here we describe the occurrence and characteristics of two such behaviors, stimulus grasping in operantly key-pecking pigeons and intra-delay stereotypies in a delayed matching-to-sample task with budgerigars. It is argued that for a proper account of these behaviors it is necessary to refer to a behavioral systems approach that appeals to longer ranging ontogenetic and phylogenetic histories than is usually considered in the psychological literature. The gaping towards on-key stimuli by pigeons is attributed to the hypothesis that operantly conditioned key-pecks probably relate to a grasp-pecking response that is normally executed towards non-edible items covering food. The intra-delay behaviors shown by the budgerigars are assumed to have originated from stress-induced displacement responses that adventitiously came under the influence of differential reinforcement contingencies. Finally, we discuss what kinds of evidence are needed to put these hypothetical explanations on a more certain footing.

Autoshaping in the rat: conditioned licking response to a stimulus that signals sucrose reinforcement

The present experiments were designed to determine if repeated presentations of an empty sipper tube (the conditioned stimulus or CS) with the response-independent delivery of a sucrose solution (the unconditioned stimulus or US) from a second spout results in the development of Pavlovian conditioned responding. In Experiment 1, rats in the experimental condition received paired CS-US presentations whereas subjects in the control condition were exposed to random presentations of CS and US. In Experiment 2, an additional control condition (CS alone) was included and, to encourage generalized responding between the US and CS, the CS tube was filled with water for all groups. The results of both experiments indicate that the CS-directed responding in the paired CS-US condition was Pavlovian in nature. Thus, the present procedure serves as an autoshaping task in which conditioned licking is generated.

Acquisition and extinction in autoshaping

C. R. Gallistel and J. Gibbon (2000) presented quantitative data on the speed with which animals acquire behavioral responses during autoshaping, together with a statistical model of learning intended to account for them. Although this model captures the form of the dependencies among critical variables, its detailed predictions are substantially at variance with the data. In the present article, further key data on the speed of acquisition are used to motivate an alternative model of learning, in which animals can be interpreted as paying different amounts of attention to stimuli according to estimates of their differential reliabilities as predictors.

Conditioning the pecking motions of pigeons

The spatio-temporal courses of head and neck motions of pigeons while pecking at small grains are described. Single and serial pecks are distinguished but the inter- and intraindividual variability of the peck kinetics is stressed. Pigeons were then trained with instrumental conditioning procedures to speed-up their pecking. A partial reinforcement schedule where pigeons had to peck repeatedly before receiving reward led to a mild shortening of inter-peck intervals at lower reinforcement rates but surprisingly, a lengthening at higher rates. A schedule where short inter-peck intervals were differentially rewarded yielded a pronounced abbreviation of the inter-peck intervals, but this was achieved by a reduction of the movement path rather than an increase in motion velocity. A schedule whereby increased approach velocities were differentially rewarded yielded marked movement accelerations. When pigeons were rewarded for diminished approach speeds they also showed significant movement decelerations. Finally, it is shown that pigeons could learn to reliably abort their peck approach movement when a visual stimulus signalling a penalty was occasionally presented during the approach movement. The proportion of successful peck interruptions decreased as these interruption signals occurred later during the approach phase. It is concluded that the pecking of pigeons is neither an innately fixed nor a visually ballistic movement. It is instead a multiply controlled and flexibly adaptable response pattern.

Effects of primary reinforcement on pigeons' initial-link responding under a concurrent chains schedule with nondifferntial terminal links

The effect of primary reinforcement on initial-link responding under concurrent-chains schedules with nondifferential terminal links was assessed in 12 pigeons. The iniitial and terminal links were variable-interval schedules (always the same for both alternatives). The positions (left or right key) of the initial-link stimuli (red or green) were randomized while the correlation between color and food amount remained constant within each condition. The terminal-link stimuli were always presented on the center key. Except in two control groups and conditions, the terminal-link stimuli were the same color (nondifferential, blue or yellow). Over six conditions, the differences in food amont and the durations of the initial- and terminal-link schedules were manipulated. In 57 of 60 cases, birds generated choice proportions above .50 in favor of the initial-link stimlus that was correlated with the larger reinforcer. There was some indication that preference increased with shortened terminal-link durations. Because the terminal-link stimuli were nondifferential, differential responding in the initial links cannot be explained easily by conditioned reinforcement represented by the terminal-link stimuli. Thus, primiary reinforcement has a direct effect on initial-link responding in concurrent-chains schedules.

Trigeminal deafferentation and conditioned pecking in pigeons

To clarify the contribution of peripheral trigeminal input to the control of pecking behavior we examined head and jaw movement kinematics and peck localization in pigeons with surgical section of trigeminal nerves providing somatosensory input to the beak. Conditioning procedures were used to bring the pecking/grasping components of pecking under the control of a visual target. Conditioned head and jaw movements were monitored 'on-line' using movement transducers and terminal peck location was recorded using 'touch-screen' technology. The periodic delivery of a food reinforcer provided repeated opportunities to monitor the kinematics of ingestive pecks. Deafferentation produced deficits in mandibulation during ingestive pecking and in the coordination of head and jaw movements during conditioned pecking. These results are attributed to disruptions in trigeminal feedback and feedforward mechanisms, respectively. In contrast with previous studies, deafferentation did not impair the precision of peck localization. Possible reasons for the absence of localization deficits are presented. The results are discussed in relation to the role of peripheral inputs in the control of prehensile movements.

Conditioned 'prehension' in the pigeon: kinematics, coordination and stimulus control of the pecking response

Like human prehensile behavior, the pigeon's ingestive pecking response is elicited by visual stimuli conveying information about the location and size of the target. This information is used to generate localized ingestive pecks whose gapes are amplitude-scaled to seed size, prior to contact. We employed high-resolution, 'real-time' monitoring of head acceleration, jaw movements and terminal peck location to examine the kinematics, coordination and stimulus control of conditioned pecking. Conditioning procedures were used to bring pecking under the control of visual targets whose stimulus properties (size, location) were independently varied, while simultaneously monitoring pecking response parameters. Stimulus control of the transport component (peck localization) is extremely precise, even in the absence of a specific localization-dependent reinforcement contingency. Subjects also showed amplitude-scaling of gape size to the size of a visual target, but over a more restricted range than shown to food pellets of comparable sizes. Comparison of the kinematic profiles of conditioned and ingestive pecks suggests that conditioned pecking is functionally analogous to human 'pointing' rather than 'grasping' behavior.

Key-peck probability and topography in a concurrent variable-interval variable-interval schedule with food and water reinforcers

The relation between variables that modulate the probability and the topography of key pecks was examined using a concurrent variable-interval variable-interval schedule with food and water reinforcers. Measures of response probability (response rates, time allocation) and topography (peck duration, gape amplitude) were obtained in 5 water- and food-deprived pigeons. Key color signaled reinforcer type. During baseline, response rates and time allocations were greater to the food key than to the water key, and food-key pecks had larger gapes and shorter durations. Relative probability measures (for the food key) were increased by prewatering and decreased by prefeeding. Deprivation effects upon topography measures were apparent only when food- and water-key pecks were analyzed separately. Food-key gape amplitudes increased with prewatering and decreased with prefeeding. The clearest effect occurred with prewatering. There were no consistent effects upon water-key gapes. The key color-reinforcer relation was reversed for 3 pigeons to determine how response topography was modulated during the transition from food- to water-key pecks. Reacquisition was faster for the probability than for the topography measures. Analysis of gape-amplitude distributions during reversal indicated that response-form modulation proceeded through the generation of intermediate gape sizes.

Effects of food-pellet size on rate, latency, and topography of autoshaped key pecks and gapes in pigeons

Four pigeons responded under autoshaping contingencies in which different conditional stimuli (red or green keylights) were associated with unconditional stimuli of different magnitudes (large or small food pellets) over successive trials within a session. Both topography (beak opening or gape) and strength (rates and latencies of key pecks and gapes) of responding during the conditional stimuli depended on the magnitude of the correlated unconditional stimulus. Key-peck and gape rates were higher and latencies were shorter in large-pellet trials than in small-pellet trials. Gape amplitudes varied directly with pellet size, although conditional and unconditional gapes were larger than either pellet. These findings were replicated when the key colors were presented either on one or two keys and after reversals of the color-size correlations. Because the unconditional stimulus was varied through pellet size, magnitude was not confounded with food-access duration or quality. These results demonstrate the effects of the magnitude of the unconditional stimulus, in that rates and latencies of both key pecks (which are directed movements toward the key) and gapes (which are independent of the bird's position and key properties) varied with pellet size. Gape measures were unique in that two dimensions (response strength and topography) of a single response class varied simultaneously with magnitude.