Freedom and knowledge: an experimental analysis of preference in pigeons

A Touchscreen-Based, Multiple-Choice Approach to Cognitive Enrichment of Captive Rhesus Macaques (Macaca mulatta)

Research on the psychological and physiological well-being of captive animals has focused on investigating different types of social and structural enrichment. Consequently, cognitive enrichment has been understudied, despite the promising external validity, comparability, and applicability. As we aim to fill this gap, we developed an interactive, multiple-choice interface for cage-mounted touchscreen devices that rhesus monkeys (Macaca mulatta) can freely interact with, from within their home enclosure at the Cognitive Neuroscience Laboratory of the German Primate Center. The multiple-choice interface offers interchangeable activities that animals can choose and switch between. We found that all 16 captive rhesus macaques tested consistently engaged with the multiple-choice interface across 6 weekly sessions, with 11 of them exhibiting clear task preferences, and displaying proficiency in performing the selected tasks. Our approach does not require social separation or dietary restriction and is intended to increase animals' sense of competence and agency by providing them with more control over their environment. Thanks to the high level of automation, our multiple-choice interface can be easily incorporated as a standard cognitive enrichment practice across different facilities and institutes working with captive animals, particularly non-human primates. We believe that the multiple-choice interface is a sustainable, scalable, and pragmatic protocol for enhancing cognitive well-being and animal welfare in captivity.

Evidence for entropy maximisation in human free choice behaviour

The freedom to choose between options is strongly linked to notions of free will. Accordingly, several studies have shown that individuals demonstrate a preference for choice, or the availability of multiple options, over and above utilitarian value. Yet we lack a decision-making framework that integrates preference for choice with traditional utility maximisation in free choice behaviour. Here we test the predictions of an inference-based model of decision-making in which an agent actively seeks states yielding entropy (availability of options) in addition to utility (economic reward). We designed a study in which participants freely navigated a virtual environment consisting of two consecutive choices leading to reward locations in separate rooms. Critically, the choice of one room always led to two final doors while, in the second room, only one door was permissible to choose. This design allowed us to separately determine the influence of utility and entropy on participants' choice behaviour and their self-evaluation of free will. We found that choice behaviour was better predicted by an inference-based model than by expected utility alone, and that both the availability of options and the value of the context positively influenced participants' perceived freedom of choice. Moreover, this consideration of options was apparent in the ongoing motion dynamics as individuals navigated the environment. In a second study, in which participants selected between rooms that gave access to three or four doors, we observed a similar pattern of results, with participants preferring the room that gave access to more options and feeling freer in it. These results suggest that free choice behaviour is well explained by an inference-based framework in which both utility and entropy are optimised and supports the idea that the feeling of having free will is tightly related to options availability.

A practitioner's guide to emphasizing choice-making opportunities in behavioral services provided to individuals with intellectual and developmental disabilities

Promoting choice is a defining value guiding Positive Behavior Support (PBS) models for serving individuals with intellectual and/or developmental disabilities (IDD). The ability to make independent choices is of paramount importance to self-advocacy and self-determination. Promoting choice is also an essential commitment of trauma-informed care (TIC) in the provision of services to vulnerable individuals, as trauma often involves experiences in which an individual has no control over aversive events that occur, and choice-making opportunities can empower traumatized individuals to regain control over the environments they routinely encounter. However, incorporating meaningful choice making into behavioral programming is often more difficult than it seems. We synthesize the relevant, contemporary literature to provide professionals with actionable suggestions for incorporating choice making into everyday behavioral services. After summarizing the importance of promoting choice into behavioral services based on the values that define both PBS and TIC frameworks, we (a) offer a behavior-analytic interpretation of the skill of making choices, (b) synthesize key literature on how to teach choice making skills, (c) provide recommendations for the situations within one's care in which choice-making opportunities may be most beneficial, and (d) discuss some of the barriers and potential solutions to incorporating choice-making opportunities for individuals with IDD.

Prolegomena to any future philosophy of behavior analysis as a science

This tribute to Howard Rachlin speculates about scholarly work that might have been. It explores how behavioral data might bear on philosophical issues, with examples that might be called case studies in experimental philosophy. In 1964, an issue of the Proceedings of the American Philosophical Society served a similar function. It was entitled "Psychology: A Behavioral Interpretation"; the papers included "Will," "Experience," "Appetite," "Humors," "Anxiety," and "Man." This presentation imagines what a contemporary project devoted to philosophical and behavior analytic perspectives on the topics of causation, freedom and volition, good and evil, time, words, and mind might have looked like. Along the way it notes how the project would have benefitted from Howard Rachlin's seminal contributions to both behavior analysis and philosophy. If ever such a project comes to pass, it will inevitably bear the stamp of his contributions.

A method for studying reinforcement factors controlling impulsive choice for use in behavioral neuroscience

Although procedures originating within the experimental analysis of behavior commonly are used in behavioral neuroscience to produce behavioral endpoints, they are used less often to analyze the behavioral processes involved, particularly at the level of individual organisms (see Soto, 2020). Concurrent-chains procedures have been used extensively to study choice and to quantify relations between various dimensions of reinforcement and preference. Unfortunately, parametric analysis of those relations using traditional steady-state, single-subject experimental designs can be time-consuming, often rendering these procedures impractical for use in behavioral neuroscience. The purpose of this paper is to describe how concurrent-chains procedures can be adapted to allow for parametric examination of effects of the reinforcement dimensions involved in impulsive choice (magnitude and delay) within experimental sessions in rats. Data are presented indicating that this procedure can produce relatively consistent within-session estimates of sensitivity to reinforcement in individual subjects, and that these estimates can be modified by neurobiological manipulation (drug administration). These data suggest that this type of procedure offers a promising approach to the study of neurobiological mechanisms of complex behavior in individual organisms, which could facilitate a more fruitful relationship between behavior analysis and behavioral neuroscience.

The effect of choice on intentional and incidental memory

Recent studies have revealed that memory performance is better when participants have the opportunity to make a choice regarding the experimental task (choice condition) than when they do not have such a choice (fixed condition). These studies, however, used intentional memory tasks, leaving open the question whether the choice effect also applies to incidental memory. In the current study, we first repeated the choice effect on the 24-h delayed intentional memory performance (experiment 1). Next, using an incidental paradigm in which participants were asked to judge the category of the items instead of intentionally memorizing them, we observed the choice effect on judgment during encoding and memory performance in a 24-h delayed surprise test (experiment 2). Participants judged more accurately and quickly and had better recognition memory for items in the choice condition than for items in the fixed condition. These results are discussed in terms of the role of choice in both intentional and incidental memory.

Toward trauma-informed applications of behavior analysis

Despite a growing acknowledgement of the importance of understanding the impacts of trauma on therapeutic approaches across human service disciplines, discussions of trauma have been relatively infrequent in the behavior analytic literature. In this paper, we delineate some of the barriers to discussing and investigating trauma in applied behavior analysis (ABA) and describe how the core commitments of trauma-informed care could be applied to behavior analysis. We then provide some examples of how trauma-informed care might be incorporated into ABA practice. We conclude by suggesting opportunities to approach trauma as a viable avenue for behavior analytic research and argue that omitting trauma-informed care from ABA could be detrimental not only to the public perception of ABA, but to the effectiveness of our assessment and treatment procedures.

The Relevance of Operant Behavior in Conceptualizing the Psychological Well-Being of Captive Animals

The term "psychological well-being" is used in reference to husbandry with animals in human care settings such as research, agriculture, and zoos. This article seeks to clarify and conceptualize the term based upon two approaches that draw from several bodies of literature: the experimental analysis of behavior, experimental psychology, animal welfare and husbandry, farm animal behavior, zoo husbandry, and ethology. One approach focuses on the presence of problem behavior such as stereotypies, depressive-like behavior, and aggression, and emphasizes the conditions under which aberrant behavior in animals under human care occurs. The second approach examines what might be considered wellness by emphasizing opportunities to engage with its environment, or the absence of such opportunities, even if problematic behavior is not exhibited. Here, access to an interactive environment is relatively limited so opportunities for operant (voluntary) behavior could be considered. Designing for operant behavior provides opportunities for variability in both behavior and outcomes. Operant behavior also provides control over the environment, a characteristic that has been a core assumption of well-being. The importance of interactions with one's environment is especially evident in observations that animals prefer opportunities to work for items necessary for sustenance, such as food, over having them delivered freely. These considerations raise the importance of operant behavior to psychological well-being, especially as benefits to animals under human care.

The functional equivalence of two variants of the suboptimal choice task: choice proportion and response latency as measures of value

In the suboptimal-choice task, birds systematically choose the leaner but informative option (suboptimal) over the richer but non-informative option (optimal). The task has two variations. In the standard task, the optimal option includes two terminal link stimuli. In the original task, it includes a single terminal link stimulus. Two models, the temporal information account (Cunningham and Shahan, J Exp Psychol Anim Learn Cogn 44:1-22, 2018) and the ∆-∑ hypothesis (González et al., J Exp Anal Behav 113:591-608, 2020), presuppose that these procedures are equivalent, but no formal comparison is available. Here we test whether or not these procedures are functionally equivalent. One group of pigeons was trained with the standard procedure, another group with the original procedure, and a third group was trained with a hybrid of the other two (i.e., the two options were the optimal links of the standard and original procedures). Our findings indicate that the number of terminal link stimuli in the optimal option is inconsequential vis-à-vis choice. Moreover, our findings also indicate that latencies to respond are a sensitive metric of value and choice. As predicted by the Sequential Choice Model, we were able to predict simultaneous choices from the latencies of sequential choices and observed a substantial shortening of latencies during simultaneous choices.

Preference for free or forced choice in Sumatran orangutans (Pongo abelii)

Empirical investigations of humans, pigeons, rats, and monkeys have indicated that these species will select free over forced choice, even when faced with identical outcomes. However, the same has yet to be quantitatively confirmed in nonhuman great apes. This experiment is the first systematic investigation of preference for free or forced choice in great apes using a paradigm in which extraneous variables are highly controlled. Three orangutans were given a choice of one of two virtual routes, one that provided a choice and one that did not via a touchscreen computer program. Choice of either route was rewarded with the same type and quantity of food. Initial results indicated a preference for free choice across all three participants. However, in two control conditions, orangutans' preferences varied, suggesting a weaker tendency to exercise choice than species previously tested. We suggest further investigation of preference for free and forced choice in orangutans and other great apes through alternative experimental paradigms that focus on increasing the fidelity of free and forced choice options.

Reinforcement uncertainty enhances preference for choice in humans

Under concurrent-chains schedules of reinforcement, participants often prefer situations that allow selection among alternatives (free choice) to situations that do not (forced choice). The present experiment examined the effects of reinforcement probability on choice preferences. Preferences for free versus forced choice were measured under a condition in which participants' choices were always reinforced (reinforcement probability of 1.0) and a condition in which outcomes were uncertain (reinforcement probability of 0.5). Forty-four college students participated and preferences were examined under a concurrent-chains schedule of reinforcement. Participants preferred free choice under uncertain reinforcement, but a bias toward free choice was not observed when reinforcement was certain. These results align with previous findings of preference for free choice under conditions of uncertainty, but suggest that preference may be dependent upon probabilistic reinforcement contingencies in the terminal links of the concurrent-chains arrangement. Thus, reinforcement probability is an important variable to consider when conducting similar studies on the value of choice.

Influence of multiple action-outcome associations on the transition dynamics toward an optimal choice in rats

When faced with familiar versus novel options, animals may exploit the acquired action-outcome associations or attempt to form new associations. Little is known about which factors determine the strategy of choice behavior in partially comprehended environments. Here we examine the influence of multiple action-outcome associations on choice behavior in the context of rewarding outcomes (food) and aversive outcomes (electric foot-shock). We used a nose-poke paradigm with rats, incorporating a dilemma between a familiar option and a novel, higher-value option. In Experiment 1, two groups of rats were trained with different outcome schedules: either a single action-outcome association ("Reward-Only") or dual action-outcome associations ("Reward-Shock"; with the added opportunity to avoid an electric foot-shock). In Experiment 2, we employed the same paradigm with two groups of rats performing the task under dual action-outcome associations, with different levels of threat (a low- or high-amplitude electric foot-shock). The choice behavior was clearly influenced by the action-outcome associations, with more efficient transition dynamics to the optimal choice with dual rather than single action-outcome associations. The level of threat did not affect the transition dynamics. Taken together, the data suggested that the strategy of choice behavior was modulated by the information complexity of the environment.

Irrational choice behavior in human and nonhuman primates

Choice behavior in humans has motivated a large body of research with a focus on whether decisions can be considered to be rational. In general, humans prefer having choice, as do a number of other species that have been tested, even though having increased choice does not necessarily yield a positive outcome. Humans have been found to choose an option more often only because the opportunity to select it was diminishing, an example of a deviation from economic rationality. Here we extend this paradigm to nonhuman primates in an effort to understand the mechanisms underlying this finding. In this study, we presented two groups of laboratory monkeys, capuchins (Cebus apella) and rhesus macaques (Macaca mulatta), as well as human subjects, with a computerized task in which subjects were presented with two differently colored icons. When the subject selected an icon, differing numbers of food pellets were dispensed (or points were assigned), making each icon correspond to a certain level of risk (one icon yielded 1 or 4 pellets/points and the other yielded 2 or 3). Initially, both options remained constantly available and we established choice preference scores for each subject. Then, we assessed preference patterns once the options were not continuously available. Specifically, choosing one icon would cause the other to shrink in size on the screen and eventually disappear if never selected. Selecting it would restore it to its full size. As predicted, humans shifted their risk preferences in the diminishing options phase, choosing to click on both icons more equally in order to keep both options available. At the group level, capuchin monkeys showed this pattern as well, but there was a great deal of individual variability in both capuchins and macaques. The present work suggests that there is some degree of continuity between human and nonhuman primates in the desire to have choice simply for the sake of having choice.

Do monkeys choose to choose?

Both empirical and anecdotal evidence supports the idea that choice is preferred by humans. Previous research has demonstrated that this preference extends to nonhuman animals, but it remains largely unknown whether animals will actively seek out or prefer opportunities to choose. Here we explored the issue of whether capuchin and rhesus monkeys choose to choose. We used a modified version of the SELECT task-a computer program in which monkeys can choose the order of completion of various psychomotor and cognitive tasks. In the present experiments, each trial began with a choice between two icons, one of which allowed the monkey to select the order of task completion, and the other of which led to the assignment of a task order by the computer. In either case, subjects still had to complete the same number of tasks and the same number of task trials. The tasks were relatively easy, and the monkeys responded correctly on most trials. Thus, global reinforcement rates were approximately equated across conditions. The only difference was whether the monkey chose the task order or it was assigned, thus isolating the act of choosing. Given sufficient experience with the task icons, all monkeys showed a significant preference for choice when the alternative was a randomly assigned order of tasks. To a lesser extent, some of the monkeys maintained a preference for choice over a preferred, but computer-assigned, task order that was yoked to their own previous choice selection. The results indicated that monkeys prefer to choose when all other aspects of the task are equated.

Selection of Forced- and Free-Choice by Monkeys (Macaca Fascicularis)

Using a concurrent-chains procedure, the present study examined preference of monkeys between forced-choice and free-choice with multiple alternatives. The forced-choice was followed by the reinforcer with a probability of .8. The free-choice was between reinforcement alternatives of high probability ( p = .8) and low probability ( p = .5) in which the monkeys mostly chose the former. The number of keys with each kind of alternative in the free-choice was manipulated under four conditions. Preference for free-choice increased with the number of keys for the preferred alternative. However, the number of keys for the less-preferred alternative had no influence on preference. Effects of the less-preferred alternative on preference were inconsistent with previous studies using pigeons.

Effects of the relative values of alternatives on preference for free-choice in humans

Organisms often prefer conditions that allow selection among alternatives (free-choice) to conditions that do not (forced-choice), particularly when response alternatives in free-choice produce equal or greater reinforcer magnitudes than those available under forced-choice. We present data on free-choice preference for human participants who gained or lost points by selecting images of cards on a computer screen under a concurrent-chains schedule. Responding during the initial link gained access to a terminal link offering a single-card set (forced-choice) or a three-card set (free-choice). The alternatives in free-choice produced reinforcer magnitudes (points) that were: (a) equal to forced-choice; (b) equal to and greater than forced-choice; and (c) equal to and less than forced-choice. Participants showed reliable preference for free-choice under some conditions; however, preference decreased as reinforcer magnitude for some alternatives in free-choice was reduced. This occurred even though it was possible to obtain the same number of points across free- and forced-choice. Although preference for free-choice was clearly demonstrated, the effect of points available in the terminal link suggests that this phenomenon is subject to modulation by other processes, such as reinforcement or punishment by obtained outcomes in the terminal link. Context (reinforcer-gain or -loss) was not a reliable predictor of preference.

Effects of number of alternatives on choice in humans

The present study examined human choice between a task including a single alternative (i.e. a single-alternative task), and a task including multiple alternatives (i.e. a multi-alternative task). The subjects were divided into three groups and were exposed to three conditions. The multi-alternative task included two alternatives in two groups and included three alternatives in one group. An alternative in the single-alternative task was identical to one in the multi-alternative task. Amount of reinforcement for alternatives specific to the multi-alternative tasks was manipulated under three conditions in each group. When the amount of reinforcement was equal to or larger than that in the alternative common to both tasks, the subjects preferred the multi-alternative task over the single-alternative task. When amount of reinforcement was smaller, there was no difference in preference between the multi-alternative and the single-alternative tasks. In addition, the degree of preference for the multi-alternative task depended on the number of alternatives that produced larger reinforcers.

Value of freedom to choose encoded by the human brain

Humans and animals value the opportunity to choose by preferring alternatives that offer more rather than fewer choices. This preference for choice may arise not only from an increased probability of obtaining preferred outcomes but also from the freedom it provides. We used human neuroimaging to investigate the neural basis of the preference for choice as well as for the items that could be chosen. In each trial, participants chose between two options, a monetary amount option and a "choice option." The latter consisted of a number that corresponded to the number of everyday items participants would subsequently be able to choose from. We found that the opportunity to choose from a larger number of items was equivalent to greater amounts of money, indicating that participants valued having more choice; moreover, participants varied in the degree to which they valued having the opportunity to choose, with some valuing it more than the increased probability of obtaining preferred items. Neural activations in the mid striatum increased with the value of the opportunity to choose. The same region also coded the value of the items. Conversely, activation in the dorsolateral striatum was not related to the value of the items but was elevated when participants were offered more choices, particularly in those participants who overvalued the opportunity to choose. These data suggest a functional dissociation of value representations within the striatum, with general representations in mid striatum and specific representations of the value of freedom provided by the opportunity to choose in dorsolateral striatum.

Decision processes in choice overload: a product of delay and probability discounting?

Recent research in the behavioral decision making literature has demonstrated that humans hyperbolically discount the subjective value of options as the number of options increases (Reed et al., 2012). These findings provide a cognitive-behavioral synthesis of the "choice overload" phenomenon, also known as the "paradox of choice." Specifically, these findings suggest that temporal discounting may serve as the underlying process contributing to this effect. As an extension, this study examined the effects of reward magnitude sizes had on rates temporal and options discounting. This manipulation was conducted to determine what role temporal discounting plays in discounting of options. The present results suggest that temporal discounting may not be the only process contributing to the choice overload effect.

Self-controlled learning: the importance of protecting perceptions of competence

Recent studies examining the role of self-controlled feedback have shown that learners ask for feedback after what they believe was a “good” rather than “poor” trial. Also, trials on which participants request feedback are often more accurate than those without feedback. The present study examined whether manipulating participants’ perception of “good” performance would have differential effects on learning. All participants practiced a coincident-anticipation timing task with a self-controlled feedback schedule during practice. Specifically, they were able to ask for feedback after 3 trials in each of three 10-trial practice blocks. While one group (Self-30) was told that an error of 30 ms or less would be considered good performance, another group (Self-4) was informed that an error of 4 ms or less would be considered a good trial. A third, self-control group (Self) did not receive any information about what constituted good performance. The results showed that participants of all groups asked for feedback primarily after relatively good trials. At the end of practice, both the Self-30 and Self groups demonstrated greater perceived competence and self-efficacy than the Self-4 group. The Self-30 and Self groups also performed with greater accuracy and less variability in retention and transfer (non-dominant hand) 1 day later. The present findings indicated that the typical learning benefits of self-controlled practice can be thwarted by depriving learners of the opportunity of experiencing competence through good performance. They add to the accumulating evidence of motivational influences on motor learning.

Effects of differential reinforcement and rules with feedback on preference for choice and verbal reports

We evaluated the effects of differential reinforcement and accurate verbal rules with feedback on the preference for choice and the verbal reports of 6 adults. Participants earned points on a probabilistic schedule by completing the terminal links of a concurrent-chains arrangement in a computer-based game of chance. In free-choice terminal links, participants selected 3 numbers from an 8-number array; in restricted-choice terminal links participants selected the order of 3 numbers preselected by a computer program. A pop-up box then informed the participants if the numbers they selected or ordered matched or did not match numbers generated by the computer but not displayed; matching in a trial resulted in one point earned. In baseline sessions, schedules of reinforcement were equal across free- and restricted-choice arrangements and a running tally of points earned was shown each trial. The effects of differentially reinforcing restricted-choice selections were evaluated using a reversal design. For 4 participants, the effects of providing a running tally of points won by arrangement and verbal rules regarding the schedule of reinforcement were also evaluated using a nonconcurrent multiple-baseline-across-participants design. Results varied across participants but generally demonstrated that (a) preference for choice corresponded more closely to verbal reports of the odds of winning than to reinforcement schedules, (b) rules and feedback were correlated with more accurate verbal reports, and (c) preference for choice corresponded more highly to the relative number of reinforcements obtained across free- and restricted-choice arrangements in a session than to the obtained probability of reinforcement or to verbal reports of the odds of winning.

Resistance to change and preference for variable versus fixed response sequences

In Experiment 1, 4 pigeons were trained on a multiple chain schedule in which the initial link was a variable-interval (VI) 20-s schedule signalled by a red or green center key, and terminal links required four responses made to the left (L) and/or right (R) keys. In the REPEAT component, signalled by red keylights, only LRLR terminal-link response sequences were reinforced, while in the VARY component, signalled by green keylights, terminal-link response sequences were reinforced if they satisfied a variability criterion. The reinforcer rate for both components was equated by adjusting the reinforcer probability for correct REPEAT sequences across sessions. Results showed that initial- and terminal-link responding in the VARY component was generally more resistant to prefeeding, extinction, and response-independent food than responding in the REPEAT component. In Experiment 2, the REPEAT and VARY contingencies were arranged as terminal links of a concurrent chain and the relative reinforcer rate was manipulated across conditions. For all pigeons, initial-link response allocation was biased toward the alternative associated with the VARY terminal link. These results replicate previous reports that operant variation is more resistant to change than operant repetition (Doughty & Lattal, 2001), and show that variation is preferred to repetition with reinforcer-related variables controlled. Behavioral momentum theory (Nevin & Grace, 2000) predicts the covariation of preference and resistance to change in Experiments 1 and 2, but does not explain why these aspects of behavior should depend on contingencies that require repetition or variation.

Discounting the freedom to choose: implications for the paradox of choice

Organisms prefer to make their own choices. However, emerging research from behavioral decision making sciences has demonstrated that there are boundaries to the preference for choice. Specifically, many decision makers find an extensive array of choice options to be aversive, often leading to negative emotional states and poor behavioral outcomes. This study examined the degree to which human participants discounted hypothetical rewards that were (a) delayed, (b) probabilistic, and (c) chosen from a large array of options. The present results suggest that the "paradox of choice" effect may be explained within a discounting model for individual patterns of decision making.

Effects of a history of differential reinforcement on preference for choice

The effects of a history of differential reinforcement for selecting a free-choice versus a restricted-choice stimulus arrangement on the subsequent responding of 7 undergraduates in a computer-based game of chance were examined using a concurrent-chains arrangement and a multiple-baseline-across-participants design. In the free-choice arrangement, participants selected three numbers, in any order, from an array of eight numbers presented on the computer screen. In the restricted-choice arrangement, participants selected the order of three numbers preselected from the array of eight by a computer program. In initial sessions, all participants demonstrated no consistent preference or preference for restricted choice. Differential reinforcement of free-choice selections resulted in increased preference for free choice immediately and in subsequent sessions in the absence of programmed differential outcomes. For 5 participants, changes in preference for choice were both robust and lasting, suggesting that a history of differential reinforcement for choice may affect preference for choice.

Signalled and unsignalled percentage reinforcement of performance under a chained schedule

Pigeons were trained to peck a key under a chained fixed-ratio 15 fixed-interval 25-sec schedule of food presentation. In Experiment 1, blocks of sessions in which 100%, 75%, 50%, and 25% of the sequences ended with food presentation were conducted. When food presentation was omitted, a timeout of equal duration replaced it. As the frequency of food presentation decreased so did the frequency of completing the chained schedule. In Experiment 2, 75%, 50%, or 25% of the sequences terminated with food presentation and outcomes were signalled, i.e., completion of the fixed ratio resulted in either a stimulus correlated with the fixed-interval 25-sec schedule or a stimulus correlated with extinction. As the frequency of food presentation decreased, the number of sequences completed per session increased for two pigeons and remained high for a third. In Experiments 3 and 4, assessments of the effects of signalling the outcome of the chained schedule were made with response-independent presentation of events at the end of the sequence. Again, signalling the outcome of the chained schedule led to more chains being completed per session than did not signalling the outcome. Stimuli differentially paired with food presentation have powerful behavioral effects that may be attributed to the potency of these stimuli as conditioned reinforcers.

Incentive theory: II. Models for choice

Incentive theory is extended to account for concurrent chained schedules of reinforcement. The basic model consists of additive contributions from the primary and secondary effects of reinforcers, which serve to direct the behavior activated by reinforcement. The activation is proportional to the rate of reinforcement and interacts multiplicatively with the directive effects. The two free parameters are q, the slope of the delay of reinforcement gradient, whose value is constant across many experiments, and b, a bias parameter. The model is shown to provide an excellent description of all results from studies that have varied the terminal-link schedules, and of many of the results from studies that have varied initial-link schedules. The model is extended to diverse modifications of the terminal links, such as varied amount of reinforcement, varied signaling of the terminal-link schedules, and segmentation of the terminal-link schedules. It is demonstrated that incentive theory provides an accurate and integrated account of many of the phenomena of choice.

Human observing: Maintained by stimuli correlated with reinforcement but not extinction

College students received points exchangeable for money (reinforcement) on a variable-time 60-second schedule that alternated randomly with an extinction component. Subjects were informed that responding would not influence either the rate or distribution of reinforcement. Instead, presses on either of two levers ("observing responses") produced stimuli. In each of four experiments, stimuli positively correlated with reinforcement and/or stimuli uncorrelated with reinforcement were each chosen over stimuli correlated with extinction. These results are consistent with prior results from pigeons in supporting the conditioned-reinforcement hypothesis of observing and in not supporting the uncertainty-reduction hypothesis.

Escape from freedom: Choosing not to choose in pigeons

Preference for the availability of food-reinforcement alternatives was investigated with Rachlin and Green's (1972) concurrent-chains self-control paradigm. The terminal link of one chain made available a choice between immediate access to food for T seconds and delayed access to food for 4 seconds. The terminal link of the other chain provided only delayed access to food. When T was reduced to .25 seconds, pigeons began to select the delayed-access key in both terminal links. Even so, the pigeons strongly preferred constraint over choice. This effect could not be accounted for by differences in the actual amount of food obtained in the terminal links, by avoidance of the immediate-reinforcement key when not presented as part of a choice, or by avoidance of a multi-key terminal link. Rather, constraint was preferred over freedom. Apparently, the preference for choice is determined by the particular type of choice offered.

Some factors that influence the acquisition of complex, stereotyped, response sequences in pigeons

Two pigeons were required to peck six to nine illuminated response keys. A response on any one of the keys darkened that key. When each key had been darkened, a reinforcer was delivered. No specific sequence of key pecking was ever required. The keys were presented in various matrices: three by two, three by three, horizontal rows, and vertical columns. The keys either presented the same stimulus, white light; or each key presented a different stimulus, a color or form. The results indicated that although there were 720 to 362,880 different sequences that would produce reinforcement, each bird developed a particular, stereotyped sequence that dominated its behavior. Variability among the birds across phases yielded less than 60 sequences, .0001 to 6 percent of the possible sequences. The data suggest that a reinforcement contingency that includes "free choice" of response sequence will produce stereotypical response sequences that function as complex "units" of behavior.

Multiple and concurrent schedule performance: independence from concurrent and successive schedule contexts

Six pigeons were trained on multiple variable-interval schedules and performance was measured in the presence or absence of another variable-interval schedule (the common schedule) arranged concurrently with both components. Manipulations included varying the rate of reinforcement on the common schedule, leaving the common schedule unchanged while the components of the multiple schedule were varied, varying the multiple schedule components in the absence of the common schedule, and varying one component of the multiple schedule while the other component and the common schedule were unchanged. The normal rate-increasing and rate-decreasing effects of reinforcement rate increase were found, except that changing one multiple schedule component did not affect the response rate in the successively available common schedule component. Both concurrent and multiple schedule performance undermatched obtained reinforcement-rate ratios, but the degree of undermatching in multiple schedules was reliably greater. Allocation of responses between multiple schedule components was unaffected by the concurrent availability of reinforcement, and allocation of responses between concurrent schedules was unaffected by the successive availability of different reinforcement rates.

Preference for free choice over forced choice in pigeons

In a six-key chamber variable-interval initial links of concurrent-chain schedules operated on two lower white keys. Terminal links operated on four upper keys; green keys were correlated with fixed-interval reinforcement and red keys with extinction. Free-choice terminal links arranged three green keys and one red key; forced-choice terminal links arranged one green key and three red keys. Thus, terminal links were equivalent in number, variety, and information value (in bits) of the keylights. Preferences (relative initial-link rates) were studied both with location of the odd key color varying over successive terminal links and with the odd color fixed at key locations that had controlled either relatively high or relatively low terminal-link response rates. Free choice was consistently preferred to forced choice. Magnitude of preference did not vary systematically with terminal-link response rate or stimulus control by green and red keys. The origins of free-choice preference could be ontogenic or phylogenic: organisms may learn that momentarily preferred alternatives are more often available in free than in forced choice, and evolutionary contingencies may favor the survival of organisms that prefer free to forced choice.

How to spend a token? Trade-offs between food variety and food preference in tufted capuchin monkeys (Cebus apella)

Humans and non-human animals often choose among different alternatives by seeking variety. Here we assessed whether variety-seeking, i.e. the tendency to look for diversity in services and goods, occurs in capuchin monkeys--South-American primates which--as humans--are omnivorous and susceptible to food monotony. Capuchins chose between a Variety-token, that allowed to select one among 10 different foods (one more-preferred and nine less-preferred) and a Monotony-token, that--upon exchange with the experimenter--either allowed to select one among 10 units of the same more-preferred food or gave access to one unit of the more-preferred food. To examine how food preference affects variety-seeking, in the B-condition we presented nine moderately preferred foods, whereas in the C-condition we presented nine low-preferred foods. Overall, capuchins preferred the Variety-token over the Monotony-token and often selected one of the less-preferred foods. These results suggest that variety-seeking is rooted in our evolutionary history, and that it satisfies the need of experiencing stimulation from the environment; at the ultimate level, variety-seeking may allow the organism to exploit novel foods and obtain a correct nutritional intake. Finally, variety-seeking could have contributed to the transition from barter to money in many human cultures.

On Choice, Preference, and Preference for Choice

In this paper, we examine several common everyday meanings of choice, propose behavioral definitions of choice, choosing, and preference, and recommend ways for behavioral researchers to talk consistently about these concepts. We also examine the kinds of performance in the contexts of various procedures that might be appropriately described as a preference for choice. In our view, the most appropriate procedure for demonstrating preference for choice as a consequence is a concurrent chains method, in which choice is a reinforcer for an approach response. The single-stimulus procedure, however, is more appropriate for demonstrating preference for choice as an antecedent.

Variation, repetition, and choice

Experiment 1 investigated the controlling properties of variability contingencies on choice between repeated and variable responding. Pigeons were exposed to concurrent-chains schedules with two alternatives. In the REPEAT alternative, reinforcers in the terminal link depended on a single sequence of four responses. In the VARY alternative, a response sequence in the terminal link was reinforced only if it differed from the n previous sequences (lag criterion). The REPEAT contingency generated low, constant levels of sequence variation whereas the VARY contingency produced levels of sequence variation that increased with the lag criterion. Preference for the REPEAT alternative tended to increase directly with the degree of variation required for reinforcement. Experiment 2 examined the potential confounding effects in Experiment 1 of immediacy of reinforcement by yoking the interreinforcer intervals in the REPEAT alternative to those in the VARY alternative. Again, preference for REPEAT was a function of the lag criterion. Choice between varying and repeating behavior is discussed with respect to obtained behavioral variability, probability of reinforcement, delay of reinforcement, and switching within a sequence.

Effects of experience on preference between forced and free choice

Preference between forced choice and free choice in concurrent-chain schedules of reinforcement was investigated in pigeons after exposure to particular combinations of terminal links. In Experiment 1, in which terminal links always ended with reinforcers, one of three pairs of terminal links was arranged as preexposure: (a) both terminal links had only one key (forced choice), (b) both terminal links had two keys (free choice), or (c) a combination of forced and free choice was arranged across sessions. In test sessions following the preexposure, pigeons' preferences rapidly shifted to the terminal links with which they had no recent experience. In Experiment 2, the same procedure was repeated except that each terminal link ended intermittently with reinforcers with a probability of .5 and there was no terminal-link arrangement with a combination of free and forced choice. Pigeons showed the same preference changes as in Experiment 1, but the preference changes did not appear immediately at the beginning of test sessions. These data suggest that recent previous experience was a more important determinant of preference than the difference between forced-choice and free-choice terminal links.

Free-choice preference under uncertainty

Preferences in pigeons for free choice over forced choice under uncertain contingencies were compared with the one under certain contingencies in multiple concurrent-chain schedules of reinforcement. The uncertain condition examined the preference for two alternatives over one alternative when reinforcement probability at the end of the terminal link equalled 0.5, and with all keys in each terminal link lit green (two mixed fixed-interval extinction keys versus one mixed fixed-interval extinction key). Key and schedule arrangement in the certain condition was the same except that a peck on any terminal-link key after the FI interval always produced food. When naive pigeons were first exposed to uncertain contingencies, preference for two lit keys over one lit key was observed, and the preference was confirmed by sequential reversals of the terminal-link contingencies. However, no consistent preference was observed when uncertain condition followed the certain condition. Under certain contingencies, unlike the earlier experiments, very small and inconsistent preferences for free choice were demonstrated. A possible reason for the different preferences in the uncertain conditions was that pigeons may lessen their sensitivity to the circumstances with uncertainty by any history or carry-over effect of the prior contingency.

Mechanics of the animate

Behavior is treated as basic physics. Dimensions are identified and their transformations from physical specification to axes in behavioral space are suggested. Responses are treated as action patterns arrayed along a continuum of activation energy. Behavior is seen as movement along a trajectory through this behavior space. Incentives or reinforcers are attractors in behavior space, at the centers of basins of lowered potential. Trajectories impinging on such basins may be captured; repeated capture will warp the trajectory toward a geodesic, a process called conditioning. Conditioning is enhanced by contiguity, the proximity between the measured behavior and the incentive at the end of the trajectory, and by contingency, the depth of the trajectory below the average level of the potential energy landscape. Motivation is seen as the potential of an organism for motion under the forces impinging on it. Degree of motivation is characterized by the depth of the potential field, with low motivation corresponding to a flat field and a flat gradient of activation energy. Drives are the forces of incentives propagated through behavior space. Different laws for the attenuation of drive with behavioral distance are discussed, as is the dynamics of action. The basic postulate of behavior mechanics is incentive-tracking in behavior space, the energy for which is provided by decreases in potential. The relation of temporal gradients to response differentiation and temporal discrimination is analyzed. Various two-body problems are sketched to illustrate the application of these ideas to association, choice, scalar timing, self-control, and freedom.