Observing and conditioned reinforcement

To know or not to know? Curiosity and the value of prospective information in animals

Humans and other animals often seek instrumental information to strategically improve their decisions in the present. Our curiosity also leads us to acquire non-instrumental information that is not immediately useful but can be encoded in memory and stored for use in the future by means of episodic recall. Despite its adaptive benefits and central role in human cognition, questions remain about the cognitive mechanisms and evolutionary origins that underpin curiosity. Here, we comparatively review recent empirical studies that some authors have suggested reflects curiosity in nonhuman animals. We focus on findings from laboratory tasks in which individuals can choose to gain advanced information about uncertain future outcomes, even though the information cannot be used to increase future rewards and is often costly. We explore the prevalence of preferences in these tasks across animals, discuss the theoretical advances that they have promoted, and outline some limitations in contemporary research. We also discuss several features of human curiosity that can guide future empirical research aimed at characterising and understanding curiosity in animals. Though the prevalence of curiosity in animals is actively debated, we surmise that investigating behavioural candidates for curiosity-motivated behaviour in a broader range of species and contexts, should help promote theoretical advances in our understanding of cognitive principles and evolutionary pressures that support curiosity-driven behaviour.

On the value of advanced information about delayed rewards

In a variety of laboratory preparations, several animal species prefer signaled over unsignaled outcomes. Here we examine whether pigeons prefer options that signal the delay to reward over options that do not and how this preference changes with the ratio of the delays. We offered pigeons repeated choices between two alternatives leading to a short or a long delay to reward. For one alternative (informative), the short and long delays were reliably signaled by different stimuli (e.g., SS for short delays, SL for long delays). For the other (non-informative), the delays were not reliably signaled by the stimuli presented (S1 and S2). Across conditions, we varied the durations of the short and long delays, hence their ratio, while keeping the average delay to reward constant. Pigeons preferred the informative over the non-informative option and this preference became stronger as the ratio of the long to the short delay increased. A modified version of the Δ-Σ hypothesis (González et al., J Exp Anal Behav 113(3):591-608. https://doi.org/10.1002/jeab.595 , 2020a) incorporating a contrast-like process between the immediacies to reward signaled by each stimulus accounted well for our findings. Functionally, we argue that a preference for signaled delays hinges on the potential instrumental advantage typically conveyed by information.

More complex incidental bidirectional naming results from exposure alone

Incidental bidirectional naming (Inc-BiN) has been defined as a verbal developmental cusp whereby children demonstrate learning the names of things as listener and speaker as a function of observation alone. Stimulus characteristics have been found to affect performance in tests for Inc-BiN. To further explore this effect, Experiment 1 compared untaught listener and speaker responses for novel familiar-type versus novel nonfamiliar-type stimuli with 20 first-grade students following naming experiences in which the participants observed each visual stimulus five times while hearing its name. Participants performed significantly better with familiar-type than with nonfamiliar-type stimuli. Experiment 2 examined the effects of a repeated-probe intervention to induce Inc-BiN with nonfamiliar-type stimuli. Participants were six first-grade students who demonstrated incidental unidirectional naming (i.e., acquired names as listener from exposure alone). Implementation of the intervention was staggered across dyads of participants in a multiple-probe, simultaneous-treatments design. One participant in each dyad received the intervention with nonfamiliar-type stimuli only and the other with both nonfamiliar- and familiar-type stimuli. Pre- and postintervention Inc-BiN probes with stimuli not included in the intervention suggested both conditions were effective in establishing Inc-BiN for nonfamiliar-type stimuli. These findings have implications for understanding the mechanisms underlying Inc-BiN.

An evaluation of methods for studying the effects of conditioned reinforcement on human choice

Shahan et al. (2006) found that the relative rate of pigeons' pecking on two observing responses (i.e., responses that only produced an S+ or stimulus correlated with primary reinforcement) was well described by the relative rate of S+ delivery. Researchers have not evaluated the effects of S+ delivery rate in a concurrent observing response procedure with human subjects, so the necessary procedural modifications for studying the effects of conditioned reinforcement on human choice remain unclear. The purpose of the current study was to conduct an additive component analysis of modifications to the procedures of Shahan et al. (2006). We evaluated the additive effects of introducing response cost, a changeover response, and ordinal discriminative stimuli on correspondence with the results of Shahan et al. and the quality of fits of the generalized matching equation. When our procedures were most similar to those of Shahan et al., we observed low rates of observing and indifference between the two observing responses. For the group of subjects with whom all three additive components were included, we obtained the highest level of sensitivity to relative rate of S+ delivery, but the slope and R² of our fits of the generalized matching equation were still much lower than those obtained by Shahan et al. Potential reasons for these discrepancies, methods of resolving them, and implications for future research are discussed.

Visuospatial information foraging describes search behavior in learning latent environmental features

In the real world, making sequences of decisions to achieve goals often depends upon the ability to learn aspects of the environment that are not directly perceptible. Learning these so-called latent features requires seeking information about them. Prior efforts to study latent feature learning often used single decisions, used few features, and failed to distinguish between reward-seeking and information-seeking. To overcome this, we designed a task in which humans and monkeys made a series of choices to search for shapes hidden on a grid. On our task, the effects of reward and information outcomes from uncovering parts of shapes could be disentangled. Members of both species adeptly learned the shapes and preferred to select tiles expected to be informative earlier in trials than previously rewarding ones, searching a part of the grid until their outcomes dropped below the average information outcome-a pattern consistent with foraging behavior. In addition, how quickly humans learned the shapes was predicted by how well their choice sequences matched the foraging pattern, revealing an unexpected connection between foraging and learning. This adaptive search for information may underlie the ability in humans and monkeys to learn latent features to support goal-directed behavior in the long run.

Paradoxical choice and the reinforcing value of information

Signals that reduce uncertainty can be valuable because well-informed decision-makers can better align their preferences to opportunities. However, some birds and mammals display an appetite for informative signals that cannot be used to increase returns. We explore the role that reward-predictive stimuli have in fostering such preferences, aiming at distinguishing between two putative underlying mechanisms. The 'information hypothesis' proposes that reducing uncertainty is reinforcing per se, somewhat consistently with the concept of curiosity: a motivation to know in the absence of tractable extrinsic benefits. In contrast, the 'conditioned reinforcement hypothesis', an associative account, proposes asymmetries in secondarily acquired reinforcement: post-choice stimuli announcing forthcoming rewards (S⁺) reinforce responses more than stimuli signalling no rewards (S^-) inhibit responses. In three treatments, rats faced two equally profitable options delivering food probabilistically after a fixed delay. In the informative option (Info), food or no food was signalled immediately after choice, whereas in the non-informative option (NoInfo) outcomes were uncertain until the delay lapsed. Subjects preferred Info when (1) both outcomes were explicitly signalled by salient auditory cues, (2) only forthcoming food delivery was explicitly signalled, and (3) only the absence of forthcoming reward was explicitly signalled. Acquisition was slower in (3), when food was not explicitly signalled, showing that signals for positive outcomes have a greater influence on the development of preference than signals for negative ones. Our results are consistent with an elaborated conditioned reinforcement account, and with the conjecture that both uncertainty reduction and conditioned reinforcement jointly act to generate preference.

Neural circuitry of information seeking

Humans and animals navigate uncertain environments by seeking information about the future. Remarkably, we often seek information even when it has no instrumental value for aiding our decisions - as if the information is a source of value in its own right. In recent years, there has been a flourishing of research into these non-instrumental information preferences and their implementation in the brain. Individuals value information about uncertain future rewards, and do so for multiple reasons, including valuing resolution of uncertainty and overweighting desirable information. The brain motivates this information seeking by tapping into some of the same circuitry as primary rewards like food and water. However, it also employs cortex and basal ganglia circuitry that predicts and values information as distinct from primary reward. Uncovering how these circuits cooperate will be fundamental to understanding information seeking and motivated behavior as a whole, in our increasingly complex and information-rich world.

The value of what's to come: Neural mechanisms coupling prediction error and the utility of anticipation

Having something to look forward to is a keystone of well-being. Anticipation of future reward, such as an upcoming vacation, can often be more gratifying than the experience itself. Theories suggest the utility of anticipation underpins various behaviors, ranging from beneficial information-seeking to harmful addiction. However, how neural systems compute anticipatory utility remains unclear. We analyzed the brain activity of human participants as they performed a task involving choosing whether to receive information predictive of future pleasant outcomes. Using a computational model, we show three brain regions orchestrate anticipatory utility. Specifically, ventromedial prefrontal cortex tracks the value of anticipatory utility, dopaminergic midbrain correlates with information that enhances anticipation, while sustained hippocampal activity mediates a functional coupling between these regions. Our findings suggest a previously unidentified neural underpinning for anticipation's influence over decision-making and unify a range of phenomena associated with risk and time-delay preference.

Checking responses of goal- and sign-trackers are differentially affected by threat in a rodent analog of obsessive-compulsive disorder

In obsessive–compulsive disorder (OCD), functional behaviors such as checking that a door is locked become dysfunctional, maladaptive, and debilitating. However, it is currently unknown how aversive and appetitive motivations interact to produce functional and dysfunctional behavior in OCD. Here we show a double dissociation in the effects of anxiogenic cues and sensitivity to rewarding stimuli on the propensity to develop functional and dysfunctional checking behavior in a rodent analog of OCD, the observing response task (ORT). While anxiogenic manipulations of perceived threat (presentation of threat-associated contextual cues) and actual threat (punishment of incorrect responding on the ORT) enhanced functional checking, dysfunctional checking was unaffected. In contrast, rats that had previously been identified as “sign-trackers” on an autoshaping task—and therefore were highly sensitive to the incentive salience of appetitive environmental cues—selectively showed elevated levels of dysfunctional checking under a range of conditions, but particularly so under conditions of uncertainty. These data indicate that functional and dysfunctional checking are dissociable and supported by aversive and appetitive motivational processes, respectively. While functional checking is modulated by perceived and actual threat, dysfunctional checking recruits appetitive motivational processes, possibly akin to the “incentive habits” that contribute to drug-seeking in addiction.

Dissociable dopaminergic and pavlovian influences in goal-trackers and sign-trackers on a model of compulsive checking in OCD

RATIONALE

Checking is a functional behaviour that provides information to guide behaviour. However, in obsessive-compulsive disorder (OCD), checking may escalate to dysfunctional levels. The processes underpinning the transition from functional to dysfunctional checking are unclear but may be associated with individual differences that support the development of maladaptive behaviour. We examined one such predisposition, sign-tracking to a pavlovian conditioned stimulus, which we previously found associated with dysfunctional checking. How sign-tracking interacts with another treatment with emerging translational validity for OCD-like checking, chronic administration of the dopamine D₂ receptor agonist quinpirole, is unknown.

OBJECTIVES

We tested how functional and dysfunctional checking in the rat observing response task (ORT) was affected by chronic quinpirole administration in non-autoshaped controls and autoshaped animals classified as sign-trackers or goal-trackers.

METHODS

Sign-trackers or goal-trackers were trained on the ORT before the effects of chronic quinpirole administration on checking were assessed. Subsequently, the effects on checking of different behavioural challenges, including reward omission and the use of unpredictable reinforcement schedules, were tested.

RESULTS

Prior autoshaping increased checking. Sign-trackers and goal-trackers responded differently to quinpirole sensitization, reward omission and reinforcement uncertainty. Sign-trackers showed greater elevations in dysfunctional checking, particularly during uncertainty. By contrast, goal-trackers predominantly increased functional checking responses, possibly in response to reduced discrimination accuracy in the absence of cues signalling which lever was currently active.

CONCLUSIONS

The results are discussed in terms of how pavlovian associations influence behaviour that becomes compulsive in OCD and how this may be dependent on striatal dopamine D₂ receptors.

Effects of schedule type and terminal-link duration on choice in concurrent chains

Four pigeons responded in a three-component concurrent chains procedure in which the terminal-link schedules were either both fixed-interval (FI FI), both variable-interval (VI VI), both mixed-interval (MI MI) or variable-interval fixed-interval (VI FI). Across components within sessions, overall terminal-link duration was varied while schedule types were varied across conditions. For the conditions with homogeneous schedules, the strongest preference was obtained with FI FI, intermediate with MI MI, and weakest with VI VI. Preference increased with overall terminal-link duration for all schedules, but the increase was more rapid for VI FI. The hyperbolic value-added model (HVA) and cumulative decision model (CDM) were fitted to the choice data and accounted for 83.9% and 76.1% of the variance, respectively. However, deviations from both models' predictions were systematic. A modification of the CDM, which assumed that reinforcers delivered after variable delays were more effective, improved the fit of the model and substantially reduced the systematic deviations in residuals. Comparable modifications of HVA produced only limited improvement. Results show that preference in concurrent chains with homogeneous terminal links depends on the degree of variability in reinforcer delays, and that the CDM can provide an excellent account of results across different terminal-link schedules.

Environment tracking and signal following in a reinforcer-ratio reversal procedure

Several studies suggest that the degree of control by reinforcer ratios (environment tracking) and by exteroceptive stimuli that signal future reinforcer availability (signal following) depends on environmental certainty: As reinforcers become more likely at one location, environmental contingencies exert stronger control and exteroceptive stimuli exert weaker control. This research has not yet been extended to environments in which reinforcer availability changes across time, even though such changes are present in most natural environments. Thus, in the present experiment, we examined environment tracking and signal following in a concurrent schedule in which the reinforcer ratio reversed to its reciprocal 30 s after a reinforcer delivery and keylight-color stimuli signaled the likely or definite time or location of the next reinforcer. Across conditions, we manipulated environmental certainty by varying the probability of reinforcer deliveries on the locally richer key. This made the location of future reinforcers at a particular time more or less certain, but did not change the overall reinforcer ratio. Changes in local environmental certainty had little to no effect on environment tracking and signal following; in all conditions, keylight-color stimuli strongly controlled choice and reinforcer ratios exerted weak control. The present findings suggest that the extent of environment tracking and signal following is primarily determined by global, not local, environmental certainty.

How I Learned to Stop Worrying and Love Replication Failures

Worries about the reproducibility of experiments in the behavioral and social sciences arise from evidence that many published reports contain false positive results. Misunderstanding and misuse of statistical procedures are key sources of false positives. In behavior analysis, however, statistical procedures have not been used much. Instead, the investigator must show that the behavior of an individual is consistent over time within an experimental condition, that the behavior changes systematically across conditions, and that these changes can be reproduced - and then the whole pattern must be shown in additional individuals. These high standards of within- and between-subject replication protect behavior analysis from the publication of false positive findings. When a properly designed and executed experiment fails to replicate a previously published finding, the failure exposes flaws in our understanding of the phenomenon under study - perhaps in recognizing the boundary conditions of the phenomenon, identifying the relevant variables, or bringing the variables under sufficient control. We must accept the contradictory findings as valid and pursue an experimental analysis of the possible reasons. In this way, we resolve the contradiction and advance our science. To illustrate, two research programs are described, each initiated because of a replication failure.

Role of the medial prefrontal cortex and nucleus accumbens in an operant model of checking behaviour and uncertainty

Background:

Excessive checking is a common, debilitating symptom of obsessive–compulsive disorder. To further examine cognitive processes underpinning checking behaviour, and clarify how and why checking develops, we designed a novel operant paradigm for rats, the observing response task. The present study used the observing response task to investigate checking behaviour following excitotoxic lesions of the medial prefrontal cortex, nucleus accumbens core and dorsal striatum, brain regions considered to be of relevance to obsessive–compulsive disorder.

Methods:

In the observing response task, rats pressed an ‘observing’ lever for information (provided by light onset) about the location of an ‘active’ lever that provided food reinforcement. Following training, rats received excitotoxic lesions of the regions described above and performance was evaluated post-operatively before histological processing.

Results:

Medial prefrontal cortex lesions selectively increased functional checking with a less-prominent effect on non-functional checking and reduced discrimination accuracy during light information periods. Rats with nucleus accumbens core lesions made significantly more checking responses than sham-lesioned rats, including both functional and non-functional checking. Dorsal striatum lesions had no direct effect on checking per se, but reduced both active and inactive lever presses, and therefore changed the relative balance between checking responses and instrumental responses.

Conclusions:

These results suggest that the medial prefrontal cortex and nucleus accumbens core are important in the control of checking, perhaps via their role in processing uncertainty of reinforcement, and that dysfunction of these regions may therefore promote excessive checking behaviour, possibly relevant to obsessive-compulsive disorder.

When good news leads to bad choices

Pigeons and other animals sometimes deviate from optimal choice behavior when given informative signals for delayed outcomes. For example, when pigeons are given a choice between an alternative that always leads to food after a delay and an alternative that leads to food only half of the time after a delay, preference changes dramatically depending on whether the stimuli during the delays are correlated with (signal) the outcomes or not. With signaled outcomes, pigeons show a much greater preference for the suboptimal alternative than with unsignaled outcomes. Key variables and research findings related to this phenomenon are reviewed, including the effects of durations of the choice and delay periods, probability of reinforcement, and gaps in the signal. We interpret the available evidence as reflecting a preference induced by signals for good news in a context of uncertainty. Other explanations are briefly summarized and compared.

Conditioned reinforcement and information theory reconsidered

The idea that stimuli might function as conditioned reinforcers because of the information they convey about primary reinforcers has a long history in the study of learning. However, formal application of information theory to conditioned reinforcement has been largely abandoned in modern theorizing because of its failures with respect to observing behavior. In this paper we show how recent advances in the application of information theory to Pavlovian conditioning offer a novel approach to conditioned reinforcement. The critical feature of this approach is that calculations of information are based on reductions of uncertainty about expected time to primary reinforcement signaled by a conditioned reinforcer. Using this approach, we show that previous failures of information theory with observing behavior can be remedied, and that the resulting framework produces predictions similar to Delay Reduction Theory in both observing-response and concurrent-chains procedures. We suggest that the similarity of these predictions might offer an analytically grounded reason for why Delay Reduction Theory has been a successful theory of conditioned reinforcement. Finally, we suggest that the approach provides a formal basis for the assertion that conditioned reinforcement results from Pavlovian conditioning and may provide an integrative approach encompassing both domains.

Irrational choice and the value of information

Irrational decision making in humans and other species challenges the use of optimality in behavioural biology. Here we show that such observations are in fact powerful tools to understand the adaptive significance of behavioural mechanisms. We presented starlings choices between probabilistic alternatives, receiving or not information about forthcoming, delayed outcomes after their choices. Subjects could not use this information to alter the outcomes. Paradoxically, outcome information induced loss-causing preference for the lower probability option. The effect depended on time under uncertainty: information given just after each choice caused strong preference for lower probability, but information just before the outcome did not. A foraging analysis shows that these preferences would maximize gains if post-choice information were usable, as when predators abandon a chase when sure of the prey escaping. Our study illustrates how experimentally induced irrational behaviour supports rather than weakens the evolutionary optimality approach to animal behaviour.

Is the Source of Reinforcement for Naming Multiple Conditioned Reinforcers for Observing Responses?

Naming refers to the incidental acquisition of word-object relations as listener and speaker without explicit reinforcement. To investigate possible sources of reinforcement for naming, we examined the effects of a procedure for conditioning reinforcement for observing responses on the emergence of naming in children who previously lacked it. The participants were three 5- to 7-year-old children with and without diagnoses of autism spectrum disorder. During the intervention, either visual or auditory stimuli were first conditioned as reinforcers for observing responses. Then, neutral visual or auditory stimuli were paired with the conditioned visual or auditory stimuli until both visual and auditory stimuli acquired reinforcing properties for observing. Following this intervention, the participants demonstrated naming of stimuli that had been used in pretests for naming, as well as on a novel set of stimuli. We observed increases in echoic responding in conjunction with the emergence of naming and conditioned reinforcement for both observing responses. We interpret the data as suggesting that listener and speaker repertoires are joined for naming only when both visual and auditory stimuli reinforce the observing responses of looking and listening simultaneously.

The effect of the magnitude of the food deprivation motivating operation on free operant preference in mice

A number of recent studies have demonstrated that organisms prefer stimuli correlated with food under high deprivation conditions over stimuli correlated with food under low deprivation conditions. The purpose of the present study was to extend the literature on this phenomenon by testing for preference under extinction conditions, testing for preference at baseline, employing a free operant preference test, and using mice as subjects. Our results appear to support the existing literature in that most subjects preferred a stimulus correlated with food under high deprivation conditions in the post-training preference test. We provide an analysis of this phenomenon based on the concept of the motivating operation (MO) and discuss how this analysis suggests a number of avenues for further research on this topic.

When animals misbehave: analogs of human biases and suboptimal choice

Humans tend to value rewards more if they have had to work hard to obtain them (justification of effort). Similarly they tend to persist in a task even when they would be better off beginning a new one (sunk cost). Humans also often give greater value to objects of good quality than the same objects together with objects of lesser quality (the less is more effect). Commercial gambling (lotteries and slot machines) is another example of suboptimal choice by humans because on average the rewards are less than the investment. In another example of a systematic bias, when humans try to estimate the probability of the occurrence of a low probability event, they often give too much weight to the results of a test, in spite of the fact that the known probability of a false alarm reduces the predictive value of the test (base rate neglect). In each of these examples, we have found that pigeons show a similar tendency to choose suboptimally. When one can show comparable findings of suboptimal choice in animals it suggests that whereas culture may reinforce certain suboptimal behavior, the behavior is likely to result from the overgeneralization of basic behavioral processes or predisposed heuristics that may have been appropriate in natural environments. This article is part of a Special Issue entitled: "Tribute to Tom Zentall."

Reprint of "Suboptimal choice by pigeons: an analog of human gambling behavior"

Human gambling often involves the choice of a low probability but high valued outcome over a high probability (certain) low valued outcome (not gambling) that is economically more optimal. We have developed an analog of gambling in which pigeons prefer a suboptimal alternative that infrequently provides a signal for a high probability (or high magnitude) of reinforcement over an optimal alternative that always provides a signal for a lower probability (or lower magnitude) of reinforcement. We have identified two mechanisms that may be responsible for this suboptimal behavior. First, the effect of nonreinforcement results in considerably less inhibition of choice than ideally it should. Second, the frequency of the occurrence of the signal for a high probability or high magnitude of reinforcement is less important than ideally it should. Also analogous to human gambling is the finding that pigeons that are normally food restricted choose suboptimally, whereas those that are minimally food restricted choose optimally. In addition, pigeons that are singly housed choose suboptimally, whereas those that are exposed to a more enriched environment choose less suboptimally. We believe that these findings have implications for the understanding and treatment of problem gambling behavior.

Observing of chain-schedule stimuli

A classical-conditioning account of the processes maintaining behavior under chained schedules entails a backward transmission of conditioned-reinforcement effects. Assessing this process in traditional chain schedules is limited because the response maintained by stimulus onset accompanied by each link in a chain schedule may also be maintained by the primary reinforcer. In the present experiment, an observing response was used to measure the conditioned-reinforcing effects of stimuli associated with a three-link chain variable-time (VT) food schedule, and resistance-to-change tests (extinction and prefeeding) were implemented to examine if a backward transmission of reinforcement effects occur. Four pigeons served as subjects. Observing was maintained by the production of stimuli correlated with links of a three-link chain VT schedule with the middle-link stimulus maintaining the highest rate of observing, followed by the initial-link stimulus and the terminal-link stimulus maintaining the lowest observing rate. Results from resistance-to-change tests of extinction and prefeeding were not supportive of a backward transmission of reinforcement effects and in general, the pattern of resistance-to-change was forward. Based on past and current research, it appears that a backward pattern of relative rate decreases in responses maintained by stimuli correlated with a chain schedule due to disruption (i.e., extinction and prefeeding) is not a ubiquitous process that is evident within different chain-schedule arrangements.

Suboptimal choice by pigeons: an analog of human gambling behavior

Human gambling often involves the choice of a low probability but high valued outcome over a high probability (certain) low valued outcome (not gambling) that is economically more optimal. We have developed an analog of gambling in which pigeons prefer a suboptimal alternative that infrequently provides a signal for a high probability (or high magnitude) of reinforcement over an optimal alternative that always provides a signal for a lower probability (or lower magnitude) of reinforcement. We have identified two mechanisms that may be responsible for this suboptimal behavior. First, the effect of nonreinforcement results in considerably less inhibition of choice than ideally it should. Second, the frequency of the occurrence of the signal for a high probability or high magnitude of reinforcement is less important than ideally it should. Also analogous to human gambling is the finding that pigeons that are normally food restricted choose suboptimally, whereas those that are minimally food restricted choose optimally. In addition, pigeons that are singly housed choose suboptimally, whereas those that are exposed to a more enriched environment choose less suboptimally. We believe that these findings have implications for the understanding and treatment of problem gambling behavior.

Do pigeons prefer information in the absence of differential reinforcement?

Prior research has indicated that pigeons do not prefer an alternative that provides a sample (for matching to sample) over an alternative that does not provide a sample (i.e., there is no indication of which comparison stimulus is correct). However, Zentall and Stagner (Journal of Experimental Psychology. Animal Behavior Processes 36:506-509, 2010) showed that when delay of reinforcement was controlled, pigeons had a strong preference for matching over pseudomatching (i.e., there was a sample, but it did not indicate which comparison stimulus was correct). Experiment 1 of the present study replicated and extended the results of the Zentall and Stagner (Journal of Experimental Psychology. Animal Behavior Processes 36:506-509, 2010) study by including an identity relation between the sample and one of the comparison stimuli in both the matching and pseudomatching tasks. In Experiment 2, in which we asked whether the pigeons would still prefer matching if we equated the two tasks for probability of reinforcement, we found no systematic preference for matching over pseudomatching. Thus, it appears that in the absence of differential reinforcement, the information provided by a sample that signals which of the two comparison stimuli is correct is insufficient to produce a preference for that alternative.

Do Pigeons Gamble? I Wouldn’t Bet Against It

Human gambling generally involves suboptimal choice because the expected return is usually less than the investment. We have found that animals, too, choose suboptimally under similar choice conditions. Pigeons, like human gamblers, show an impaired ability to objectively assess overall probabilities and amounts of reinforcement when a rare, high-value outcome (analogous to a jackpot in human gambling) is presented in the context of more frequently occurring losses. More specifically, pigeons prefer a low-probability, high-reward outcome over a guaranteed low-reward outcome with a higher overall value. Furthermore, manipulations assumed to increase impulsivity (pigeons maintained at higher levels of motivation for food and pigeons housed in individual cages) result in increased suboptimal choice. They do so presumably because they function to increase attraction to the signal for the low-probability, high-reward outcomes rather than consider the more global probability of reinforcement associated with each alternative.

Environmental enrichment affects suboptimal, risky, gambling-like choice by pigeons

Pigeons prefer a risky option with a low probability of a high payoff over a less risky option that results in more food. This finding is analogous to suboptimal human monetary gambling because in both cases there appears to be an overemphasis of the occurrence of the winning event and an underemphasis of the losing event. In the present research, we found that pigeons that were exposed to an enriched environment (a large cage with three other pigeons for 4 h a day) were less likely to show this suboptimal choice behavior compared with typically housed laboratory pigeons in a control group. These results have implications for the mechanisms underlying suboptimal choice by humans (e.g., problem gamblers), and they suggest that a enriched environment may allow for enhanced self-control.

Effects of conditioning voices as reinforcers for listener responses on rate of learning, awareness, and preferences for listening to stories in preschoolers with autism

We used a delayed non-concurrent pre- and post-intervention probe design to test the effects of a voice conditioning protocol (VCP) with 3 preschoolers with autism on (a) rate of acquisition of listener curricular objectives, (b) observing voices and the presence of adults across 3 settings, (c) selecting to listen to adults tell stories in free play setting, and (d) the occurrence of stereotypy in the story setting. The VCP conditioned voices as reinforcers for listening to recordings of voices via stimulus-stimulus pairing, which resulted in the children listening to audio recordings of voices in 90% of intervals in 5-min concurrent-operant preference tests. After voices became conditioned reinforcers, all 3 children's learning accelerated; 2 children's observing responses increased in the 3 settings; and 2 children selected to listen to stories and also showed decreased stereotypy in the story setting. The data suggest that conditioned reinforcement for observing responses may be a verbal behavior developmental cusp that acts to accelerate learning that involves listening, and that the cusp may be induced using the VCP.

Judgment and decision making: Behavioral approaches

The area of judgment and decision making has given rise to the study of many interesting phenomena, including reasoning fallacies, which are also of interest to behavior analysts. Indeed, techniques and principles of behavior analysis may be applied to study these fallacies. This article reviews research from a behavioral perspective that suggests that humans are not the information-seekers we sometimes suppose ourselves to be. Nor do we utilize information effectively when it is presented. This is shown from the results of research utilizing matching to sample and other behavioral tools (monetary reward, feedback, instructional control) to study phenomena such as the conjunction fallacy, base-rate neglect, and probability matching. Research from a behavioral perspective can complement research from other perspectives in furthering our understanding of judgment and decision making.

Reflections on stimulus control

The topic of stimulus control is too broad and complex to be traceable here. It would probably take a two-semester course to cover just the highlights of that field's evolution. The more restricted topic of equivalence relations has itself become so broad that even an introductory summary requires more time than we have available. An examination of relations between equivalence and the more general topic of stimulus control, however, may reveal characteristics of both the larger and the more limited field that have not been generally discussed. Consideration of these features may in turn foster future developments within each area. I speak, of course, about aspects of stimulus control that my own experiences have made salient to me; others would surely emphasize different characteristics of the field. It is my hope that cooperative interactions among researchers and theorists who approach stimulus control from different directions will become more common than is currently typical.

Conditioned reinforcement: Experimental and theoretical issues

The concept of conditioned reinforcement has received decreased attention in learning textbooks over the past decade, in part because of criticisms of its validity by major behavior theorists and in part because its explanatory function in a variety of different conditioning procedures has become uncertain. Critical data from the major procedures that have been used to investigate the concept (second-order schedules, chain schedules, concurrent chains, observing responses, delay-of-reinforcement procedures) are reviewed, along with the major issues of interpretation. Although the role played by conditioned reinforcement in some procedures remains unresolved, the results taken together leave little doubt that the underlying idea of conditioned value is a critical component of behavior theory that is necessary to explain many different types of data. Other processes (marking, bridging) may also operate to produce effects similar to those of conditioned reinforcement, but these clearly cannot explain the full domain of experimental effects ascribed to conditioned reinforcement and should be regarded as complements to the concept rather than theoretical competitors. Examples of practical and theoretical applications of the concept of conditioned reinforcement are also considered.

Theoretical implications of the neurotic paradox as a problem in behavior theory: An experimental resolution

Why do human phobias last for months or years when such behavior should undergo extinction? This failure of extinction or persistence of self-defeating behavior of human disorders was labeled by Mowrer as the neurotic paradox. The paradox is cited by an ever-increasing number of critics who challenge any laboratory-based learning model of human psychopathology. Laboratory research, of course, omits essential requirements in the analysis of behavior, and the principles derived from such analyses must be combined in order to explain complex human behavaior. Validation for a behavioral model can thus be achieved if (a) basic principles inferred from observation of humans treated with a laboratory-derived extinction procedure (e.g., implosive therapy) are combined with (b) principles examined in laboratory research that are combined to generate unique predictions that correspond to known features of human phobic behavior. The latter evidence is briefly reviewed in research demonstrating sustained responding over one thousand consecutive active avoidance responses with complete avoidance of the "phobic" CS for an initial single shock trial. Differential reinforcement for responses to early sequential stimuli depends on minimal work requirement, and reinforcement by timeout from avoidance. This combination of factors effectively precludes extinction to main conditioned aversive stimuli for nonhumans, as it does for human phobias. Support for a laboratory model of human phobia is thereby attained.