Instructions and reinforcement in the observing behavior of adults and children

Free operant observing in humans: a translational approach to compulsive certainty seeking

Excessive checking is reported in non-clinical populations and is a pervasive symptom in obsessive compulsive disorder (OCD). We implemented a free-operant task in humans, previously used in rats, wherein participants can "check" to reduce uncertainty. Participants can press an observing key to ascertain which of two main keys will, if pressed, currently lead to rewards. Over a series of experiments, we found that punishment robustly increased observing in non-clinical participants and that observing persisted long after punishment was removed. Moreover, participants appeared insensitive to the initial costs of checking, and a threefold increase in the effort required to observe served to deter participants only to a limited degree. We also assessed observing in OCD patients with no known comorbidities. The patients observed more than control participants and were abnormally insensitive to the introduction of punishment. These findings support the translational value of the task, with similar behaviours in humans and rodents. This paradigm may serve as a unifying platform, promoting interaction between different approaches to analyse adaptive and maladaptive certainty seeking behaviours. Specifically, we demonstrate how seemingly disparate theoretical and empirical approaches can be reconciled synergistically to promote a combined behavioural and cognitive account of certainty seeking.

Effects of different accessibility of reinforcement schedules on choice in humans

Based on the delay-reduction hypothesis, a less profitable schedule should be rejected if its duration exceeds the mean delay to reinforcement. It should be accepted if its duration is shorter than the mean delay. This was tested for humans, using a successive-choice schedule. The accessibility of the less profitable (variable-interval 18 s) schedule was varied by changing the duration (in terms of a fixed interval) of the waiting-time component preceding its presentation. Forty-eight students were randomly assigned to three groups. In Phase 1, the duration of the less profitable schedule equaled the mean delay to reinforcement in all groups. In Phase 2, waiting time preceding the less profitable schedule was reduced in Group 1 and increased in Group 2. Thus, the schedule was correlated either with a relative delay increase (Group 1) or a delay reduction (Group 2). In Group 3, conditions remained unchanged. As predicted, acceptance of the less profitable schedule decreased in Group 1 and increased in Group 2. The increased acceptance in Group 2 was accompanied by a decreased acceptance of the more profitable (variable-interval 3 s) schedule, resembling a pattern of negative contrast. Response rates were higher under the component preceding (a) the more profitable schedule in Group 1 and (b) the less profitable schedule in Group 2. Implications for the modification of human choice behavior are discussed.

Conditioned reinforcement of human observing behavior by descriptive and arbitrary verbal stimuli

College students earned monetary reinforcers by pressing a key according to a compound schedule with variable-interval and extinction components. Pressing additional keys occasionally produced displays of either of two verbal stimuli; one was uncorrelated with the schedule components, and the other was correlated with the extinction component. In Experiments 1 and 2, the display area of the apparatus was blank unless an observing key was pressed, whereupon a descriptive message appeared. Most students preferred an uncorrelated stimulus stating that "Some of this time scores are TWICE AS LIKELY as normal, and some of this time NO SCORES can be earned" over a stimulus stating that "At this time NO SCORES can be earned." In Experiment 3, the display area indicated that "The Current Status of the Program is: NOT SHOWN." Presses on the observing keys replaced this message with stimuli that provided arbitrary labels for the schedule conditions. All of the students preferred a stimulus stating that "The Current Status of the Program is: B" over an uncorrelated stimulus stating that "The Current Status of the Program is: either A or B." Thus, under some circumstances, observing was maintained by a stimulus correlated with extinction-a finding that poses a challenge for Pavolvian accounts of conditioned reinforcement. Differences in the maintenance of observing by the descriptive and arbitrary stimuli may be attributed to differences in either the strength or nature of the instructional control exerted by the verbal stimuli.

Concurrent performance in a three-alternative choice situation: response allocation in a Rock/Paper/Scissors game

Adult human subjects engaged in a simulated Rock/Paper/Scissors game against a computer opponent. The computer opponent's responses were determined by programmed probabilities that differed across 10 blocks of 100 trials each. Response allocation in Experiment 1 was well described by a modified version of the generalized matching equation, with undermatching observed in all subjects. To assess the effects of instructions on response allocation, accurate probability-related information on how the computer was programmed to respond was provided to subjects in Experiment 2. Five of 6 subjects played the counter response of the computer's dominant programmed response near-exclusively (e.g., subjects played paper almost exclusively if the probability of rock was high), resulting in minor overmatching, and higher reinforcement rates relative to Experiment 1. On the whole, the study shows that the generalized matching law provides a good description of complex human choice in a gaming context, and illustrates a promising set of laboratory methods and analytic techniques that capture important features of human choice outside the laboratory.

The Sharing Game: Fairness in resource allocation as a function of incentive, gender, and recipient types

Economic games involving allocation of resources have been a useful tool for the study of decision making for both psychologists and economists. In two experiments involving a repeated-trials game over twenty opportunities, undergraduates made choices to distribute resources between themselves and an unseen, passive other either optimally (for themselves) but non-competitively, equally but non-optimally, or least optimally but competitively. Surprisingly, whether participants were told that the anonymous other was another student or a computer did not matter. Using such terms as “game” and “player” in the course of the session was associated with an increased frequency of competitive behavior. Males were more optimal than females: a gender-by-incentive interaction was found in the first experiment. In agreement with prior research, participants whose resources were backed by monetary incentive acted the most optimally. Overall, equality was the modal strategy employed, although it is clear that motivational context affects the allocation of resources.

How reinforcer type affects choice in economic games

Behavioral economists stress that experiments on judgment and decision-making using economic games should be played with real money if the results are to have generality. Behavior analysts have sometimes disputed this contention and have reported results in which hypothetical rewards and real money have produced comparable outcomes. We review studies that have compared hypothetical and real money and discuss the results of two relevant experiments. In the first, using the Sharing Game developed in our laboratory, subjects' choices differed markedly depending on whether the rewards were real or hypothetical. In the second, using the Ultimatum and Dictator Games, we again found sharp differences between real and hypothetical rewards. However, this study also showed that time off from a tedious task could serve as a reinforcer every bit as potent as money. In addition to their empirical and theoretical contributions, these studies make the methodological point that meaningful studies may be conducted with economic games without spending money: time off from a tedious task can serve as a powerful reward.

Influence of temporal context on value in the multiple-chains and successive-encounters procedures

This set of studies explored the influence of temporal context across multiple-chain and multiple-successive-encounters procedures. Following training with different temporal contexts, the value of stimuli sharing similar reinforcement schedules was assessed by presenting these stimuli in concurrent probes. The results for the multiple-chain schedule indicate that temporal context does impact the value of a conditioned reinforcer consistent with delay-reduction theory, such that a stimulus signaling a greater reduction in delay until reinforcement has greater value. Further, nonreinforced stimuli that are concurrently presented with the preferred terminal link also have greater value, consistent with value transfer. The effects of context on value for conditions with the multiple-successive-encounters procedure, however, appear to depend on whether the search schedule or alternate handling schedule was manipulated, as well as on whether the tested stimuli were the rich or lean schedules in their components. Overall, the results help delineate the conditions under which temporal context affects conditioned-reinforcement value (acting as a learning variable) and the conditions under which it does not (acting as a performance variable), an issue of relevance to theories of choice.

Observing behavior: effects of rate and magnitude of primary reinforcement

Four experiments examined the free-operant observing behavior of rats. In Experiment 1, observing was a bitonic function of random-ratio schedule requirements for the primary reinforcer. In Experiment 2, decreases in the magnitude of the primary reinforcer decreased observing. Experiment 3 examined observing when a random-ratio schedule or a yoked random-time schedule of primary reinforcement was in effect across conditions. Removing the response requirement for the primary reinforcer increased observing, suggesting that the effects of the random-ratio schedule in Experiment 1 likely were due to an interaction between observing and responding for the primary reinforcer. In Experiment 4, decreasing the rate of primary reinforcement by increasing the duration of a random-time schedule decreased observing monotonically. Overall, these results suggest that observing decreases with decreases in the rate or magnitude of the primary reinforcer, but that behavior related to the primary reinforcer can affect observing and potentially affect measurement of conditioned reinforcing value.

Observing behavior in a computer game

Contingencies studied in lever-pressing procedures were incorporated into a popular computer game, "Star Trek," played by college students. One putative reinforcer, the opportunity to destroy Klingon invaders, was scheduled independently of responding according to a variable-time schedule that alternated unpredictably with equal periods of Klingon unavailability (mixed variable time, extinction schedule of reinforcement). Two commands ("observing responses") each produced stimuli that were either correlated or uncorrelated with the two components. In several variations of the basic game, an S-, or bad news, was not as reinforcing as an S+, or good news. In addition, in other conditions for the same subjects observing responses were not maintained better by bad news than by an uninformative stimulus. In both choices, more observing tended to be maintained by an S- for response-independent Klingons when its information could be (and was) used to advantage with respect to other types of reinforcement in the situation (Parts 1 and 2) than when the information could not be so used (Part 3). The findings favor the conditioned reinforcement hypothesis of observing behavior over the uncertainty-reduction hypothesis. This extends research to a more natural setting and to multialternative concurrent schedules of events of seemingly intrinsic value.