Rats' preferences in the suboptimal choice procedure: Evaluating the impact of reinforcement probability and conditioned inhibitors

Paradoxical choice and the reinforcing value of information

Signals that reduce uncertainty can be valuable because well-informed decision-makers can better align their preferences to opportunities. However, some birds and mammals display an appetite for informative signals that cannot be used to increase returns. We explore the role that reward-predictive stimuli have in fostering such preferences, aiming at distinguishing between two putative underlying mechanisms. The 'information hypothesis' proposes that reducing uncertainty is reinforcing per se, somewhat consistently with the concept of curiosity: a motivation to know in the absence of tractable extrinsic benefits. In contrast, the 'conditioned reinforcement hypothesis', an associative account, proposes asymmetries in secondarily acquired reinforcement: post-choice stimuli announcing forthcoming rewards (S⁺) reinforce responses more than stimuli signalling no rewards (S^-) inhibit responses. In three treatments, rats faced two equally profitable options delivering food probabilistically after a fixed delay. In the informative option (Info), food or no food was signalled immediately after choice, whereas in the non-informative option (NoInfo) outcomes were uncertain until the delay lapsed. Subjects preferred Info when (1) both outcomes were explicitly signalled by salient auditory cues, (2) only forthcoming food delivery was explicitly signalled, and (3) only the absence of forthcoming reward was explicitly signalled. Acquisition was slower in (3), when food was not explicitly signalled, showing that signals for positive outcomes have a greater influence on the development of preference than signals for negative ones. Our results are consistent with an elaborated conditioned reinforcement account, and with the conjecture that both uncertainty reduction and conditioned reinforcement jointly act to generate preference.

The essential value of the alternatives of the suboptimal choice procedure is different for pigeons and rats

Suboptimal choice is the preference for a discriminative alternative with low probability of reinforcement, over a non-discriminative alternative with higher probability of reinforcement. Pigeons consistently prefer the discriminative alternative, whereas rats prefer the non-discriminative; the variables accounting for this difference are not yet clear. The economic concepts related to demand curves have been used to calculate the essential value of different types of reinforcers, so they may be useful to compare the value of the alternatives in the suboptimal choice procedure. The goal of this study was to calculate the essential value of each of the alternatives presented in the suboptimal choice procedure to assess if pigeons (Experiment 1) and rats (Experiment 2) value them differently. In both experiments, the fixed ratio requirement in the initial link was increased throughout sessions in order to obtain the demand curve and calculate the essential value by fitting the exponential-demand model. A Bayesian Linear Mixed-Effects Model indicated that pigeons had higher essential values for the discriminative alternative, whereas rats obtained higher essential values for the non-discriminative alternative. These results suggest that preferences in the suboptimal choice procedure are indeed based on the essential value of the alternatives, and provide a new paradigm to study the variables affecting this phenomenon.

Outcome expectancy and suboptimal risky choice in nonhuman primates

Animals will favor a risky option when a stimulus signaling reward bridges the choice and the outcome. The present experiments investigated signal-induced risky choices and reward-outcome expectations in rhesus and capuchin monkeys. Risky choice was assessed by preference for a large-probabilistic reward over a modest-certain reward. Outcome expectancy was assessed by providing a truncation-response to shorten the delay period. In Experiment 1 both species generally favored the risky option compared to a safe option when the outcomes were signaled and generally shortened the delays except when a signaled-loss stimulus was presented. The use of the delay-truncation response suggested that the monkeys were sensitive to the information conveyed by the stimulus. Experiments 2 and 3 were designed to investigate whether the delay-truncation response used by capuchin monkeys was strategically used reflecting explicit decision-making versus a conditioned response to reward stimuli. A perceptual judgment task was included and the selective use of the delay-truncation response on unsignaled correct trials may suggest the involvement of metacognitive processes. The capuchin monkeys generally truncated the delays except under conditions where reward would not be expected (risky-loss or incorrect-judgment). When the outcomes were unsignaled during the delay some capuchin monkeys were less likely to truncate the delay following an incorrect task response. Overall, the monkeys: (1) made more risky choices when the outcomes were signaled - consistent with gambling-like behavior. (2) selectively truncated the unsignaled delays when rewards could be anticipated (even when metacognitive-like awareness guided anticipation) - suggesting that delay truncation responses reflect explicit outcome expectancy.

The functional equivalence of two variants of the suboptimal choice task: choice proportion and response latency as measures of value

In the suboptimal-choice task, birds systematically choose the leaner but informative option (suboptimal) over the richer but non-informative option (optimal). The task has two variations. In the standard task, the optimal option includes two terminal link stimuli. In the original task, it includes a single terminal link stimulus. Two models, the temporal information account (Cunningham and Shahan, J Exp Psychol Anim Learn Cogn 44:1-22, 2018) and the ∆-∑ hypothesis (González et al., J Exp Anal Behav 113:591-608, 2020), presuppose that these procedures are equivalent, but no formal comparison is available. Here we test whether or not these procedures are functionally equivalent. One group of pigeons was trained with the standard procedure, another group with the original procedure, and a third group was trained with a hybrid of the other two (i.e., the two options were the optimal links of the standard and original procedures). Our findings indicate that the number of terminal link stimuli in the optimal option is inconsequential vis-à-vis choice. Moreover, our findings also indicate that latencies to respond are a sensitive metric of value and choice. As predicted by the Sequential Choice Model, we were able to predict simultaneous choices from the latencies of sequential choices and observed a substantial shortening of latencies during simultaneous choices.

Suboptimal choice by pigeons is eliminated when key-pecking behavior is replaced by treadle-pressing

In the study of suboptimal choice, a reliable result is that pigeons strongly prefer an alternative that signals whether a reinforcer will be delivered or not over another alternative without that information even if the first provides a lower probability of reinforcement. In the aforementioned research, key pecking has been the operant response and illuminated keys the discriminative stimuli. In the present study we modified both of these aspects of the procedure in order to analyze the generality of suboptimal preferences of pigeons and to investigate the effect of changes in the incentive salience of the discriminative stimuli. To accomplish this, we presented pigeons a choice situation with the same parameters of reinforcement than previous research, but with treadle pressing as the choice response and ambient lights as discriminative stimuli. Under these conditions, most of the pigeons showed optimal behavior and a high degree of discrimination of the stimuli associated with the discriminative alternative. A control condition with key pecking as choice response and keylights as discriminative stimuli showed that the same pigeons turned to be suboptimal, a result that discards the possibility that the optimality found in the main condition was a consequence of a particular characteristic of our sample of subjects or of our procedure. We discuss the influence that the attribution of incentive salience to the discriminative stimuli has on suboptimal choice in both pigeons and rats.

Rats' optimal choice behavior in a gambling-like task

Among the different procedures that model gambling behavior in non-human animals, the "suboptimal choice procedure" has been extensively employed for analyzing the impact of environmental cues on choice behavior. It has been repeatedly demonstrated that pigeons prefer an alternative that infrequently presents a stimulus that signals a larger amount of reinforcement, than another alternative that always presents a stimulus associated with a smaller amount of reinforcement, even though the net rate of reinforcement is lower in the former. In the present study, we tested rats in the magnitude version of the suboptimal choice procedure. Eight rats were given a choice between two alternatives: a) one in which a stimulus predicting the delivery of ten pellets was presented with probability (p) = 0.2 and a stimulus predicting zero pellets was presented with p = 0.8, and b) one in which either of two stimuli predicted the delivery of three pellets with p = 1.0. Contrary to the consistent and robust suboptimal behavior of pigeons, rats preferred the optimal alternative. This effect occurred despite the high index of discrimination of the stimuli associated with the different outcomes shown by the rats. The relevance of this result to the development of animal models of gambling behavior is discussed.