Pigeons acquire the 1-back task: Implications for implicit versus explicit learning?

State-transition-free reinforcement learning in chimpanzees (Pan troglodytes)

The outcome of an action often occurs after a delay. One solution for learning appropriate actions from delayed outcomes is to rely on a chain of state transitions. Another solution, which does not rest on state transitions, is to use an eligibility trace (ET) that directly bridges a current outcome and multiple past actions via transient memories. Previous studies revealed that humans (Homo sapiens) learned appropriate actions in a behavioral task in which solutions based on the ET were effective but transition-based solutions were ineffective. This suggests that ET may be used in human learning systems. However, no studies have examined nonhuman animals with an equivalent behavioral task. We designed a task for nonhuman animals following a previous human study. In each trial, participants chose one of two stimuli that were randomly selected from three stimulus types: a stimulus associated with a food reward delivered immediately, a stimulus associated with a reward delivered after a few trials, and a stimulus associated with no reward. The presented stimuli did not vary according to the participants' choices. To maximize the total reward, participants had to learn the value of the stimulus associated with a delayed reward. Five chimpanzees (Pan troglodytes) performed the task using a touchscreen. Two chimpanzees were able to learn successfully, indicating that learning mechanisms that do not depend on state transitions were involved in the learning processes. The current study extends previous ET research by proposing a behavioral task and providing empirical data from chimpanzees.

1-Back reinforcement symbolic-matching by humans: How do they learn it?

For humans, a distinction has been made between implicit and explicit learning. Implicit learning is thought to involve automatic processes of the kind involved in much Pavlovian conditioning, while explicit learning is thought to involve conscious hypothesis testing and rule formation, in which the subject's statement of the rule has been taken as evidence of explicit learning. Various methods have been used to determine if nonverbal animals are able to learn a task explicitly - among these is the 1-back reinforcement task in which feedback from performance on the current conditional discrimination trial is provided only after completion of the following trial. We propose that it is not whether an organism can learn the task, but whether they learn it rapidly, all-or-none, that provides a better distinction between the two kinds of learning. We had humans learn a symbolic matching, 1-back reinforcement task. Almost half of the subjects failed to learn the task, and of those who did, none described the 1-back rule. Thus, it is possible to learn this task without learning the 1-back rule. Furthermore, the backward learning functions for humans differ from those of pigeons. Human subjects who learned the task did so all-or-none, suggesting explicit learning. In earlier research with pigeons, they too showed significant learning of this task; however, backward learning functions suggested that they did so gradually over the course of several sessions of training and to a lower level of asymptotic accuracy than the humans, a result suggesting implicit learning was involved.

Implicit learning of the one-back reinforcement matching-mismatching task by pigeons

Humans can learn tasks explicitly, as they can often describe the rules they have used to learn the task.^1,2,3 Animals, however, are thought to learn tasks implicitly (i.e., purely associatively).^2,3 That is, they gradually learn the correlation or association between the stimulus (or response) and the outcome. Both humans and pigeons can learn matching, where a sample stimulus indicates which one of two stimuli matches the sample. The 1-back reinforcement task is a difficult version of matching in which a correct response on trial N is rewarded only following a response on trial N + 1 (independent of the response on trial N + 1),⁴ and the correct response on trial N + 1 indicates whether a reward will occur on trial N + 2, and so forth. Humans do not appear to be able to learn the 1-back rule.⁵ Pigeons, however, do show 1-back reinforcement learning,^6,7 and they appear to do so implicitly by gradually learning the correlation between their response on one trial and the outcome on the next trial (because all other relations are uncorrelated with the outcome). They learn the task slowly and to a level below what would be expected had they learned it explicitly. The present results, together with research with humans,⁷ suggest that there are times when human explicit learning may interfere with the ability of humans to learn. Pigeons, however, are not "distracted" by attempts at explicit learning, and thus they are able to learn this and other similar tasks.^6,7,8.

1-Back reinforcement matching and mismatching by pigeons: Implicit or explicit learning?

In human learning a distinction has been made between implicit and explicit learning. Implicit learning is thought involve automatic processes of the kind involved in Pavlovian conditioning, while explicit learning is thought to involve conscious hypothesis testing and rule formation, in which the ability to report the rule used to learn the task is taken as evidence. Because non-verbal animals cannot provide such evidence, several indirect methods have been proposed. One of these methods is faster learning by humans of certain explicitly learned tasks than implicitly learned tasks, but pigeons do not show a similar difference. Another method involves the 1-back-reinforcement conditional discrimination (if A choose X, if B choose Y) in which feedback following the conditional response is delayed until the next trial. It has been argued that implicit learning cannot occur over the delay between the conditional response and the reinforcer on the next trial, yet, it has been found that monkeys can learn this 1-back reinforcement task. We have argued that such learning can occur implicitly. We have found that pigeons, a species not thought to learn explicitly, can show significant learning of both 1-back reinforcement matching and 1-back reinforcement mismatching, two versions of the 1-back-reinforcement conditional discrimination. We propose that the evidence for explicit learning by non-verbal animals suffers from alternative simpler accounts because the rationale for explicit learning is based on assumptions that likely are not correct.