Dissociations between medial prefrontal cortical subregions in the modulation of learning and action

Mini-review: Prediction errors, attention and associative learning

Most modern theories of associative learning emphasize a critical role for prediction error (PE, the difference between received and expected events). One class of theories, exemplified by the Rescorla-Wagner (1972) model, asserts that PE determines the effectiveness of the reinforcer or unconditioned stimulus (US): surprising reinforcers are more effective than expected ones. A second class, represented by the Pearce-Hall (1980) model, argues that PE determines the associability of conditioned stimuli (CSs), the rate at which they may enter into new learning: the surprising delivery or omission of a reinforcer enhances subsequent processing of the CSs that were present when PE was induced. In this mini-review we describe evidence, mostly from our laboratory, for PE-induced changes in the associability of both CSs and USs, and the brain systems involved in the coding, storage and retrieval of these altered associability values. This evidence favors a number of modifications to behavioral models of how PE influences event processing, and suggests the involvement of widespread brain systems in animals' responses to PE.

Posterior parietal cortex is critical for the encoding, consolidation, and retrieval of a memory that guides attention for learning

Within most contemporary learning theories, reinforcement prediction error, the difference between the obtained and expected reinforcer value, critically influences associative learning. In some theories, this prediction error determines the momentary effectiveness of the reinforcer itself, such that the same physical event produces more learning when its presentation is surprising than when it is expected. In other theories, prediction error enhances attention to potential cues for that reinforcer by adjusting cue-specific associability parameters, biasing the processing of those stimuli so that they more readily enter into new associations in the future. A unique feature of these latter theories is that such alterations in stimulus associability must be represented in memory in an enduring fashion. Indeed, considerable data indicate that altered associability may be expressed days after its induction. Previous research from our laboratory identified brain circuit elements critical to the enhancement of stimulus associability by the omission of an expected event, and to the subsequent expression of that altered associability in more rapid learning. Here, for the first time, we identified a brain region, the posterior parietal cortex, as a potential site for a memorial representation of altered stimulus associability. In three experiments using rats and a serial prediction task, we found that intact posterior parietal cortex function was essential during the encoding, consolidation, and retrieval of an associability memory enhanced by surprising omissions. We discuss these new results in the context of our previous findings and additional plausible frontoparietal and subcortical networks.

Attention, learning, and the value of information

Despite many studies on selective attention, fundamental questions remain about its nature and neural mechanisms. Here I draw from the animal and machine learning fields that describe attention as a mechanism for active learning and uncertainty reduction and explore the implications of this view for understanding visual attention and eye movement control. I propose that a closer integration of these different views has the potential greatly to expand our understanding of oculomotor control and our ability to use this system as a window into high level but poorly understood cognitive functions, including the capacity for curiosity and exploration and for inferring internal models of the external world.

Role of amygdala central nucleus in feature negative discriminations

Consistent with a popular theory of associative learning, the Pearce-Hall (1980) model, the surprising omission of expected events enhances cue associability (the ease with which a cue may enter into new associations), across a wide variety of behavioral training procedures. Furthermore, previous experiments from this laboratory showed that these enhancements are absent in rats with impaired function of the amygdala central nucleus (CeA). A notable exception to these assertions is found in feature negative (FN) discrimination learning, in which a "target" stimulus is reinforced when it is presented alone but nonreinforced when it is presented in compound with another, "feature" stimulus. According to the Pearce-Hall model, reinforcer omission on compound trials should enhance the associability of the feature relative to control training conditions. However, prior experiments have shown no evidence that CeA lesions affect FN discrimination learning. Here we explored this apparent contradiction by evaluating the hypothesis that the surprising omission of an event confers enhanced associability on a cue only if that cue itself generates the disconfirmed prediction. Thus, in a FN discrimination, the surprising omission of the reinforcer on compound trials would enhance the associability of the target stimulus but not that of the feature. Our data confirmed this hypothesis and showed this enhancement to depend on intact CeA function, as in other procedures. The results are consistent with modern reformulations of both cue and reward processing theories that assign roles for both individual and aggregate error terms in associative learning.