From conditioning to category learning: An adaptive network model

The error in total error reduction

Most models of human and animal learning assume that learning is proportional to the discrepancy between a delivered outcome and the outcome predicted by all cues present during that trial (i.e., total error across a stimulus compound). This total error reduction (TER) view has been implemented in connectionist and artificial neural network models to describe the conditions under which weights between units change. Electrophysiological work has revealed that the activity of dopamine neurons is correlated with the total error signal in models of reward learning. Similar neural mechanisms presumably support fear conditioning, human contingency learning, and other types of learning. Using a computational modeling approach, we compared several TER models of associative learning to an alternative model that rejects the TER assumption in favor of local error reduction (LER), which assumes that learning about each cue is proportional to the discrepancy between the delivered outcome and the outcome predicted by that specific cue on that trial. The LER model provided a better fit to the reviewed data than the TER models. Given the superiority of the LER model with the present data sets, acceptance of TER should be tempered.

Hippocampal activation during extinction learning predicts occurrence of the renewal effect in extinction recall

The renewal effect describes the reoccurrence of a previously extinguished response in situations where the context of extinction differs from that of acquisition, thus illustrating the context-dependency of extinction learning. A number of studies on contextual fear extinction have implicated hippocampus and vmPFC in processing and retrieval of context both during extinction learning and recall of extinction. In this functional magnetic resonance imaging (fMRI) study we explored the neural correlates of the renewal effect in associative learning, using a predictive learning task that required participants to learn relations between cues and outcomes presented in particular contexts. During extinction in a novel context, compared to extinction in a context identical to the acquisition context, participants who exhibited the renewal effect (REN) showed increased activation in brain regions including bilateral posterior hippocampus and left parahippocampal gyrus. This activation pattern was absent in participants that did not show the renewal effect (NOREN). In direct comparisons between the groups, the REN group exhibited higher activation in bilateral hippocampus, while the NOREN group showed higher activation in left dlPFC (BA 46) and right anterior cingulate (BA 32). During extinction recall, stimuli that had been extinguished in a different context were again presented in the context of acquisition. Here both groups exhibited predominantly prefrontal activation, with the REN group's focus upon bilateral OFC (BA 47) and bilateral vmPFC (BA 10), while the NOREN group showed generally more widespread activation, predominantly in large clusters of dlPFC (BA 8,9,45). In a direct comparison, the REN group showed higher activation than the NOREN group in left vmPFC (BA 10), while NOREN participants exhibited more activation in dlPFC (BA 9, 46). Activation in left vmPFC during extinction recall correlated with the number of renewal effect responses, while the dlPFC activation showed a negative correlation with renewal effect responses. These results highlight the differential activation patterns of processes that will eventually produce or not produce a renewal effect, indicating that during extinction learning hippocampus encodes the relation between context and cue-outcome, while in extinction recall vmPFC is active to retrieve this association.

Stress impairs retrieval of extinguished and unextinguished associations in a predictive learning task

Recovery effects which can frequently be observed after a seemingly successful extinction procedure indicate that extinction does not lead to an erasure of the memory trace. Investigating factors which modulate the retrieval of extinction memory is highly relevant for basic science and clinical applications alike. This study investigated the effect of stress on the retrieval of extinguished and unextinguished stimulus-outcome associations in a predictive learning task. In this task, participants had to imagine being the doctor of a patient who sometimes suffers from stomach trouble after meals in his favorite restaurants. They were presented with different food stimuli while having to predict the occurrence or non-occurrence of stomach trouble. As extinction memory is modulated by context, we manipulated contextual cues so that initial acquisition of critical associations occurred in context (restaurant frame) A on day one, whereas associations were reversed in context B (extinction, day two). On the third day, participants were either stressed (exposed to the socially evaluated cold pressor task (SECPT); n=21) or subjected to a control condition (n=21) shortly before extinction memory retrieval was tested (in contexts A and B). Salivary cortisol and blood pressure measures as well as subjective ratings indicated that stress induction was successful. When retrieval of extinguished associations was tested on day three, participants' predictions reflected a renewal effect, as indicated by stronger recovery of responding in the acquisition context compared to the extinction context. Compared to controls, stressed participants showed impaired retrieval of extinguished and unextinguished associations. Contextual cues abolished the stress-induced memory impairment for unextinguished but not for extinguished associations. These findings might help to explain why stress leads to the reoccurrence of symptoms in affective disorders.

Neural evidence for adaptive strategy selection in value-based decision-making

In everyday life, humans often encounter complex environments in which multiple sources of information can influence their decisions. We propose that in such situations, people select and apply different strategies representing different cognitive models of the decision problem. Learning advances by evaluating the success of using a strategy and eventually by switching between strategies. To test our strategy selection model, we investigated how humans solve a dynamic learning task with complex auditory and visual information, and assessed the underlying neural mechanisms with functional magnetic resonance imaging. Using the model, we were able to capture participants' choices and to successfully attribute expected values and reward prediction errors to activations in the dopaminoceptive system (e.g., ventral striatum [VS]) as well as decision conflict to signals in the anterior cingulate cortex. The model outperformed an alternative approach that did not update decision strategies, but the relevance of information itself. Activation of sensory areas depended on whether the selected strategy made use of the respective source of information. Selection of a strategy also determined how value-related information influenced effective connectivity between sensory systems and the VS. Our results suggest that humans can structure their search for and use of relevant information by adaptively selecting between decision strategies.

Probabilistically-Cued Patterns Trump Perfect Cues in Statistical Language Learning

Probabilistically-cued co-occurrence relationships between word categories are common in natural languages but difficult to acquire. For example, in English, determiner-noun and auxiliary-verb dependencies both involve co-occurrence relationships, but determiner-noun relationships are more reliably marked by correlated distributional and phonological cues, and appear to be learned more readily. We tested whether experience with co-occurrence relationships that are more reliable promotes learning those that are less reliable using an artificial language paradigm. Prior experience with deterministically-cued contingencies did not promote learning of less reliably-cued structure, nor did prior experience with relationships instantiated in the same vocabulary. In contrast, prior experience with probabilistically-cued co-occurrence relationships instantiated in different vocabulary did enhance learning. Thus, experience with co-occurrence relationships sharing underlying structure but not vocabulary may be an important factor in learning grammatical patterns. Furthermore, experience with probabilistically-cued co-occurrence relationships, despite their difficultly for naïve learners, lays an important foundation for learning novel probabilistic structure.

Categorization of birds, mammals, and chimeras by pigeons

Identifying critical features that control categorization of complex polymorphous pictures by animals remains a challenging and important problem. Toward this goal, experiments were conducted to isolate the properties controlling the categorization of two pictorial categories by pigeons. Pigeons were trained in a go/no-go task to categorize black and white line drawings of birds and mammals. They were then tested with a variety of familiar and novel exemplars of these categories to examine the features controlling this categorization. These tests suggested the pigeons were segregating and using the principal axis of orientation of the animal figures as the primary means of discriminating each category, although other categorical and item-specific cues were likely involved. This perceptual/cognitive reduction of the categorical stimulus space to a few visual features or dimensions is likely a characteristic of this species' processing of complex pictorial discrimination problems and is a critical property for theoretical accounts of this behavior.

Normal and impaired spelling in a connectionist dual-route architecture

This paper presents a dual-route connectionist model of spelling, in which one route maps directly from sound to spelling (phonemes to graphemes), while in the other route the mapping is mediated by a further level of representation. The direct route is implemented as a two-layer associative network, with syllabically structured phonemic (input) and graphemic (output) representations, which comes to behave as a productive sound-to-spelling conversion mechanism through the exposure to a corpus of monosyllabic words. The mediated route is modelled as a frequency-sensitive lexical pathway. Nodes representing more frequent words become activated more rapidly than those of lower-frequency words. Access to both routes occurs in parallel, and the final spelling is determined by the combined output of both routes. We show that the model accounts for a wide range of data from normal spellers (including nonword spelling, the variability in vowel spelling and the effect of surrounding phonological context, frequency effect and its interaction with spelling regularity). We also investigate the effect of a selective lesion to the lexical route in which the ceiling of lexical activation is lowered. This manipulation produces a model with surface dysgraphic characteristics, which is tested against data from two impaired subjects. As well as simulating the classic surface dysgraphic profile, including a frequency by regularity interaction, the model exhibits a phenomenon that has only recently been reported, and which provides strong evidence for the idea that multiple routes are active in parallel, and combine to produce the final spelling.

Equilibria of perceptrons for simple contingency problems

The contingency between cues and outcomes is fundamentally important to theories of causal reasoning and to theories of associative learning. Researchers have computed the equilibria of Rescorla-Wagner models for a variety of contingency problems, and have used these equilibria to identify situations in which the Rescorla-Wagner model is consistent, or inconsistent, with normative models of contingency. Mathematical analyses that directly compare artificial neural networks to contingency theory have not been performed, because of the assumed equivalence between the Rescorla-Wagner learning rule and the delta rule training of artificial neural networks. However, recent results indicate that this equivalence is not as straightforward as typically assumed, suggesting a strong need for mathematical accounts of how networks deal with contingency problems. One such analysis is presented here, where it is proven that the structure of the equilibrium for a simple network trained on a basic contingency problem is quite different from the structure of the equilibrium for a Rescorla-Wagner model faced with the same problem. However, these structural differences lead to functionally equivalent behavior. The implications of this result for the relationships between associative learning, contingency theory, and connectionism are discussed.

An instance theory of associative learning

We present and test an instance model of associative learning. The model, Minerva-AL, treats associative learning as cued recall. Memory preserves the events of individual trials in separate traces. A probe presented to memory contacts all traces in parallel and retrieves a weighted sum of the traces, a structure called the echo. Learning of a cue-outcome relationship is measured by the cue's ability to retrieve a target outcome. The theory predicts a number of associative learning phenomena, including acquisition, extinction, reacquisition, conditioned inhibition, external inhibition, latent inhibition, discrimination, generalization, blocking, overshadowing, overexpectation, superconditioning, recovery from blocking, recovery from overshadowing, recovery from overexpectation, backward blocking, backward conditioned inhibition, and second-order retrospective revaluation. We argue that associative learning is consistent with an instance-based approach to learning and memory.

Modeling learned categorical perception in human vision

A long standing debate in cognitive neuroscience has been the extent to which perceptual processing is influenced by prior knowledge and experience with a task. A converging body of evidence now supports the view that a task does influence perceptual processing, leaving us with the challenge of understanding the locus of, and mechanisms underpinning, these influences. An exemplar of this influence is learned categorical perception (CP), in which there is superior perceptual discrimination of stimuli that are placed in different categories. Psychophysical experiments on humans have attempted to determine whether early cortical stages of visual analysis change as a result of learning a categorization task. However, while some results indicate that changes in visual analysis occur, the extent to which earlier stages of processing are changed is still unclear. To explore this issue, we develop a biologically motivated neural model of hierarchical vision processes consisting of a number of interconnected modules representing key stages of visual analysis, with each module learning to exhibit desired local properties through competition. With this system level model, we evaluate whether a CP effect can be generated with task influence to only the later stages of visual analysis. Our model demonstrates that task learning in just the later stages is sufficient for the model to exhibit the CP effect, demonstrating the existence of a mechanism that requires only a high-level of task influence. However, the effect generalizes more widely than is found with human participants, suggesting that changes to earlier stages of analysis may also be involved in the human CP effect, even if these are not fundamental to the development of CP. The model prompts a hybrid account of task-based influences on perception that involves both modifications to the use of the outputs from early perceptual analysis along with the possibility of changes to the nature of that early analysis itself.

Linking neurogenetics and individual differences in language learning: the dopamine hypothesis

Fundamental advances in neuroscience have come from investigations into neuroplasticity and learning. These investigations often focus on identifying universal principles across different individuals of the same species. Increasingly, individual differences in learning success have also been observed, such that any seemingly universal principle might only be applicable to a certain extent within a particular learner. One potential source of this variation is individuals' genetic differences. Adult language learning provides a unique opportunity for understanding individual differences and genetic bases of neuroplasticity because of the large individual differences in learning success that have already been documented, and because of the body of empirical work connecting language learning and neurocognition. In this article, we review the literature on the genetic bases of neurocognition, especially studies examining polymorphisms of dopamine (DA)-related genes and procedural learning. This review leads us to hypothesize that there may be an association between DA-related genetic variation and language learning differences. If this hypothesis is supported by future empirical findings we suggest that it may point to neurogenetic markers that allow for language learning to be personalized.

Granularity and the acquisition of grammatical gender: how order-of-acquisition affects what gets learned

Why do adult language learners typically fail to acquire second languages with native proficiency? Does prior linguistic experience influence the size of the "units" adults attend to in learning, and if so, how does this influence what gets learned? Here, we examine these questions in relation to grammatical gender, which adult learners almost invariably struggle to master. We present a model of learning that predicts that exposure to smaller units (such as nouns) before exposure to larger linguistic units (such as sentences) can critically impair learning about predictive relations between units: such as that between a noun and its article. This prediction is then confirmed by a study of adult participants learning grammatical gender in an artificial language. Adults learned both nouns and their articles better when they were first heard nouns used in context with their articles prior to hearing the nouns individually, compared with learners who first heard the nouns in isolation, prior to hearing them used in context. In the light of these results, we discuss the role gender appears to play in language, the importance of meaning in artificial grammar learning, and the implications of this work for the structure of L2-training.

Event-related brain responses as correlates of changes in predictive and affective values of conditioned stimuli

Previous evidence suggests that the judged predictive strength of one cue may be influenced by the predictive strengths of other pretrained cues (prediction errors). In the present study, we examined affective ratings and event-related brain responses from 18 healthy participants during an aversive conditioning task in which affective values of previously trained conditioned stimuli were modified through a blocking procedure. The task was divided into two phases. During the training phase, single stimulus A (e.g., red square) was always followed by aversive picture stimuli, while single stimulus B (e.g., yellow square) was signaling the absence of aversive stimulation. During the blocking phase, compound stimuli consisted of the combination of one single trained stimulus (A or B) and one new somatosensory stimulus were also followed by the presence of aversive stimulation. Results indicated that single stimulus A elicited greater ERP amplitudes and theta power, and was rated as more unpleasant than single stimulus B during the training phase. Moreover, single stimulus B elicited greater ERP amplitudes than stimulus A, as well as greater theta power and more unpleasant ratings during the blocking as compared with the training phase. By contrast, no changes in ERP amplitudes and theta power were observed for stimulus A. Our findings provide neurophysiological and behavioral evidence for an increased affective processing of conditioned stimuli when compound stimuli were introduced, but only if the target CS was previously trained to signal the absence of aversive stimulation.

Uncertainty and cognitive control

A growing trend of neuroimaging, behavioral, and computational research has investigated the topic of outcome uncertainty in decision-making. Although evidence to date indicates that humans are very effective in learning to adapt to uncertain situations, the nature of the specific cognitive processes involved in the adaptation to uncertainty are still a matter of debate. In this article, we reviewed evidence suggesting that cognitive control processes are at the heart of uncertainty in decision-making contexts. Available evidence suggests that: (1) There is a strong conceptual overlap between the constructs of uncertainty and cognitive control; (2) There is a remarkable overlap between the neural networks associated with uncertainty and the brain networks subserving cognitive control; (3) The perception and estimation of uncertainty might play a key role in monitoring processes and the evaluation of the "need for control"; (4) Potential interactions between uncertainty and cognitive control might play a significant role in several affective disorders.

Negative Transfer in Human Associative Learning

Models of attentional allocation in associative learning are typically structured according to one of two guiding principles: the predictiveness principle, which posits that attention is paid to cues that have reliably predicted an outcome in the past, or the uncertainty principle, which states that attention is paid to cues about which little is known. Both principles are well supported by studies of animals. However, in studies of human learning, there is very little direct empirical support for the uncertainty principle. In the study reported here, we addressed this gap by investigating negative transfer, a phenomenon that may provide unique support for the uncertainty principle. In two human learning experiments using an allergist task, we replicated the primary findings of previous research on animal learning. We believe that these data provide the first direct evidence for the uncertainty principle in human associative learning.

Cognitive impairment in acquired brain injury: a predictor of rehabilitation outcomes and an opportunity for novel interventions

Cognitive impairment is a common sequela in acquired brain injury and one that predicts rehabilitation outcomes. There is emerging evidence that impairments in cognitive functions can be manipulated by both pharmacologic and nonpharmacologic interventions to improve rehabilitation outcomes. By using stroke as a model for acquired brain injury, we review the evidence that links cognitive impairment to poor rehabilitation outcomes and discuss possible mechanisms to explain this association. Furthermore, we examine nascent promising research that suggests that interventions that target cognitive impairments can lead to better rehabilitation outcomes.

Differentiable contributions of human amygdalar subregions in the computations underlying reward and avoidance learning

To understand how the human amygdala contributes to associative learning, it is necessary to differentiate the contributions of its subregions. However, major limitations in the techniques used for the acquisition and analysis of functional magnetic resonance imaging (fMRI) data have hitherto precluded segregation of function with the amygdala in humans. Here, we used high-resolution fMRI in combination with a region-of-interest-based normalization method to differentiate functionally the contributions of distinct subregions within the human amygdala during two different types of instrumental conditioning: reward and avoidance learning. Through the application of a computational-model-based analysis, we found evidence for a dissociation between the contributions of the basolateral and centromedial complexes in the representation of specific computational signals during learning, with the basolateral complex contributing more to reward learning, and the centromedial complex more to avoidance learning. These results provide unique insights into the computations being implemented within fine-grained amygdala circuits in the human brain.

Making the illusory correlation effect appear and then disappear: The effects of increased learning

The acquisition of a negative evaluation of a fictitious minority social group in spite of the absence of any objective correlation between group membership and negative behaviours was described by Hamilton and Gifford (1976) as an instance of an illusory correlation. We studied the acquisition and attenuation through time of this correlation learning effect. In two experiments we asked for participants’ judgements of two fictitious groups using an online version of a group membership belief paradigm. We tested how judgements of the two groups changed as a function of the amount of training they received. Results suggest that the perception of the illusory correlation effect is initially absent, emerges with intermediate amounts of absolute experience, but diminishes and is eliminated with increased experience. This illusory correlation effect can be considered to reflect incomplete learning rather than a bias due to information loss in judgements or distinctiveness.

Missing the forest for the trees: object-discrimination learning blocks categorization learning

Growing evidence indicates that error-driven associative learning underlies the ability of nonhuman animals to categorize natural images. This study explored whether this form of learning might also be at play when people categorize natural objects in photographs. Two groups of college students (a blocking group and a control group) were trained on a categorization task and then tested with novel photographs from each category; however, only the blocking group received pretraining on a task that required the discrimination of objects from the same category. Because of this earlier noncategorical discrimination learning, the blocking group performed well in the categorization task from the outset, and this strong initial performance reduced the likelihood of category learning driven by error. There was far less transfer of categorical responding during testing in the blocking group than in the control group; this finding suggests that learning the specific properties of each photographic image in pretraining blocked later learning of an open-ended category.

Intact implicit learning in autism spectrum conditions

Individuals with autism spectrum condition (ASC) have diagnostic impairments in skills that are associated with an implicit acquisition; however, it is not clear whether ASC individuals show specific implicit learning deficits. We compared ASC and typically developing (TD) individuals matched for IQ on five learning tasks: four implicit learning tasks--contextual cueing, serial reaction time, artificial grammar learning, and probabilistic classification learning tasks--that used procedures expressly designed to minimize the use of explicit strategies, and one comparison explicit learning task, paired associates learning. We found implicit learning to be intact in ASC. Beyond no evidence of differences, there was evidence of statistical equivalence between the groups on all the implicit learning tasks. This was not a consequence of compensation by explicit learning ability or IQ. Furthermore, there was no evidence to relate implicit learning to ASC symptomatology. We conclude that implicit mechanisms are preserved in ASC and propose that it is disruption by other atypical processes that impact negatively on the development of skills associated with an implicit acquisition.

Error-driven learning in visual categorization and object recognition: a common-elements model

A wealth of empirical evidence has now accumulated concerning animals' categorizing photographs of real-world objects. Although these complex stimuli have the advantage of fostering rapid category learning, they are difficult to manipulate experimentally and to represent in formal models of behavior. We present a solution to the representation problem in modeling natural categorization by adopting a common-elements approach. A common-elements stimulus representation, in conjunction with an error-driven learning rule, can explain a wide range of experimental outcomes in animals' categorization of naturalistic images. The model also generates novel predictions that can be empirically tested. We report 2 experiments that show how entirely hypothetical representational elements can nevertheless be subject to experimental manipulation. The results represent the first evidence of error-driven learning in natural image categorization, and they support the idea that basic associative processes underlie this important form of animal cognition.

Conditional routing of information to the cortex: a model of the basal ganglia's role in cognitive coordination

The basal ganglia play a central role in cognition and are involved in such general functions as action selection and reinforcement learning. Here, we present a model exploring the hypothesis that the basal ganglia implement a conditional information-routing system. The system directs the transmission of cortical signals between pairs of regions by manipulating separately the selection of sources and destinations of information transfers. We suggest that such a mechanism provides an account for several cognitive functions of the basal ganglia. The model also incorporates a possible mechanism by which subsequent transfers of information control the release of dopamine. This signal is used to produce novel stimulus-response associations by internalizing transferred cortical representations in the striatum. We discuss how the model is related to production systems and cognitive architectures. A series of simulations is presented to illustrate how the model can perform simple stimulus-response tasks, develop automatic behaviors, and provide an account of impairments in Parkinson's and Huntington's diseases.

Studying the use of base rates: Normal science or shifting paradigm?

The underutilization of base rates is a consistent finding. The strong claim that base rates are ignored has been rejected and this needs no further emphasis. Following the path of “normal science,” research examines the conditions predicting changes in the degree of underutilization. A scientific revolution that might dethrone the heuristics and biases paradigm is not in sight.

The base rate fallacy reconsidered: Descriptive, normative, and methodological challenges

We have been oversold on the base rate fallacy in probabilistic judgment from an empirical, normative, and methodological standpoint. At the empirical level, a thorough examination of the base rate literature (including the famous lawyer–engineer problem) does not support the conventional wisdom that people routinely ignore base rates. Quite the contrary, the literature shows that base rates are almost always used and that their degree of use depends on task structure and representation. Specifically, base rates play a relatively larger role in tasks where base rates are implicitly learned or can be represented in frequentist terms. Base rates are also used more when they are reliable and relatively more diagnostic than available individuating information. At the normative level, the base rate fallacy should be rejected because few tasks map unambiguously into the narrow framework that is held up as the standard of good decision making. Mechanical applications of Bayes's theorem to identify performance errors are inappropriate when (1) key assumptions of the model are either unchecked or grossly violated, and (2) no attempt is made to identify the decision maker's goals, values, and task assumptions. Methodologically, the current approach is criticized for its failure to consider how the ambiguous, unreliable, and unstable base rates of the real world are and should be used. Where decision makers' assumptions and goals vary, and where performance criteria are complex, the traditional Bayesian standard is insufficient. Even where predictive accuracy is the goal in commonly defined problems, there may be situations (e.g., informationally redundant environments) in which base rates can be ignored with impunity. A more ecologically valid research program is called for. This program should emphasize the development of prescriptive theory in rich, realistic decision environments.

The implicit use of base rates in experiential and ecologically valid tasks

When base rates are learned and used in an experiential manner subjects show better base rate use, perhaps because the implicit learning system is engaged. A causal framework in which base rates are relevant might also be necessary. Humans might thus perform better on more ecologically valid tasks, which are likely to contain those three components.

Base rates do not constrain nonprobability judgments

Base rates have no necessary relation to judgments that are not themselves probabilities. There is no logical imperative, for instance, that behavioral base rates must affect causal attributions or that base rate information should affect judgments of legal liability. Decision theorists should be cautious in arguing that base rates place normative constraints on judgments of anything other than posterior probabilities.

Judgment under uncertainty: Evolution may not favor a probabilistic calculus

The environment in which humans evolved is strongly and positively autocorrelated in space and time. Probabilistic judgments based on the assumption of independence may not yield evolutionarily adaptive behavior. A number of “faults” of human reasoning are not faulty under fuzzy arithmetic, a nonprobabilistic calculus of reasoning under uncertainty that may be closer to that underlying human decision making.

Fallacy and controversy about base rates

Koehler's target article attempts a balanced view of the relevance of knowledge of base rates to judgments of subjective or credal probability, but he is not sensitive enough to the difference between requiring and permitting the equation of probability judgments with base rates, the interaction between precision of base rate and reference class information, and the possibility of indeterminate probability judgment.