The Role of the Basal Ganglia in Category Learning

Movement-related activity in the internal globus pallidus of the parkinsonian macaque

Although the basal ganglia (BG) plays a central role in the motor symptoms of Parkinson's disease, few studies have investigated the influence of parkinsonism on movement-related activity in the BG. Here, we studied the perimovement activity of neurons in globus pallidus internus (GPi) of non-human primates before and after the induction of parkinsonism by administration of 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP). Neuronal responses were equally common in the parkinsonian brain as seen prior to MPTP and the distribution of different response types was largely unchanged. The slowing of behavioral reaction times and movement durations following the induction of parkinsonism was accompanied by a prolongation of the time interval between neuronal response onset and movement initiation. Neuronal responses were also reduced in magnitude and prolonged in duration after the induction of parkinsonism. Importantly, those two effects were more pronounced among decrease-type responses, and they persisted after controlling for MPTP-induced changes in the trial-by-trial timing of neuronal responses. Following MPTP The timing of neuronal responses also became uncoupled from the time of movement onset and more variable from trial-to-trial. Overall, the effects of MPTP on temporal features of neural responses correlated most consistently with the severity of parkinsonian motor impairments whereas the changes in response magnitude and duration were either anticorrelated with symptom severity or inconsistent. These findings point to a potential previously underappreciated role for abnormalities in the timing of GPi task-related activity in the generation of parkinsonian motor signs.

The gradual coevolution of syntactic combinatorics and categorization under the effects of human self-domestication: a proposal

The gradual emergence of syntax has been claimed to be engaged in a feedback loop with Human Self-Domestication (HSD), both processes resulting from, and contributing to, enhanced connectivity in selected cortico-striatal networks, which is the mechanism for attenuating reactive aggression, the hallmark of HSD, but also the mechanism of cross-modality, relevant for syntax. Here, we aim to bridge the gap between these brain changes and further changes facilitated by the gradual complexification of grammars. We propose that increased cross-modality would have enabled and supported, more specifically, a feedback loop between categorization abilities relevant for vocabulary building and the gradual emergence of syntactic structure, including Merge. In brief, an enhanced categorization ability not only brings about more distinct categories, but also a critical number of tokens in each category necessary for Merge to take off in a systematic and productive fashion; in turn, the benefits of expressive capabilities brought about by productive Merge encourage more items to be categorized, and more categories to be formed, thus further potentiating categorization abilities, and with it, syntax again. We support our hypothesis with evidence from the domains of language development and animal communication, but also from biology, neuroscience, paleoanthropology, and clinical linguistics.

A computational model of prefrontal and striatal interactions in perceptual category learning

Work on multiple-system theories of cognition mostly focused on the systems themselves, while limited work has been devoted to understanding the interactions between systems. Generally, multiple-system theories include a model-based decision system supported by the prefrontal cortex and a model-free decision system supported by the striatum. Here we propose a neurobiological model to describe the interactions between model-based and model-free decision systems in category learning. The proposed model used spiking neurons to simulate activity of the hyperdirect pathway of the basal ganglia. The hyperdirect pathway acts as a gate for the response signal from the model-free system located in the striatum. We propose that the model-free system's response is inhibited when the model-based system is in control of the response. The new model was used to simulate published data from young adults, people with Parkinson's disease, and aged-matched older adults. The simulation results further suggest that system-switching ability may be related to individual differences in executive function. A new behavioral experiment tested this model prediction. The results show that an updating score predicts the ability to switch system in a categorization task. The article concludes with new model predictions and implications of the results for research on system interactions.

Hierarchical categorization learning is associated with representational changes in the dorsal striatum and posterior frontal and parietal cortex

Learning and recognition can be improved by sorting novel items into categories and subcategories. Such hierarchical categorization is easy when it can be performed according to learned rules (e.g., "if car, then automatic or stick shift" or "if boat, then motor or sail"). Here, we present results showing that human participants acquire categorization rules for new visual hierarchies rapidly, and that, as they do, corresponding hierarchical representations of the categorized stimuli emerge in patterns of neural activation in the dorsal striatum and in posterior frontal and parietal cortex. Participants learned to categorize novel visual objects into a hierarchy with superordinate and subordinate levels based on the objects' shape features, without having been told the categorization rules for doing so. On each trial, participants were asked to report the category and subcategory of the object, after which they received feedback about the correctness of their categorization responses. Participants trained over the course of a one-hour-long session while their brain activation was measured using functional magnetic resonance imaging. Over the course of training, significant hierarchy learning took place as participants discovered the nested categorization rules, as evidenced by the occurrence of a learning trial, after which performance suddenly increased. This learning was associated with increased representational strength of the newly acquired hierarchical rules in a corticostriatal network including the posterior frontal and parietal cortex and the dorsal striatum. We also found evidence suggesting that reinforcement learning in the dorsal striatum contributed to hierarchical rule learning.

Dorsomedial striatum, but not dorsolateral striatum, is necessary for rat category learning

Categorization is an adaptive cognitive function that allows us to generalize knowledge to novel situations. Converging evidence from neuropsychological, neuroimaging, and neurophysiological studies suggest that categorization is mediated by the basal ganglia; however, there is debate regarding the necessity of each subregion of the basal ganglia and their respective functions. The current experiment examined the roles of the dorsomedial striatum (DMS; homologous to the head of the caudate nucleus) and dorsolateral striatum (DLS; homologous to the body and tail of the caudate nucleus) in category learning by combining selective lesions with computational modeling. Using a touchscreen apparatus, rats were trained to categorize distributions of visual stimuli that varied along two continuous dimensions (i.e., spatial frequency and orientation). The tasks either required attention to one stimulus dimension (spatial frequency or orientation; 1D tasks) or both stimulus dimensions (spatial frequency and orientation; 2D tasks). Rats with NMDA lesions of the DMS were impaired on both the 1D tasks and 2D tasks, whereas rats with DLS lesions showed no impairments. The lesions did not affect performance on a discrimination task that had the same trial structure as the categorization tasks, suggesting that the category impairments effected processes relevant to categorization. Model simulations were conducted using a neural network to assess the effect of the DMS lesions on category learning. Together, the results suggest that the DMS is critical to map category representations to appropriate behavioral responses, whereas the DLS is not necessary for categorization.

Impaired and Spared Auditory Category Learning in Developmental Dyslexia

Categorization has a deep impact on behavior, but whether category learning is served by a single system or multiple systems remains debated. Here, we designed two well-equated nonspeech auditory category learning challenges to draw on putative procedural (information-integration) versus declarative (rule-based) learning systems among adult Hebrew-speaking control participants and individuals with dyslexia, a language disorder that has been linked to a selective disruption in the procedural memory system and in which phonological deficits are ubiquitous. We observed impaired information-integration category learning and spared rule-based category learning in the dyslexia group compared with the neurotypical group. Quantitative model-based analyses revealed reduced use of, and slower shifting to, optimal procedural-based strategies in dyslexia with hypothesis-testing strategy use on par with control participants. The dissociation is consistent with multiple category learning systems and points to the possibility that procedural learning inefficiencies across categories defined by complex, multidimensional exemplars may result in difficulty in phonetic category acquisition in dyslexia.

An adaptive linear filter model of procedural category learning

We use a feature-based association model to fit grouped and individual level category learning and transfer data. The model assumes that people use corrective feedback to learn individual feature to categorization-criterion correlations and combine those correlations additively to produce classifications. The model is an Adaptive Linear Filter (ALF) with logistic output function and Least Mean Squares learning algorithm. Categorization probabilities are computed by a logistic function. Our data span over 31 published data sets. Both at grouped and individual level analysis levels, the model performs remarkably well, accounting for large amounts of available variances. When fitted to grouped data, it outperforms alternative models. When fitted to individual level data, it is able to capture learning and transfer performance with high explained variances. Notably, the model achieves its fits with a very minimal number of free parameters. We discuss the ALF's advantages as a model of procedural categorization, in terms of its simplicity, its ability to capture empirical trends and its ability to solve challenges to other associative models. In particular, we discuss why the model is not equivalent to a prototype model, as previously thought.

Delaying feedback compensates for impaired reinforcement learning in developmental dyslexia

A theoretical framework suggests that developmental dyslexia is characterized by abnormalities in brain structures underlying the procedural learning and memory systems while the declarative learning and memory systems are presumed to remain intact or even enhanced (Procedural Deficit Hypothesis). This notion has been supported by a substantial body of research, which focused on each system independently. However, less attention has been paid to interactions between these memory systems which may provide insights as to learning situations and conditions in which learning in dyslexia can be improved. The current study was undertaken to examine these important but unresolved issues. To this end, probabilistic reinforcement learning and episodic memory tasks were examined in participants with dyslexia and neurotypicals simultaneously within a single task. Feedback timing presentation was manipulated, building on prior research indicating that delaying feedback timing shifts striatal-based probabilistic learning, to become more hippocampal-dependent. It was hypothesized that if the procedural learning and memory systems are impaired in dyslexia, performance will be impaired under conditions that encourage procedural memory engagement (immediate feedback trials) but not under conditions that promote declarative memory processing (long delayed feedback trials). It was also predicted that the ability to incidentally acquire episodic information would be preserved in dyslexia. The results supported these predictions. Participants with dyslexia were impaired in probabilistic learning of cue-outcome associations compared to neurotypicals in an immediate feedback condition, but not when feedback on choices was presented after a long delay. Furthermore, participants with dyslexia demonstrated similar performance to neurotypicals in a task requiring incidental episodic memory formation. These findings attest to a dissociation between procedural-based and declarative-based learning in developmental dyslexia within a single task, a finding that adds discriminative validity to the Procedural Deficit Hypothesis. Just as important, the present findings suggest that training conditions designed to shift the load from midbrain/striatal systems to declarative memory mechanisms have the potential to compensate for impaired learning in developmental dyslexia.

Pigeons acquire the 1-back task: Implications for implicit versus explicit learning?

In humans, a distinction can be made between implicit or procedural learning (involving stimulus-response associations) and explicit or declarative learning (involving verbalizable rules) that is relatively easy to make in verbal humans. According to several investigators, it is also possible to make such a distinction in nonverbal animals. One way is by training them on a conditional discrimination task (e.g., matching-to-sample) in which reinforcement for correct choice on the current trial is delayed until after a choice is made on the next trial - a method known as the 1-back procedure. According to Smith, Jackson, and Church ( Journal of Comparative Psychology, 134(4), 423-434, 2020), the delay between the sample-correct-comparison response on one trial and reinforcement obtained on the next trial is too long for implicit (associative) learning. Thus, according to this theory, learning must be explicit. In the present experiments we trained pigeons using the 1-back procedure. In Experiment 1, pigeons were trained on red/green 1-back matching using a non-correction procedure. Some of the pigeons showed significant learning. When a correction procedure was introduced, all the pigeons showed evidence of learning. In Experiment 2, new pigeons learned red/green 1-back matching with the correction procedure. In Experiment 3, new pigeons learned symbolic 1-back matching with yellow and blue conditional stimuli and red/green choice stimuli. Thus, pigeons can learn using 1-back reinforcement. Although it would appear that the pigeons acquired this task explicitly, we believe that this procedure does not adequately distinguish between implicit and explicit learning.

Modulation of Dopamine for Adaptive Learning: A Neurocomputational Model

There have been many proposals that learning rates in the brain are adaptive, in the sense that they increase or decrease depending on environmental conditions. The majority of these models are abstract and make no attempt to describe the neural circuitry that implements the proposed computations. This article describes a biologically detailed computational model that overcomes this shortcoming. Specifically, we propose a neural circuit that implements adaptive learning rates by modulating the gain on the dopamine response to reward prediction errors, and we model activity within this circuit at the level of spiking neurons. The model generates a dopamine signal that depends on the size of the tonically active dopamine neuron population and the phasic spike rate. The model was tested successfully against results from two single-neuron recording studies and a fast-scan cyclic voltammetry study. We conclude by discussing the general applicability of the model to dopamine mediated tasks that transcend the experimental phenomena it was initially designed to address.

A role for the medial temporal lobes in category learning

Despite much research, the role of the medial temporal lobes (MTL) in category learning is unclear. Two unstructured categorization experiments explored conditions that might recruit MTL category learning and memory systems—namely, whether the stimulus display includes one or two stimuli, and whether category membership depends on configural properties of the stimulus features. The results supported three conclusions. First, in agreement with prior research, learning with single stimulus displays depended on striatal-mediated procedural learning. Second, and most important, learning with pair displays was mediated by MTL declarative memory systems. Third, the use of stimuli in which category membership depends on configural properties of the stimulus features made MTL learning slightly more likely. Overall, the results suggested that the MTL are most likely to mediate learning when the participant must decide which of two configural stimuli belongs to a selected category.

The role of differential outcomes-based feedback on procedural memory

In categorization tasks, two memory systems may be involved in the learning of categories: one explicit and rule-based system and another implicit and procedure-based system. Learning of rule-based categories relies on some form of explicit reasoning, whereas procedural memory underlies information-integration category-learning tasks, in which performance is maximized only if information of two (or more) dimensions is integrated. The present study aimed at investigating the role of how feedback is administered, whether differential or non-differential, in procedural learning. An information-integration category-learning task was designed, where the to-be-categorized stimuli differed in two dimensions. Participants were randomly assigned to two groups: one group received the reinforcers for correct categorizations differentially, one for each category (the differential outcomes procedure, DOP), and the other group received the reinforcers randomly (the non-differential outcomes procedure, NOP). The participants of the DOP group showed better procedural learning in the categorization task, compared to the NOP group. Moreover, the analysis of learning strategies revealed that more participants developed more optimal strategies in the DOP group than in the NOP group. These results extend the benefits of the differential outcomes-based feedback to non-declarative memory tasks and help better understand the role of feedback in procedural learning.

From learning to action: the integration of dorsal striatal input and output pathways in instrumental conditioning

Considerable evidence suggests that the learning and performance of instrumental actions depend on activity in basal ganglia circuitry; however, these two functions have generally been considered independently. Whereas research investigating the associative mechanisms underlying instrumental conditioning has identified critical cortical and limbic input pathways to the dorsal striatum, the performance of instrumental actions has largely been attributed to activity in the dorsal striatal output pathways, with direct and indirect pathway projection neurons mediating action initiation, perseveration and cessation. Here, we discuss evidence that the dorsal striatal input and basal ganglia output pathways mediate the learning and performance of instrumental actions, respectively, with the dorsal striatum functioning as a transition point. From this perspective, the issue of how multiple striatal inputs are integrated at the level of the dorsal striatum and converted into relatively restricted outputs becomes one of critical significance for understanding how learning is translated into action. So too does the question of how learning signals are modulated by recent experience. We propose that this occurs through recurrent corticostriatothalamic feedback circuits that serve to integrate performance signals by updating ongoing action-related learning.

Retinal-specific category learning

Virtually all cognitive theories of category learning (such as prototype theory^1-5 and exemplar theory^6-8) view this important skill as a high-level process that uses abstract representations of objects in the world. Because these representations are removed from visual characteristics of the display, such theories suggest that category learning occurs in higher-level (such as association) areas and therefore should be immune to the visual field dependencies that characterize processing of objects mediated by representations in low-level visual areas. Here we challenge that view by describing a fully controlled demonstration of visual-field dependence in category learning. Eye-tracking was used to control gaze while participants either learned rule-based categories known to recruit prefrontal-based explicit reasoning, or information-integration categories known to depend on basal-ganglia-mediated procedural learning⁹. Results showed that learning was visual-field dependent with information-integration categories, but we found no evidence of visual-field dependence with rule-based categories. A theoretical interpretation of this difference is offered in terms of the underlying neurobiology. Finally, these results are situated within the broad perceptual-learning literature in an attempt to motivate further research on the similarities and differences between category and perceptual learning.

Cognitive changes in conjunctive rule-based category learning: An ERP approach

When learning rule-based categories, sufficient cognitive resources are needed to test hypotheses, maintain the currently active rule in working memory, update rules after feedback, and to select a new rule if necessary. Prior research has demonstrated that conjunctive rules are more complex than unidimensional rules and place greater demands on executive functions like working memory. In our study, event-related potentials (ERPs) were recorded while participants performed a conjunctive rule-based category learning task with trial-by-trial feedback. In line with prior research, correct categorization responses resulted in a larger stimulus-locked late positive complex compared to incorrect responses, possibly indexing the updating of rule information in memory. Incorrect trials elicited a pronounced feedback-locked P300 elicited which suggested a disconnect between perception, and the rule-based strategy. We also examined the differential processing of stimuli that were able to be correctly classified by the suboptimal single-dimensional rule ("easy" stimuli) versus those that could only be correctly classified by the optimal, conjunctive rule ("difficult" stimuli). Among strong learners, a larger, late positive slow wave emerged for difficult compared with easy stimuli, suggesting differential processing of category items even though strong learners performed well on the conjunctive category set. Overall, the findings suggest that ERP combined with computational modelling can be used to better understand the cognitive processes involved in rule-based category learning.

Stress Elevates Frontal Midline Theta in Feedback-based Category Learning of Exceptions

Adapting behavior based on category knowledge is a fundamental cognitive function, which can be achieved via different learning strategies relying on different systems in the brain. Whereas the learning of typical category members has been linked to implicit, prototype abstraction learning, which relies predominantly on prefrontal areas, the learning of exceptions is associated with explicit, exemplar-based learning, which has been linked to the hippocampus. Stress is known to foster implicit learning strategies at the expense of explicit learning. Procedural, prefrontal learning and cognitive control processes are reflected in frontal midline theta (4–8 Hz) oscillations during feedback processing. In the current study, we examined the effect of acute stress on feedback-based category learning of typical category members and exceptions and the oscillatory correlates of feedback processing in the EEG. A computational modeling procedure was applied to estimate the use of abstraction and exemplar strategies during category learning. We tested healthy, male participants who underwent either the socially evaluated cold pressor test or a nonstressful control procedure before they learned to categorize typical members and exceptions based on feedback. The groups did not differ significantly in their categorization accuracy or use of categorization strategies. In the EEG, however, stressed participants revealed elevated theta power specifically during the learning of exceptions, whereas the theta power during the learning of typical members did not differ between the groups. Elevated frontal theta power may reflect an increased involvement of medial prefrontal areas in the learning of exceptions under stress.

Perceptual category learning and visual processing: An exercise in computational cognitive neuroscience

The field of computational cognitive neuroscience (CCN) builds and tests neurobiologically detailed computational models that account for both behavioral and neuroscience data. This article leverages a key advantage of CCN-namely, that it should be possible to interface different CCN models in a plug-and-play fashion-to produce a new and biologically detailed model of perceptual category learning. The new model was created from two existing CCN models: the HMAX model of visual object processing and the COVIS model of category learning. Using bitmap images as inputs and by adjusting only a couple of learning-rate parameters, the new HMAX/COVIS model provides impressively good fits to human category-learning data from two qualitatively different experiments that used different types of category structures and different types of visual stimuli. Overall, the model provides a comprehensive neural and behavioral account of basal ganglia-mediated learning.

Hierarchical control of procedural and declarative category-learning systems

Substantial evidence suggests that human category learning is governed by the interaction of multiple qualitatively distinct neural systems. In this view, procedural memory is used to learn stimulus-response associations, and declarative memory is used to apply explicit rules and test hypotheses about category membership. However, much less is known about the interaction between these systems: how is control passed between systems as they interact to influence motor resources? Here, we used fMRI to elucidate the neural correlates of switching between procedural and declarative categorization systems. We identified a key region of the cerebellum (left Crus I) whose activity was bidirectionally modulated depending on switch direction. We also identified regions of the default mode network (DMN) that were selectively connected to left Crus I during switching. We propose that the cerebellum-in coordination with the DMN-serves a critical role in passing control between procedural and declarative memory systems.

Trial-by-trial switching between procedural and declarative categorization systems

Considerable evidence suggests that human category learning recruits multiple memory systems. A popular assumption is that procedural memory is used to form stimulus-to-response mappings, whereas declarative memory is used to form and test explicit rules about category membership. The multiple systems framework has been successful in motivating and accounting for a broad array of empirical observations over the past 20 years. Even so, only a couple of studies have examined how the different categorization systems interact. Both previous studies suggest that switching between explicit and procedural responding is extremely difficult. But they leave unanswered the critical questions of whether trial-by-trial system switching is possible, and if so, whether it is qualitatively different than trial-by-trial switching between two explicit tasks. The experiment described in this article addressed these questions. The results (1) confirm that effective trial-by-trial system switching, although difficult, is possible; (2) suggest that switching between tasks mediated by different memory systems is more difficult than switching between two declarative memory tasks; and (3) point to a serious shortcoming of current category-learning theories.

Dissociable changes in functional network topology underlie early category learning and development of automaticity

Recent work has shown that multimodal association areas-including frontal, temporal, and parietal cortex-are focal points of functional network reconfiguration during human learning and performance of cognitive tasks. On the other hand, neurocomputational theories of category learning suggest that the basal ganglia and related subcortical structures are focal points of functional network reconfiguration during early learning of some categorization tasks but become less so with the development of automatic categorization performance. Using a combination of network science and multilevel regression, we explore how changes in the connectivity of small brain regions can predict behavioral changes during training in a visual categorization task. We find that initial category learning, as indexed by changes in accuracy, is predicted by increasingly efficient integrative processing in subcortical areas, with higher functional specialization, more efficient integration across modules, but a lower cost in terms of redundancy of information processing. The development of automaticity, as indexed by changes in the speed of correct responses, was predicted by lower clustering (particularly in subcortical areas), higher strength (highest in cortical areas), and higher betweenness centrality. By combining neurocomputational theories and network scientific methods, these results synthesize the dissociative roles of multimodal association areas and subcortical structures in the development of automaticity during category learning.

Multiple stages of learning in perceptual categorization: evidence and neurocomputational theory

Virtually all current theories of category learning assume that humans learn new categories by gradually forming associations directly between stimuli and responses. In information-integration category-learning tasks, this purported process is thought to depend on procedural learning implemented via dopamine-dependent cortical-striatal synaptic plasticity. This article proposes a new, neurobiologically detailed model of procedural category learning that, unlike previous models, does not assume associations are made directly from stimulus to response. Rather, the traditional stimulus-response (S-R) models are replaced with a two-stage learning process. Multiple streams of evidence (behavioral, as well as anatomical and fMRI) are used as inspiration for the new model, which synthesizes evidence of multiple distinct cortical-striatal loops into a neurocomputational theory. An experiment is reported to test a priori predictions of the new model that: (1) recovery from a full reversal should be easier than learning new categories equated for difficulty, and (2) reversal learning in procedural tasks is mediated within the striatum via dopamine-dependent synaptic plasticity. The results confirm the predictions of the new two-stage model and are incompatible with existing S-R models.

What is automatized during perceptual categorization?

An experiment is described that tested whether stimulus-response associations or an abstract rule are automatized during extensive practice at perceptual categorization. Twenty-seven participants each completed 12,300 trials of perceptual categorization, either on rule-based (RB) categories that could be learned explicitly or information-integration (II) categories that required procedural learning. Each participant practiced predominantly on a primary category structure, but every third session they switched to a secondary structure that used the same stimuli and responses. Half the stimuli retained their same response on the primary and secondary categories (the congruent stimuli) and half switched responses (the incongruent stimuli). Several results stood out. First, performance on the primary categories met the standard criteria of automaticity by the end of training. Second, for the primary categories in the RB condition, accuracy and response time (RT) were identical on congruent and incongruent stimuli. In contrast, for the primary II categories, accuracy was higher and RT was lower for congruent than for incongruent stimuli. These results are consistent with the hypothesis that rules are automatized in RB tasks, whereas stimulus-response associations are automatized in II tasks. A cognitive neuroscience theory is proposed that accounts for these results.

One Giant Leap for Categorizers: One Small Step for Categorization Theory

We explore humans’ rule-based category learning using analytic approaches that highlight their psychological transitions during learning. These approaches confirm that humans show qualitatively sudden psychological transitions during rule learning. These transitions contribute to the theoretical literature contrasting single vs. multiple category-learning systems, because they seem to reveal a distinctive learning process of explicit rule discovery. A complete psychology of categorization must describe this learning process, too. Yet extensive formal-modeling analyses confirm that a wide range of current (gradient-descent) models cannot reproduce these transitions, including influential rule-based models (e.g., COVIS) and exemplar models (e.g., ALCOVE). It is an important theoretical conclusion that existing models cannot explain humans’ rule-based category learning. The problem these models have is the incremental algorithm by which learning is simulated. Humans descend no gradient in rule-based tasks. Very different formal-modeling systems will be required to explain humans’ psychology in these tasks. An important next step will be to build a new generation of models that can do so.

Structure and function of large-scale brain systems

This article introduces the functional neuroanatomy of large-scale brain systems. Both the structure and functions of these brain networks are presented. All human behavior is the result of interactions within and between these brain systems. This system of brain function completely changes our understanding of how cognition and behavior are organized within the brain, replacing the traditional lesion model. Understanding behavior within the context of brain network interactions has profound implications for modifying abstract constructs such as attention, learning, and memory. These constructs also must be understood within the framework of a paradigm shift, which emphasizes ongoing interactions within a dynamically changing environment.

Large-scale brain systems and subcortical relationships: the vertically organized brain

This article reviews the vertical organization of the brain. The cortico-basal ganglia and the cerebro-cerebellar circuitry systems are described as fundamental to cognitive and behavioral control. The basal ganglia anticipate and guide implicitly learned behaviors on the basis of experienced reward outcomes. The cerebellar-cortical network anticipates sensorimotor outcomes, allowing behaviors to be adapted across changing settings and across contexts. These vertically organized systems, operating together, represent the underpinning of cognitive control. The medial temporal lobe system, and its development, is also reviewed in order to better understand how brain systems interact for both implicit and explicit cognitive control.

The C957T polymorphism in the dopamine receptor D₂ gene modulates domain-general category learning

Adaptive learning from reward and punishment is vital for human survival. Striatal and frontal dopaminergic activities are associated with adaptive learning. For example, the C957T single nucleotide polymorphism of the dopamine receptor D2 (DRD2) gene alters striatal D2 receptor availability and affects individuals' adaptive learning ability. Specifically, individuals with the T/T genotype, which is associated with higher striatal D2 availability, show enhanced learning from negative outcomes. Prior work examining DRD2 genetic variability has focused primarily on frontally mediated reflective learning that is under effortful, conscious control. However, less is known about a more automatic, striatally mediated reflexive learning. Here we examined the extent to which this polymorphism differentially influences reflective and reflexive learning across visual and auditory modalities. We employed rule-based (RB) and information-integration (II) category learning paradigms that target reflective and reflexive learning, respectively. Results revealed an advantage in II category learning but poorer RB category learning in T/T homozygotes. The pattern of results was consistent across sensory modalities. These findings suggest that this DRD2 polymorphism exerts opposite influences on domain-general frontally mediated reflective learning and striatally mediated reflexive learning.

Ego depletion interferes with rule-defined category learning but not non-rule-defined category learning

Considerable research on category learning has suggested that many cognitive and environmental factors can have a differential effect on the learning of rule-defined (RD) categories as opposed to the learning of non-rule-defined (NRD) categories. Prior research has also suggested that ego depletion can temporarily reduce the capacity for executive functioning and cognitive flexibility. The present study examined whether temporarily reducing participants’ executive functioning via a resource depletion manipulation would differentially impact RD and NRD category learning. Participants were either asked to write a story with no restrictions (the control condition), or without using two common letters (the ego depletion condition). Participants were then asked to learn either a set of RD categories or a set of NRD categories. Resource depleted participants performed more poorly than controls on the RD task, but did not differ from controls on the NRD task, suggesting that self regulatory resources are required for successful RD category learning. These results lend support to multiple systems theories and clarify the role of self-regulatory resources within this theory.

Interpretive monitoring in the caudate nucleus

In a dynamic environment an organism has to constantly adjust ongoing behavior to adapt to a given context. This process requires continuous monitoring of ongoing behavior to provide its meaningful interpretation. The caudate nucleus is known to have a role in behavioral monitoring, but the nature of these signals during dynamic behavior is still unclear. We recorded neuronal activity in the caudate nucleus in monkeys during categorization behavior that changed rapidly across contexts. We found that neuronal activity maintained representation of the identity and context of a recently categorized stimulus, as well as interpreted the behavioral meaningfulness of the maintained trace. The accuracy of this cognitive monitoring signal was highest for behavior for which subjects were prone to make errors. Thus, the caudate nucleus provides interpretive monitoring of ongoing behavior, which is necessary for contextually specific decisions to adapt to rapidly changing conditions.

DOI:http://dx.doi.org/10.7554/eLife.03727.001

The ability to adapt behavior in a changing environment is a hallmark of intelligent systems. From adjusting our driving speed to match road conditions to responding to a last-minute change of plans, mental flexibility underpins much of our day-to-day functioning.

To perform optimally, an animal must continuously monitor its own behavior and adjust it according to circumstances. A region of the brain called the caudate nucleus is thought to contribute to this process by keeping track of the relation between an action and its outcomes, but it is not clear how it monitors cognitive aspects of ongoing behavior.

Yanike and Ferrera have clarified this process by recording electrical activity from the caudate nucleus in two monkeys as they categorized visual stimuli. The monkeys viewed a moving stimulus and classified it as ‘fast’ or ‘slow’ relative to a reference speed that varied from trial to trial. The monkeys were trained to use two different references speeds and were told which reference speed to use at the start of each trial. They used an eye movement to indicate their decision.

Most neurons within the caudate nucleus responded after the monkey had made a decision, suggesting that these neurons might be involved in evaluating the decision that had just been made. The response of the neurons depended on the stimulus speed, and also on the category (fast or slow) in which the stimulus belonged. This observation indicates that the caudate nucleus tracked the context (reference speed) as well as the stimulus speed.

Yanike and Ferrera also showed that the response of the entire population of caudate neurons could be decoded to reveal both the speed of the stimulus and whether the monkey had categorized it as fast or slow. This shows that after a decision has been made, neurons continue to signal both the stimulus and the context in which that stimulus was presented. Such ‘post-decision’ monitoring is important for anticipating the outcome of the decision. Overall the results suggest that the caudate nucleus helps animals to adapt their behavior to rapidly changing circumstances by supporting decision-making that takes context into account.

DOI:http://dx.doi.org/10.7554/eLife.03727.002

The Role of Corticostriatal Systems in Speech Category Learning

One of the most difficult category learning problems for humans is learning nonnative speech categories. While feedback-based category training can enhance speech learning, the mechanisms underlying these benefits are unclear. In this functional magnetic resonance imaging study, we investigated neural and computational mechanisms underlying feedback-dependent speech category learning in adults. Positive feedback activated a large corticostriatal network including the dorsolateral prefrontal cortex, inferior parietal lobule, middle temporal gyrus, caudate, putamen, and the ventral striatum. Successful learning was contingent upon the activity of domain-general category learning systems: the fast-learning reflective system, involving the dorsolateral prefrontal cortex that develops and tests explicit rules based on the feedback content, and the slow-learning reflexive system, involving the putamen in which the stimuli are implicitly associated with category responses based on the reward value in feedback. Computational modeling of response strategies revealed significant use of reflective strategies early in training and greater use of reflexive strategies later in training. Reflexive strategy use was associated with increased activation in the putamen. Our results demonstrate a critical role for the reflexive corticostriatal learning system as a function of response strategy and proficiency during speech category learning.

Tests of a Dual-systems Model of Speech Category Learning

In the visual domain, more than two decades of work posits the existence of dual category learning systems. The reflective system uses working memory to develop and test rules for classifying in an explicit fashion. The reflexive system operates by implicitly associating perception with actions that lead to reinforcement. Dual-systems models posit that in learning natural categories, learners initially use the reflective system and with practice, transfer control to the reflexive system. The role of reflective and reflexive systems in second language (L2) speech learning has not been systematically examined. Here monolingual, native speakers of American English were trained to categorize Mandarin tones produced by multiple talkers. Our computational modeling approach demonstrates that learners use reflective and reflexive strategies during tone category learning. Successful learners use talker-dependent, reflective analysis early in training and reflexive strategies by the end of training. Our results demonstrate that dual-learning systems are operative in L2 speech learning. Critically, learner strategies directly relate to individual differences in category learning success.

Elevated depressive symptoms enhance reflexive but not reflective auditory category learning

In vision an extensive literature supports the existence of competitive dual-processing systems of category learning that are grounded in neuroscience and are partially-dissociable. The reflective system is prefrontally-mediated and uses working memory and executive attention to develop and test rules for classifying in an explicit fashion. The reflexive system is striatally-mediated and operates by implicitly associating perception with actions that lead to reinforcement. Although categorization is fundamental to auditory processing, little is known about the learning systems that mediate auditory categorization and even less is known about the effects of individual difference in the relative efficiency of the two learning systems. Previous studies have shown that individuals with elevated depressive symptoms show deficits in reflective processing. We exploit this finding to test critical predictions of the dual-learning systems model in audition. Specifically, we examine the extent to which the two systems are dissociable and competitive. We predicted that elevated depressive symptoms would lead to reflective-optimal learning deficits but reflexive-optimal learning advantages. Because natural speech category learning is reflexive in nature, we made the prediction that elevated depressive symptoms would lead to superior speech learning. In support of our predictions, individuals with elevated depressive symptoms showed a deficit in reflective-optimal auditory category learning, but an advantage in reflexive-optimal auditory category learning. In addition, individuals with elevated depressive symptoms showed an advantage in learning a non-native speech category structure. Computational modeling suggested that the elevated depressive symptom advantage was due to faster, more accurate, and more frequent use of reflexive category learning strategies in individuals with elevated depressive symptoms. The implications of this work for dual-process approach to auditory learning and depression are discussed.

Toward a dual-learning systems model of speech category learning

More than two decades of work in vision posits the existence of dual-learning systems of category learning. The reflective system uses working memory to develop and test rules for classifying in an explicit fashion, while the reflexive system operates by implicitly associating perception with actions that lead to reinforcement. Dual-learning systems models hypothesize that in learning natural categories, learners initially use the reflective system and, with practice, transfer control to the reflexive system. The role of reflective and reflexive systems in auditory category learning and more specifically in speech category learning has not been systematically examined. In this article, we describe a neurobiologically constrained dual-learning systems theoretical framework that is currently being developed in speech category learning and review recent applications of this framework. Using behavioral and computational modeling approaches, we provide evidence that speech category learning is predominantly mediated by the reflexive learning system. In one application, we explore the effects of normal aging on non-speech and speech category learning. Prominently, we find a large age-related deficit in speech learning. The computational modeling suggests that older adults are less likely to transition from simple, reflective, unidimensional rules to more complex, reflexive, multi-dimensional rules. In a second application, we summarize a recent study examining auditory category learning in individuals with elevated depressive symptoms. We find a deficit in reflective-optimal and an enhancement in reflexive-optimal auditory category learning. Interestingly, individuals with elevated depressive symptoms also show an advantage in learning speech categories. We end with a brief summary and description of a number of future directions.

Prior experience with negative spectral correlations promotes information integration during auditory category learning

Complex sounds vary along a number of acoustic dimensions. These dimensions may exhibit correlations that are familiar to listeners due to their frequent occurrence in natural sounds-namely, speech. However, the precise mechanisms that enable the integration of these dimensions are not well understood. In this study, we examined the categorization of novel auditory stimuli that differed in the correlations of their acoustic dimensions, using decision bound theory. Decision bound theory assumes that stimuli are categorized on the basis of either a single dimension (rule based) or the combination of more than one dimension (information integration) and provides tools for assessing successful integration across multiple acoustic dimensions. In two experiments, we manipulated the stimulus distributions such that in Experiment 1, optimal categorization could be accomplished by either a rule-based or an information integration strategy, while in Experiment 2, optimal categorization was possible only by using an information integration strategy. In both experiments, the pattern of results demonstrated that unidimensional strategies were strongly preferred. Listeners focused on the acoustic dimension most closely related to pitch, suggesting that pitch-based categorization was given preference over timbre-based categorization. Importantly, in Experiment 2, listeners also relied on a two-dimensional information integration strategy, if there was immediate feedback. Furthermore, this strategy was used more often for distributions defined by a negative spectral correlation between stimulus dimensions, as compared with distributions with a positive correlation. These results suggest that prior experience with such correlations might shape short-term auditory category learning.

Automaticity and multiple memory systems

A large number of criteria have been proposed for determining when a behavior has become automatic. Almost all of these were developed before the widespread acceptance of multiple memory systems. Consequently, popular frameworks for studying automaticity often neglect qualitative differences in how different memory systems guide initial learning. Unfortunately, evidence suggests that automaticity criteria derived from these frameworks consistently misclassify certain sets of initial behaviors as automatic. Specifically, criteria derived from cognitive science mislabel much behavior still under the control of procedural memory as automatic, and criteria derived from animal learning mislabel some behaviors under the control of declarative memory as automatic. Even so, neither set of criteria make the opposite error-that is, both sets correctly identify any automatic behavior as automatic. In fact, evidence suggests that although there are multiple memory systems and therefore multiple routes to automaticity, there might nevertheless be only one common representation for automatic behaviors. A number of possible cognitive and cognitive neuroscience models of this single automaticity system are reviewed. WIREs Cogn Sci 2012, 3:363-376. doi: 10.1002/wcs.1172 For further resources related to this article, please visit the WIREs website.

Information-integration category learning and the human uncertainty response

The human response to uncertainty has been well studied in tasks requiring attention and declarative memory systems. However, uncertainty monitoring and control have not been studied in multi-dimensional, information-integration categorization tasks that rely on non-declarative procedural memory. Three experiments are described that investigated the human uncertainty response in such tasks. Experiment 1 showed that following standard categorization training, uncertainty responding was similar in information-integration tasks and rule-based tasks requiring declarative memory. In Experiment 2, however, uncertainty responding in untrained information-integration tasks impaired the ability of many participants to master those tasks. Finally, Experiment 3 showed that the deficit observed in Experiment 2 was not because of the uncertainty response option per se, but rather because the uncertainty response provided participants a mechanism via which to eliminate stimuli that were inconsistent with a simple declarative response strategy. These results are considered in the light of recent models of category learning and metacognition.

Adaptation, expertise, and giftedness: towards an understanding of cortical, subcortical, and cerebellar network contributions

Current cortico-centric models of cognition lack a cohesive neuroanatomic framework that sufficiently considers overlapping levels of function, from "pathological" through "normal" to "gifted" or exceptional ability. While most cognitive theories presume an evolutionary context, few actively consider the process of adaptation, including concepts of neurodevelopment. Further, the frequent co-occurrence of "gifted" and "pathological" function is difficult to explain from a cortico-centric point of view. This comprehensive review paper proposes a framework that includes the brain's vertical organization and considers "giftedness" from an evolutionary and neurodevelopmental vantage point. We begin by discussing the current cortico-centric model of cognition and its relationship to intelligence. We then review an integrated, dual-tiered model of cognition that better explains the process of adaptation by simultaneously allowing for both stimulus-based processing and higher-order cognitive control. We consider the role of the basal ganglia within this model, particularly in relation to reward circuitry and instrumental learning. We review the important role of white matter tracts in relation to speed of adaptation and development of behavioral mastery. We examine the cerebellum's critical role in behavioral refinement and in cognitive and behavioral automation, particularly in relation to expertise and giftedness. We conclude this integrated model of brain function by considering the savant syndrome, which we believe is best understood within the context of a dual-tiered model of cognition that allows for automaticity in adaptation as well as higher-order executive control.

Cortical and striatal contributions to automaticity in information-integration categorization

In information-integration categorization, accuracy is maximized only if information from two or more stimulus components is integrated at some pre-decisional stage. In many cases the optimal strategy is difficult or impossible to describe verbally. Evidence suggests that success in information-integration tasks depends on procedural learning that is mediated largely within the striatum. Although many studies have examined initial information-integration learning, little is known about how automaticity develops in information-integration tasks. To address this issue, each of ten human participants received feedback training on the same information-integration categories for more than 11,000 trials spread over 20 different training sessions. Sessions 2, 4, 10, and 20 were performed inside an MRI scanner. The following results stood out. 1) Automaticity developed between sessions 10 and 20. 2) Pre-automatic performance depended on the putamen, but not on the body and tail of the caudate nucleus. 3) Automatic performance depended only on cortical regions, particularly the supplementary and pre-supplementary motor areas. 4) Feedback processing was mainly associated with deactivations in motor and premotor regions of cortex, and in the ventral lateral prefrontal cortex. 5) The overall effects of practice were consistent with the existing literature on the development of automaticity.

Human category learning 2.0

During the 1990s and early 2000s, cognitive neuroscience investigations of human category learning focused on the primary goal of showing that humans have multiple category-learning systems and on the secondary goals of identifying key qualitative properties of each system and of roughly mapping out the neural networks that mediate each system. Many researchers now accept the strength of the evidence supporting multiple systems, and as a result, during the past few years, work has begun on the second generation of research questions-that is, on questions that begin with the assumption that humans have multiple category-learning systems. This article reviews much of this second generation of research. Topics covered include (1) How do the various systems interact? (2) Are there different neural systems for categorization and category representation? (3) How does automaticity develop in each system? and (4) Exactly how does each system learn?

Rule-based and information-integration category learning in normal aging

The basal ganglia and prefrontal cortex play critical roles in category learning. Both regions evidence age-related structural and functional declines. The current study examined rule-based and information-integration category learning in a group of older and younger adults. Rule-based learning is thought to involve explicit, frontally mediated processes, whereas information-integration is thought to involve implicit, striatally mediated processes. As a group, older adults showed rule-based and information-integration deficits. A series of models were applied that provided insights onto the type of strategy used to solve the task. Interestingly, when the analyses focused only on participants who used the task appropriate strategy in the final block of trials, the age-related rule-based deficit disappeared whereas the information-integration deficit remained. For this group of individuals, the final block information-integration deficit was due to less consistent application of the task appropriate strategy by older adults, and over the course of learning these older adults shifted from an explicit hypothesis-testing strategy to the task appropriate strategy later in learning. In addition, the use of the task appropriate strategy was associated with less interference and better inhibitory control for rule-based and information-information learning, whereas use of the task appropriate strategy was associated with greater working memory and better new verbal learning only for the rule-based task. These results suggest that normal aging impacts both forms of category learning and that there are some important similarities and differences in the explanatory locus of these deficits. The data also support a two-component model of information-integration category learning that includes a striatal component that mediated procedural-based learning, and a prefrontal cortical component that mediates the transition from hypothesis-testing to procedural-based strategies. Implications for independent vs. interactive category learning systems are discussed.

A computational model of how cholinergic interneurons protect striatal-dependent learning

An essential component of skill acquisition is learning the environmental conditions in which that skill is relevant. This article proposes and tests a neurobiologically detailed theory of how such learning is mediated. The theory assumes that a key component of this learning is provided by the cholinergic interneurons in the striatum known as tonically active neurons (TANs). The TANs are assumed to exert a tonic inhibitory influence over cortical inputs to the striatum that prevents the execution of any striatal-dependent actions. The TANs learn to pause in rewarding environments, and this pause releases the striatal output neurons from this inhibitory effect, thereby facilitating the learning and expression of striatal-dependent behaviors. When rewards are no longer available, the TANs cease to pause, which protects striatal learning from decay. A computational version of this theory accounts for a variety of single-cell recording data and some classic behavioral phenomena, including fast reacquisition after extinction.

Interactions between declarative and procedural-learning categorization systems

Two experiments tested whether declarative and procedural memory systems operate independently or inhibit each other during perceptual categorization. Both experiments used a hybrid category-learning task in which perfect accuracy could be achieved if a declarative strategy is used on some trials and a procedural strategy is used on others. In the two experiments, only 2 of 53 participants learned a strategy of this type. In Experiment 1, most participants appeared to use simple explicit rules, even though control participants reliably learned the procedural component of the hybrid task. In Experiment 2, participants pre-trained either with the declarative or procedural component and then transferred to the hybrid categories. Despite this extra training, no participants in either group learned to categorize the hybrid stimuli with a strategy of the optimal type. These results are inconsistent with the most prominent single- and multiple-system accounts of category learning. They also cannot be explained by knowledge partitioning, or by the hypothesis that the failure to learn was due to high switch costs. Instead, these results support the hypothesis that declarative and procedural memory systems interact during category learning.

Cortical and basal ganglia contributions to habit learning and automaticity

In the 20th century it was thought that novel behaviors are mediated primarily in cortex and that the development of automaticity is a process of transferring control to subcortical structures. However, evidence supports the view that subcortical structures, such as the striatum, make significant contributions to initial learning. More recently, there has been increasing evidence that neurons in the associative striatum are selectively activated during early learning, whereas those in the sensorimotor striatum are more active after automaticity has developed. At the same time, other recent reports indicate that automatic behaviors are striatum- and dopamine-independent, and might be mediated entirely within cortex. Resolving this apparent conflict should be a major goal of future research.

Computational Models Inform Clinical Science and Assessment: An Application to Category Learning in Striatal-Damaged Patients

In this article we develop a new model of classification that is intermediate between the static, single strategy decision-bound models and the dynamic trial by trial multiple systems model, dCOVIS. The new model, referred to as the sCOVIS model, assumes hypothesis-testing and procedural-based subsystems are active on each trial, but that the parameters that govern behavior of the system are fixed (static) within a block of trials. To determine the clinical utility of the model, it was applied to nonlinear information-integration classification data from patients with Parkinson's (PD) and Huntington's disease (HD). In one application, the models suggest that the locus of HD patients' nonlinear information-integration deficits is in their increased reliance on hypothesis-testing strategies, whereas the locus of PD patients' deficit is in the application of sub-optimal procedural-based strategies. In a second application, the weight associated with the hypothesis-testing subsystem is shown to account for a significant amount of the variance in longitudinal cognitive decline in non-demented PD patients above and beyond that predicted by accuracy alone. Together, the accuracy rate and this model index account for 72% of the total variance associated with cognitive decline in this sample of PD patients. Interestingly, the Wisconsin Card Sort task added no additional predictive power above and beyond that predicted by nonlinear accuracy alone.

Prefrontal contributions to rule-based and information-integration category learning

Previous research revealed that the basal ganglia play a critical role in category learning [Ell, S. W., Marchant, N. L., & Ivry, R. B. (2006). Focal putamen lesions impair learning in rule-based, but not information-integration categorization tasks. Neuropsychologia, 44(10), 1737-1751; Maddox, W. T. & Filoteo, J. V. (2007). Modeling visual attention and category learning in amnesiacs, striatal-damaged patients and normal aging. In Advances in Clinical-cognitive science: formal modeling and assessment of processes and symptoms (pp. 113-146). Washington DC: American Psychological Association] but less is known about the specific role of prefrontal cortical (PFC) regions in category learning. The current study examined rule-based (RB) and information-integration (II) category learning in 13 patients with damage primarily to ventral PFC regions. After 600 learning trials with feedback, patients were significantly less accurate than matched controls on both RB and II learning. Model-based analysis identified subgroups of patients whose impaired performance in each task was due to the use of sub-optimal learning strategies. Those patients impaired at either II or RB learning, performed significantly worse on the Wisconsin Card Sorting Test, a test of abstract rule formation and the ability to shift and maintain rules. Lesion analysis pointed to damage in a fairly circumscribed region of ventral medial prefrontal cortex as common to the impaired group of patients and those patients without ventral PFC damage mostly performed normally. These results provide further evidence that the ventromedial prefrontal cortex is critically important for the ability to monitor and integrate feedback in order to select and maintain optimal learning strategies.

Rule-based category learning in patients with Parkinson's disease

Measures of explicit rule-based category learning are commonly used in neuropsychological evaluation of individuals with Parkinson's disease (PD) and the pattern of PD performance on these measures tends to be highly varied. We review the neuropsychological literature to clarify the manner in which PD affects the component processes of rule-based category learning and work to identify and resolve discrepancies within this literature. In particular, we address the manner in which PD and its common treatments affect the processes of rule generation, maintenance, shifting and selection. We then integrate the neuropsychological research with relevant neuroimaging and computational modeling evidence to clarify the neurobiological impact of PD on each process. Current evidence indicates that neurochemical changes associated with PD primarily disrupt rule shifting, and may disturb feedback-mediated learning processes that guide rule selection. Although surgical and pharmacological therapies remediate this deficit, it appears that the same treatments may contribute to impaired rule generation, maintenance and selection processes. These data emphasize the importance of distinguishing between the impact of PD and its common treatments when considering the neuropsychological profile of the disease.

The effects of positive versus negative feedback on information-integration category learning

A number of studies have shown that in category learning, providing feedback about errors allows faster learning than providing feedback about correct responses. However, these previous studies used explicit, rule-based tasks in which the category structures could be separated by a simple rule that was easily verbalized. Here, the results of the first experiment known to compare the efficacy of positive versus negative feedback during information-integration category learning are reported. Information-integration tasks require participants to integrate perceptual information from incommensurable dimensions, and evidence suggests that optimal responding recruits procedural learning. The results show that although nearly all of the full-feedback control participants demonstrated information-integration learning, participants receiving either positive-only or negative-only feedback generally used explicit, rule-based strategies. It thus appears that, unlike rule-based learning, consistent information-integration learning requires full feedback. The theoretical implications of these findings for current models of information-integration learning are discussed.