Hippocampus, cortex, and basal ganglia: insights from computational models of complementary learning systems

Surprised at all the entropy: hippocampal, caudate and midbrain contributions to learning from prediction errors

Influential concepts in neuroscientific research cast the brain a predictive machine that revises its predictions when they are violated by sensory input. This relates to the predictive coding account of perception, but also to learning. Learning from prediction errors has been suggested for take place in the hippocampal memory system as well as in the basal ganglia. The present fMRI study used an action-observation paradigm to investigate the contributions of the hippocampus, caudate nucleus and midbrain dopaminergic system to different types of learning: learning in the absence of prediction errors, learning from prediction errors, and responding to the accumulation of prediction errors in unpredictable stimulus configurations. We conducted analyses of the regions of interests' BOLD response towards these different types of learning, implementing a bootstrapping procedure to correct for false positives. We found both, caudate nucleus and the hippocampus to be activated by perceptual prediction errors. The hippocampal responses seemed to relate to the associative mismatch between a stored representation and current sensory input. Moreover, its response was significantly influenced by the average information, or Shannon entropy of the stimulus material. In accordance with earlier results, the habenula was activated by perceptual prediction errors. Lastly, we found that the substantia nigra was activated by the novelty of sensory input. In sum, we established that the midbrain dopaminergic system, the hippocampus, and the caudate nucleus were to different degrees significantly involved in the three different types of learning: acquisition of new information, learning from prediction errors and responding to unpredictable stimulus developments. We relate learning from perceptual prediction errors to the concept of predictive coding and related information theoretic accounts.

Different dorsal striatum circuits mediate action discrimination and action generalization

Generalization is an important process that allows animals to extract rules from regularities of past experience and apply them to analogous situations. In particular, the generalization of previously learned actions to novel instruments allows animals to use past experience to act faster and more efficiently in an ever-changing environment. However, generalization of actions to a dissimilar instrument or situation may also be detrimental. In this study, we investigated the neural bases of action generalization and discrimination in mice trained on a lever-pressing task. Using specific schedules of reinforcement known to bias animals towards habitual or goal-directed behaviors, we confirmed that action generalization is more prominent in animals using habitual rather than goal-directed strategies. We discovered that selective excitotoxic lesions of the dorsolateral and dorsomedial striatum have opposite effects on the generalization of a previously learned action to a novel lever. Whereas lesions of the dorsolateral striatum impair action generalization, dorsomedial striatum lesions affect action discrimination and bias subjects towards action generalization. Importantly, these lesions do not affect the ability of animals to explore or match their lever-pressing rate to the reinforcement rate, or the ability to distinguish between different levers. The data presented here reveal that dorsolateral and dorsomedial striatal circuits have opposing roles in the generalization of previously learned actions to novel instruments, and suggest that these circuits compete for the expression of generalization in novel situations.

Neuropsychopharmacology and neurogenetic aspects of executive functioning: should reward gene polymorphisms constitute a diagnostic tool to identify individuals at risk for impaired judgment?

Executive functions are processes that act in harmony to control behaviors necessary for maintaining focus and achieving outcomes. Executive dysfunction in neuropsychiatric disorders is attributed to structural or functional pathology of brain networks involving prefrontal cortex (PFC) and its connections with other brain regions. The PFC receives innervations from different neurons associated with a number of neurotransmitters, especially dopamine (DA). Here we review findings on the contribution of PFC DA to higher-order cognitive and emotional behaviors. We suggest that examination of multifactorial interactions of an individual's genetic history, along with environmental risk factors, can assist in the characterization of executive functioning for that individual. Based upon the results of genetic studies, we also propose genetic mapping as a probable diagnostic tool serving as a therapeutic adjunct for augmenting executive functioning capabilities. We conclude that preservation of the neurological underpinnings of executive functions requires the integrity of complex neural systems including the influence of specific genes and associated polymorphisms to provide adequate neurotransmission.

Fear-relevant outcomes modulate the neural correlates of probabilistic classification learning

Although much work has implicated the contributions of frontostriatal and medial temporal lobe (MTL) systems during probabilistic classification learning, the impact of emotion on these learning circuits is unknown. We used a modified version of the weather prediction task in which two participant groups were scanned with identical neutral cue cards probabilistically linked to either emotional (snake/spider) or neutral (mushroom/flower) outcomes. Owing to the differences in visual information shown as outcomes, analyses were restricted to the cue phase of the trials. Learning rates did not differ between the two groups, although the Emotional group was more likely to use complex strategies and to respond more slowly during initial learning. The Emotional group had reduced frontostriatal and MTL activation relative to the Neutral group, especially for participants who scored higher on snake/spider phobia questionnaires. Accurate performance was more tied to medial prefrontal activity in the Emotional group early in training, and to MTL activity in the Neutral group later in training. Trial-by-trial fluctuations in functional connectivity between the caudate and MTL were also reduced in the Emotional group compared to the Neutral group. Across groups, reaction time indexed a switch in learning systems, with faster trials mediated by the caudate and slower trials mediated by the MTL and frontal lobe. The extent to which the caudate was activated early in training predicted later performance improvements. These results reveal insights into how emotional outcomes modulate procedural learning systems, and the dynamics of MTL-striatal engagement across training trials.

Learning, memory, and the role of neural network architecture

The performance of information processing systems, from artificial neural networks to natural neuronal ensembles, depends heavily on the underlying system architecture. In this study, we compare the performance of parallel and layered network architectures during sequential tasks that require both acquisition and retention of information, thereby identifying tradeoffs between learning and memory processes. During the task of supervised, sequential function approximation, networks produce and adapt representations of external information. Performance is evaluated by statistically analyzing the error in these representations while varying the initial network state, the structure of the external information, and the time given to learn the information. We link performance to complexity in network architecture by characterizing local error landscape curvature. We find that variations in error landscape structure give rise to tradeoffs in performance; these include the ability of the network to maximize accuracy versus minimize inaccuracy and produce specific versus generalizable representations of information. Parallel networks generate smooth error landscapes with deep, narrow minima, enabling them to find highly specific representations given sufficient time. While accurate, however, these representations are difficult to generalize. In contrast, layered networks generate rough error landscapes with a variety of local minima, allowing them to quickly find coarse representations. Although less accurate, these representations are easily adaptable. The presence of measurable performance tradeoffs in both layered and parallel networks has implications for understanding the behavior of a wide variety of natural and artificial learning systems.

Information processing systems, such as natural biological networks and artificial computational networks, exhibit a strong interdependence between structural organization and functional performance. However, the extent to which variations in structure impact performance is not well understood, particularly in systems whose functionality must be simultaneously flexible and stable. By statistically analyzing the behavior of network systems during flexible learning and stable memory processes, we quantify the impact of structural variations on the ability of the network to learn, modify, and retain representations of information. Across a range of architectures drawn from both natural and artificial systems, we show that these networks face tradeoffs between the ability to learn and retain information, and the observed behavior varies depending on the initial network state and the time given to process information. Furthermore, we analyze the difficulty with which different network architectures produce accurate versus generalizable representations of information, thereby identifying the structural mechanisms that give rise to functional tradeoffs between learning and memory.

Visual recognition memory across contexts

In two experiments, we investigated the development of representational flexibility in visual recognition memory during infancy using the Visual Paired Comparison (VPC) task. In Experiment 1, 6- and 9-month-old infants exhibited recognition when familiarization and test occurred in the same room, but showed no evidence of recognition when familiarization and test occurred in different rooms. In contrast, 12- and 18-month-old infants exhibited recognition irrespective of testing room. Thus, flexibility across a change of room was observed at a younger age than flexibility across a change of background that has previously been seen with the VPC procedure (Robinson & Pascalis, 2004). To determine if limitations in representational flexibility across a change of background could be overcome by experiences during encoding, in Experiment 2, 6-, 9-, 12- and 18-month-old infants were familiarized with a picture on multiple backgrounds. At all ages, infants showed recognition across a change in background at test. These findings indicate that dissociating an item from its context during encoding may be an important factor in understanding the representational flexibility of visual recognition memory in infancy. Developmental changes in representational flexibility are likely driven by changes in the functional maturity of the hippocampal formation, and experience.

Unconscious knowledge: A survey

The concept of unconscious knowledge is fundamental for an understanding of human thought processes and mentation in general; however, the psychological community at large is not familiar with it. This paper offers a survey of the main psychological research currently being carried out into cognitive processes, and examines pathways that can be integrated into a discipline of unconscious knowledge. It shows that the field has already a defined history and discusses some of the features that all kinds of unconscious knowledge seem to share at a deeper level. With the aim of promoting further research, we discuss the main challenges which the postulation of unconscious cognition faces within the psychological community.

Neural reuse: a fundamental organizational principle of the brain

An emerging class of theories concerning the functional structure of the brain takes the reuse of neural circuitry for various cognitive purposes to be a central organizational principle. According to these theories, it is quite common for neural circuits established for one purpose to be exapted (exploited, recycled, redeployed) during evolution or normal development, and be put to different uses, often without losing their original functions. Neural reuse theories thus differ from the usual understanding of the role of neural plasticity (which is, after all, a kind of reuse) in brain organization along the following lines: According to neural reuse, circuits can continue to acquire new uses after an initial or original function is established; the acquisition of new uses need not involve unusual circumstances such as injury or loss of established function; and the acquisition of a new use need not involve (much) local change to circuit structure (e.g., it might involve only the establishment of functional connections to new neural partners). Thus, neural reuse theories offer a distinct perspective on several topics of general interest, such as: the evolution and development of the brain, including (for instance) the evolutionary-developmental pathway supporting primate tool use and human language; the degree of modularity in brain organization; the degree of localization of cognitive function; and the cortical parcellation problem and the prospects (and proper methods to employ) for function to structure mapping. The idea also has some practical implications in the areas of rehabilitative medicine and machine interface design.

The Leabra architecture: Specialization without modularity

The posterior cortex, hippocampus, and prefrontal cortex in the Leabra architecture are specialized in terms of various neural parameters, and thus are predilections for learning and processing, but domain-general in terms of cognitive functions such as face recognition. Also, these areas are not encapsulated and violate Fodorian criteria for modularity. Anderson's terminology obscures these important points, but we applaud his overall message.

Conditional routing of information to the cortex: a model of the basal ganglia's role in cognitive coordination

The basal ganglia play a central role in cognition and are involved in such general functions as action selection and reinforcement learning. Here, we present a model exploring the hypothesis that the basal ganglia implement a conditional information-routing system. The system directs the transmission of cortical signals between pairs of regions by manipulating separately the selection of sources and destinations of information transfers. We suggest that such a mechanism provides an account for several cognitive functions of the basal ganglia. The model also incorporates a possible mechanism by which subsequent transfers of information control the release of dopamine. This signal is used to produce novel stimulus-response associations by internalizing transferred cortical representations in the striatum. We discuss how the model is related to production systems and cognitive architectures. A series of simulations is presented to illustrate how the model can perform simple stimulus-response tasks, develop automatic behaviors, and provide an account of impairments in Parkinson's and Huntington's diseases.

Instructional control of reinforcement learning: a behavioral and neurocomputational investigation

Humans learn how to behave directly through environmental experience and indirectly through rules and instructions. Behavior analytic research has shown that instructions can control behavior, even when such behavior leads to sub-optimal outcomes (Hayes, S. (Ed.). 1989. Rule-governed behavior: cognition, contingencies, and instructional control. Plenum Press.). Here we examine the control of behavior through instructions in a reinforcement learning task known to depend on striatal dopaminergic function. Participants selected between probabilistically reinforced stimuli, and were (incorrectly) told that a specific stimulus had the highest (or lowest) reinforcement probability. Despite experience to the contrary, instructions drove choice behavior. We present neural network simulations that capture the interactions between instruction-driven and reinforcement-driven behavior via two potential neural circuits: one in which the striatum is inaccurately trained by instruction representations coming from prefrontal cortex/hippocampus (PFC/HC), and another in which the striatum learns the environmentally based reinforcement contingencies, but is "overridden" at decision output. Both models capture the core behavioral phenomena but, because they differ fundamentally on what is learned, make distinct predictions for subsequent behavioral and neuroimaging experiments. Finally, we attempt to distinguish between the proposed computational mechanisms governing instructed behavior by fitting a series of abstract "Q-learning" and Bayesian models to subject data. The best-fitting model supports one of the neural models, suggesting the existence of a "confirmation bias" in which the PFC/HC system trains the reinforcement system by amplifying outcomes that are consistent with instructions while diminishing inconsistent outcomes.

Entorhinal cortex and cognition

Understanding the function of the entorhinal cortex (EC) has been an important subject over the years, not least because of its cortical intermediary to and from the hippocampus proper, and because of electrophysiological advances which have started to reveal the physiology in behaving animals. Clearly, a lot more needs to be done but is clear to date that EC is not merely a throughput station providing all hippocampal subfields with sensory information, but that processing within EC contributes significantly to attention, conditioning, event and spatial cognition possibly by compressing representations that overlap in time. These are transmitted to the hippocampus, where they are differentiated again and returned to EC. Preliminary evidence for such a role, but also their possible pitfalls are summarised.

Turning it upside down: areas of preserved cognitive function in schizophrenia

Patients with schizophrenia demonstrate marked impairments on most clinical neuropsychological tests. These findings suggest that patients suffer from a generalized form of cognitive impairment, with little evidence of spared performance documented in several large meta-analytic reviews of the clinical literature. In contrast, we review evidence for relative sparing of aspects of attention, procedural memory, and emotional processing observed in studies that have employed experimental approaches adapted from the cognitive and affective neuroscience literature. These islands of preserved performance suggest that the cognitive deficits in schizophrenia are not as general as they appear to be when assayed with clinical neuropsychological methods. The apparent contradiction in findings across methods may offer important clues about the nature of cognitive impairment in schizophrenia. The documentation of preserved cognitive function in schizophrenia may serve to sharpen hypotheses about the biological mechanisms that are implicated in the illness.

A quantum probability explanation for violations of 'rational' decision theory

Two experimental tasks in psychology, the two-stage gambling game and the Prisoner's Dilemma game, show that people violate the sure thing principle of decision theory. These paradoxical findings have resisted explanation by classical decision theory for over a decade. A quantum probability model, based on a Hilbert space representation and Schrödinger's equation, provides a simple and elegant explanation for this behaviour. The quantum model is compared with an equivalent Markov model and it is shown that the latter is unable to account for violations of the sure thing principle. Accordingly, it is argued that quantum probability provides a better framework for modelling human decision-making.

Associative memory models: from the cell-assembly theory to biophysically detailed cortex simulations

The second half of the past century saw the emergence of a theory of cortical associative memory function originating in Donald Hebb's hypotheses on activity-dependent synaptic plasticity and cell-assembly formation and dynamics. This conceptual framework has today developed into a theory of attractor memory that brings together many experimental observations from different sources and levels of investigation into computational models displaying information-processing capabilities such as efficient associative memory and holistic perception. Here, we outline a development that might eventually lead to a neurobiologically grounded theory of cortical associative memory.

Which way do I go? Neural activation in response to feedback and spatial processing in a virtual T-maze

In 2 human event-related brain potential (ERP) experiments, we examined the feedback error-related negativity (fERN), an ERP component associated with reward processing by the midbrain dopamine system, and the N170, an ERP component thought to be generated by the medial temporal lobe (MTL), to investigate the contributions of these neural systems toward learning to find rewards in a "virtual T-maze" environment. We found that feedback indicating the absence versus presence of a reward differentially modulated fERN amplitude, but only when the outcome was not predicted by an earlier stimulus. By contrast, when a cue predicted the reward outcome, then the predictive cue (and not the feedback) differentially modulated fERN amplitude. We further found that the spatial location of the feedback stimuli elicited a large N170 at electrode sites sensitive to right MTL activation and that the latency of this component was sensitive to the spatial location of the reward, occurring slightly earlier for rewards following a right versus left turn in the maze. Taken together, these results confirm a fundamental prediction of a dopamine theory of the fERN and suggest that the dopamine and MTL systems may interact in navigational learning tasks.

Age-related slowing of movement as basal ganglia dysfunction

Attributions of age-related deficits in motor function to structural changes are compromised once the elderly exhibit lower error rates. This is because performance decrements observed in older adults are attributed to inferred strategic preferences for accuracy over speed. To understand genuine age differences in performance, we argue in the following theoretical paper that research needs to resolve methodological shortcomings and account for them within theoretical models of aging. Accounts of aging need to directly manipulate or control strategic differences in performance while assessing structural deficits. When this is done, age-related changes in motor control resemble the intermittencies of control seen in basal ganglia disorders. Given homologous circuitry in the basal ganglia, such observations could generalize to age-related changes in cognitive and emotional processes.

The role of the dorsal striatum and dorsal hippocampus in probabilistic and deterministic odor discrimination tasks

Three experiments explored the contribution of the cortico-striatal system and the hippocampus system to the acquisition of solutions to simultaneous instrumental odor discriminations. Inactivation of the dorsal striatum after rats had reached criterion on a three problem probabilistic set of discriminations--A (80%) vs. B (20%), C (67%) vs. D (33%), E(67%) vs. F(33%)--impaired test performance and disrupted performance when the rats were tested with novel cue combinations (C vs. F and E vs. D), where control animals chose C and F. In contrast, inactivating the dorsal hippocampus enhanced performance on this task and on a deterministic discrimination A (100%) vs. B (0%). These results are consistent with the complementary learning systems view, which assumes that the cortico-striatal and hippocampal system capture information in parallel. How this information combines to influence task performance depends on the compatibility of the content captured by each system. These results suggest that the trial-specific information captured by the hippocampal system can be incompatible with the across-trial integration of trial outcomes captured by the cortico-striatal system.

Memory systems in the chick: regional and temporal control by noradrenaline

Learning starts with the information about a situation or experience delivered to different brain areas in terms of visual, olfactory, auditory and tactile inputs. Memory processing occurs in different brain locations in a well-defined temporal sequence of physiologically based stages and biochemical cascades. Using neuropharmacological techniques in one species and a robust bead discrimination task, we have been able to chart the passage of memory from acquisition to consolidation in the chick and to dissect out the multiple roles for noradrenaline in consolidating this memory. Fortunately only a small fraction of sensory input is remembered and it is clear that modulatory neurotransmitters play a key role in determining what is remembered. We have identified roles for noradrenaline in the mesopallium or 'avian cortex', the hippocampus, medial striatum or basal ganglia and teased out the different effects of noradrenaline in each of these areas based on the receptor subtypes activated by the transmitter and the stages on which they act. Noradrenergic input from the locus coeruleus controls memory processing at two critical times after training-acquisition (0-2.5 min after training) and consolidation (25-30 min after training). We have also elucidated some of the cellular mechanisms whereby noradrenaline achieves memory modulation and finds that it has actions on both neurones and astrocytes with particularly important effects on energy metabolism in astrocytes. The memory system of the chick is very similar to that of mammals in terms of brain regions recruited in memory processing and in the ways memory is modulated by noradrenaline.

Midazolam, hippocampal function, and transitive inference: Reply to Greene

The transitive inference (TI) task assesses the ability to generalize learned knowledge to new contexts, and is thought to depend on the hippocampus (Dusek & Eichenbaum, 1997). Animals or humans learn in separate trials to choose stimulus A over B, B over C, C over D and D over E, via reinforcement feedback. Transitive responding based on the hierarchical structure A > B > C > D > E is then tested with the novel BD pair. We and others have argued that successful BD performance by animals – and even humans in some implicit studies – can be explained by simple reinforcement learning processes which do not depend critically on the hippocampus, but rather on the striatal dopamine system. We recently showed that the benzodiazepene midazolam, which is thought to disrupt hippocampal function, profoundly impaired human memory recall performance but actually enhanced implicit TI performance (Frank, O'Reilly & Curran, 2006). We posited that midazolam biased participants to recruit striatum during learning due to dysfunctional hippocampal processing, and that this change actually supported generalization of reinforcement values. Greene (2007) questions the validity of our pharmacological assumptions and argues that our conclusions are unfounded. Here we stand by our original hypothesis, which remains the most parsimonious account of the data, and is grounded by multiple lines of evidence.

The contribution of synaptic plasticity in the basal ganglia to the processing of visual information

A mechanism for the involvement of the basal ganglia in the processing of visual information, based on dopamine-dependent modulation of the efficiency of synaptic transmission in interconnected parallel associative and limbic cortex-basal ganglia-thalamus-cortex circuits, is proposed. Each circuit consists of a visual or prefrontal area of the cortex connected with the thalamic nucleus and the corresponding areas in different nuclei of the basal ganglia. The circulation of activity in these circuits is supported by the recurrent arrival of information in the thalamus and cortex. Dopamine released in response to a visual stimulus modulates the efficiencies of "strong" and "weak" corticostriatal inputs in different directions, and the subsequent reorganization of activity in the circuit leads to disinhibition (inhibition) of the activity of those cortical neurons which are "strongly" ("weakly") excited by the visual stimulus simultaneously with dopaminergic cells. The pattern in each cortical area is the neuronal reflection of the properties of the visual stimulus processed by this area. Excitation of dopaminergic cells by the visual stimulus via the superior colliculi requires parallel activation of the disinhibitory input to the superior colliculi via the thalamus and the "direct" pathway" in the basal ganglia. The prefrontal cortex, excited by the visual stimulus via the mediodorsal nucleus of the thalamus, mediates the descending influence on the activity of dopaminergic cells, simultaneously controlling dopamine release in different areas of the striatum and thus facilitating the mutual selection of neural reflections of the individual properties of the visual stimulus and their binding into an integral image.

Is dystonic posturing during temporal lobe epileptic seizures the expression of an endogenous anticonvulsant system?

In temporal lobe epilepsy (TLE) seizures, tonic or clonic motor behaviors (TCB) are commonly associated with automatisms, versions, and vocalizations, and frequently occur during secondary generalization. Dystonias are a common finding and appear to be associated with automatisms and head deviation, but have never been directly linked to generalized tonic or clonic behaviors. The objective of the present study was to assess whether dystonias and TCB are coupled in the same seizure or are associated in an antagonistic and exclusive pattern. Ninety-one seizures in 55 patients with TLE due to mesial temporal sclerosis were analyzed. Only patients with postsurgical seizure outcome of Engel class I or II were included. Presence or absence of dystonia and secondary generalization was recorded. Occurrence of dystonia and occurrence of bilateral tonic or clonic behaviors were negatively correlated. Dystonia and TCB may be implicated in exclusive, non-coincidental, or even antagonistic effects or phenomena in TLE seizures. A neural network related to the expression of one behavioral response (e.g., basal ganglia activation and dystonia) might theoretically "displace" brain activation or disrupt the synchronism of another network implicated in pathological circuit reverberation and seizure expression. The involvement of basal ganglia in the blockade of convulsive seizures has long been observed in animal models. The question is: Do dystonia and underlying basal ganglia activation represent an attempt of the brain to block imminent secondary generalization?

Morphologic features of the amygdala and hippocampus in children and adults with Tourette syndrome

CONTEXT

Limbic portions of cortical-subcortical circuits are likely involved in the pathogenesis of Tourette syndrome (TS). They are anatomically, developmentally, neurochemically, and functionally related to the basal ganglia, and the basal ganglia are thought to produce the symptoms of tics, obsessive-compulsive disorder, and attention-deficit/hyperactivity disorder that commonly affect persons with TS.

OBJECTIVE

To study the morphologic features of the hippocampus and amygdala in children and adults with TS.

DESIGN

A cross-sectional, case-control study using anatomical magnetic resonance imaging.

SETTING

University research center.

PARTICIPANTS

A total of 282 individuals (154 patients with TS and 128 controls) aged 6 to 63 years.

MAIN OUTCOME MEASURES

Volumes and measures of surface morphologic features of the hippocampus and amygdala.

RESULTS

The overall volumes of the hippocampus and amygdala were significantly larger in the TS group. Surface analyses suggested that the increased volumes in the TS group derived primarily from the head and medial surface of the hippocampus (over the length of the dentate gyrus) and the dorsal and ventral surfaces of the amygdala (over its basolateral and central nuclei). Volumes of these subregions declined with age in the TS group but not in controls, so the subregions were significantly larger in children with TS but significantly smaller in adults with TS than in their control counterparts. In children and adults, volumes in these subregions correlated inversely with the severity of tic, obsessive-compulsive disorder, and attention-deficit/hyperactivity disorder symptoms, suggesting that enlargement of the subregions may have a compensatory and neuromodulatory effect on tic-related symptoms.

CONCLUSION

These findings are consistent with the known plasticity of the dentate gyrus and with findings from previous imaging studies suggesting the presence of failed compensatory plasticity in adults with TS who have not experienced the usual decline in symptoms during adolescence.

A hypothetical role of cortico-basal ganglia-thalamocortical loops in visual processing

The goal of the present work was to define the mechanisms underlying the contribution of sensory and limbic cortico-basal ganglia-thalamocortical loops to visual processing and its attentional modulation. We proposed that visual processing is promoted by dopamine-dependent long-term modifications of synaptic transmission in the basal ganglia that favour a selection of neocortical patterns representing a visual stimulus. This selection is the result of the opposite sign of modulation of strong and weak cortico-basal ganglia inputs and subsequent activity reorganization in each loop. Reorganization leads to disinhibition/inhibition of cortical neurons strongly/weakly excited by stimulus during dopamine release. Recruitment of the thalamo-basal ganglia-collicular pathway is proposed to be necessary for stimulus-evoked dopamine release that underlies bottom-up attentional effects. Visual excitation of the prefrontal cortex and hippocampus (via the thalamus), their cooperation in control of the basal ganglia and dopaminergic cell firing, and simultaneous modulation of activity in diverse cortico-basal ganglia-thalamocortical loops is proposed to underlie top-down attentional effects. It follows from our model that only those components of cortical responses can be modulated by attention, whose onset exceeds the latency of visual responses of dopaminergic cells (50-110 ms). This and other consequences of the model are in accordance with known experimental data.

Endless minds most beautiful

The marriage of evolution and development to produce the new discipline 'evo-devo' in biology is situated in the general history of evolutionary biology, and its significance for developmental cognitive science is discussed. The discovery and description of the highly conserved, robust and 'evolvable' mechanisms that organize the vertebrate body plan and fundamental physiology have direct implications for what we should investigate in the evolution of behavior and cognition.

Late acquisition of literacy in a native language

With event-related functional MRI (fMRI) and with behavioral measures we studied the brain processes underlying the acquisition of native language literacy. Adult dialect speakers were scanned while reading words belonging to three different conditions: dialect words, i.e., the native language in which subjects are illiterate (dialect), German words, i.e., the second language in which subjects are literate, and pseudo-words. Investigating literacy acquisition of a dialect may reveal how novel readers of a language build an orthographic lexicon, i.e., establish a link between already available semantic and phonological representations and new orthographic word forms. The main results of the study indicate that a set of regions, including the left anterior hippocampal formation and subcortical nuclei, is involved in the buildup of orthographic representations. The repeated exposure to written dialect words resulted in a convergence of the neural substrate to that of the language in which these subjects were already proficient readers. The latter result is compatible with a "fast" brain plasticity process that may be related to a shift of reading strategies.

Separate neural substrates for skill learning and performance in the ventral and dorsal striatum

It is widely accepted that the striatum of the basal ganglia is a primary substrate for the learning and performance of skills. We provide evidence that two regions of the rat striatum, ventral and dorsal, play distinct roles in instrumental conditioning (skill learning), with the ventral striatum being critical for learning and the dorsal striatum being important for performance but, notably, not for learning. This implies an actor (dorsal) versus director (ventral) division of labor, which is a new variant of the widely discussed actor-critic architecture. Our results also imply that the successful performance of a skill can ultimately result in its establishment as a habit outside the basal ganglia.

Nutrition and the development of cognitive functions: interpretation of behavioral studies in animals and human infants

A rapidly accumulating body of evidence on the neural basis of cognition suggests that cognition is not a unitary function but rather depends on the functions of multiple and dissociable neural systems. The nonlinear interactions in the differing trajectories of these systems during development result in changing patterns of cognitive functions over time; they may also lead to paradoxical outcomes, for which enhancement of one function through dietary intervention may be at the expense of another. This emerging understanding has important implications for the design and interpretation of studies on the cognitive effects of specific nutrients during development. It is important that researchers move away from global tests of development and strive rather to ensure that their choice of behavioral task is based on specific hypotheses of the systems expected to be altered by a dietary manipulation and on an understanding of which behavioral tests are valid, sensitive, and reliable indicators of this disruption. Furthermore, to understand whether accelerated or delayed development related to a particular cognitive function is beneficial or problematic, it is important to study the entire behavioral profile over different time points, rather than relying on one outcome measured at one time point. It is also necessary to control for sensory or motivational differences that will affect performance on the behavioral tasks. Implementation of these methodologic recommendations may contribute to a deeper understanding of the mechanisms involved in the nutrition-associated changes in cognitive functions and thereby aid in the development of an appropriate population-based dietary policy.

Spontaneously hypertensive, Wistar Kyoto and Sprague-Dawley rats differ in their use of place and response strategies in the water radial arm maze

This study further characterises the use of mnemonic systems in the spontaneously hypertensive rat (SHR), which is frequently used as a rodent model of attention deficit hyperactivity disorder. The objective of this study was to assess the preference of male SHR, Wistar-Kyoto (WKY) and Sprague-Dawley (SD) rats for a place or response strategy when trained on an ambiguous T-maze task, and also to examine whether all strains acquired information about both strategies during ambiguous training, regardless of their preferred strategy. In the first experiment, SHR and WKY showed a preference for a response strategy on the ambiguous T-maze task; in contrast, SD displayed a preference for a place strategy. In the second experiment, all strains demonstrated that they learned information about both the response and place strategies during ambiguous training. However, on a conditioned place preference test SHR did not display as strong a preference for the place arm as WKY and SD. This finding supports previous research in a conditioned cue preference test, in which SHR did not display a preference for the cue associated with the platform. These observations that the strains differ with respect to behavioural strategy in a learning task suggest that they differ in the underlying neural circuitry that serves goal-directed behaviour, and are consistent with SHR having deficits associated with the nucleus accumbens.

When memory fails, intuition reigns: midazolam enhances implicit inference in humans

People often make logically sound decisions using explicit reasoning strategies, but sometimes it pays to rely on more implicit "gut-level" intuition. The transitive inference paradigm has been widely used as a test of explicit logical reasoning in animals and humans, but it can also be solved in a more implicit manner. Some researchers have argued that the hippocampus supports relational memories required for making logical inferences. Here we show that the benzodiazepene midazolam, which inactivates the hippocampus, causes profound explicit memory deficits in healthy participants, but enhances their ability in making implicit transitive inferences. These results are consistent with neurocomputational models of the basal ganglia-dopamine system that learn to make decisions through positive and negative reinforcement. We suggest that disengaging the hippocampal explicit memory system can be advantageous for this more implicit form of learning.

Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal

The authors explore the division of labor between the basal ganglia-dopamine (BG-DA) system and the orbitofrontal cortex (OFC) in decision making. They show that a primitive neural network model of the BG-DA system slowly learns to make decisions on the basis of the relative probability of rewards but is not as sensitive to (a) recency or (b) the value of specific rewards. An augmented model that explores BG-OFC interactions is more successful at estimating the true expected value of decisions and is faster at switching behavior when reinforcement contingencies change. In the augmented model, OFC areas exert top-down control on the BG and premotor areas by representing reinforcement magnitudes in working memory. The model successfully captures patterns of behavior resulting from OFC damage in decision making, reversal learning, and devaluation paradigms and makes additional predictions for the underlying source of these deficits.

Differential hippocampal and prefrontal-striatal contributions to instance-based and rule-based learning

It is a topic of current interest whether learning in humans relies on the acquisition of abstract rule knowledge (rule-based learning) or whether it depends on superficial item-specific information (instance-based learning). Here, we identified brain regions that mediate either of the two learning mechanisms by combining fMRI with an experimental protocol shown to be able to dissociate both learning mechanisms. Subjects had to learn object-position conjunctions in several trials and blocks. In a learning condition, either objects (Experiment 1) or positions (Experiment 2) were held constant within-blocks. In contrast to a control condition in which object-position conjunctions were trial-unique, a performance increase within and across-blocks was observed in the learning condition of both experiments. We hypothesized that within-block learning mainly relies on instance-based processes, whereas across-block learning might depend on rule-based mechanisms. A within-block parametric fMRI analysis revealed a learning-related increase of lateral prefrontal and striatal activity and a learning-related decrease of hippocampal activity in both experiments. By contrast, across-block learning was associated with an activation modulation in distinct prefrontal-striatal brain regions, but not in the hippocampus. These data indicate that hippocampal and prefrontal-striatal brain regions differentially contribute to instance-based and rule-based learning.

Expression of protein kinase C-substrate mRNAs in the basal ganglia of adult and infant macaque monkeys

We performed in situ hybridization histochemistry on the monkey basal ganglia to investigate the mRNA localization of three protein kinase C substrates (GAP-43, MARCKS, and neurogranin), of which expression plays a role in structural changes in neurites and synapses. Weak hybridization signals for GAP-43 mRNA and intense signals for both MARCKS and neurogranin mRNAs were observed in the adult neostriatum. All three of the mRNAs were expressed in both substance P-positive direct pathway neurons and enkephalin-positive indirect pathway neurons. In the nucleus accumbens, the hybridization signals for the three mRNAs were weaker than those in the neostriatum. Double-label in situ hybridization histochemistry in the neostriatum revealed that GAP-43 and neurogranin mRNAs were expressed in a subset of MARCKS-positive neurons. While intense hybridization signals for MARCKS mRNA were observed in all of the other basal ganglia regions such as the globus pallidus, substantia innominata, subthalamic nucleus, and substantia nigra, intense signals for GAP-43 mRNA were restricted to the substantia innominata and substantia nigra pars compacta. No signal for neurogranin mRNA was observed in the basal ganglia regions outside the neostriatum and the nucleus accumbens. These results indicate that the protein kinase C substrates are abundant in some specific connections in cortico-basal ganglia circuits. Developmental analysis showed that the expression level in the putamen and nucleus accumbens, but not in the caudate nucleus, was higher in the infant than in the adult, suggesting that synaptic maturation in the caudate nucleus occurs earlier than that in the putamen and nucleus accumbens.

Banishing the homunculus: making working memory work

The prefrontal cortex has long been thought to subserve both working memory and "executive" function, but the mechanistic basis of their integrated function has remained poorly understood, often amounting to a homunculus. This paper reviews the progress in our laboratory and others pursuing a long-term research agenda to deconstruct this homunculus by elucidating the precise computational and neural mechanisms underlying these phenomena. We outline six key functional demands underlying working memory, and then describe the current state of our computational model of the prefrontal cortex and associated systems in the basal ganglia (BG). The model, called PBWM (prefrontal cortex, basal ganglia working memory model), relies on actively maintained representations in the prefrontal cortex, which are dynamically updated/gated by the basal ganglia. It is capable of developing human-like performance largely on its own by taking advantage of powerful reinforcement learning mechanisms, based on the midbrain dopaminergic system and its activation via the basal ganglia and amygdala. These learning mechanisms enable the model to learn to control both itself and other brain areas in a strategic, task-appropriate manner. The model can learn challenging working memory tasks, and has been corroborated by several important empirical studies.

A study on the role of the dorsal striatum and the nucleus accumbens in allocentric and egocentric spatial memory consolidation

There is now accumulating evidence that the striatal complex in its two major components, the dorsal striatum and the nucleus accumbens, contributes to spatial memory. However, the possibility that different striatal subregions might modulate specific aspects of spatial navigation has not been completely elucidated. Therefore, in this study, two different learning procedures were used to determine whether the two striatal components could be distinguished on the basis of their involvement in spatial learning using different frames of reference: allocentric and egocentric. The task used involved the detection of a spatial change in the configuration of four objects placed in an arena, after the mice had had the opportunity to experience the objects in a constant position for three previous sessions. In the first part of the study we investigated whether changes in the place where the animals were introduced into the arena during habituation and testing could induce a preferential use of an egocentric or an allocentric frame of reference. In the second part of the study we performed focal injections of the N-methyl-d-aspartate (NMDA) receptors' antagonist, AP-5, within the two subregions immediately after training. The results indicate that using the two behavioral procedures, the animals rely on an egocentric and an allocentric spatial frame of reference. Furthermore, they demonstrate that AP-5 (37.5, 75, and 150 ng/side) injections into the dorsal striatum selectively impaired consolidation of spatial information in the egocentric but not in the allocentric procedure. Intra-accumbens AP-5 administration, instead, impaired animals trained using both procedures.

Contributions of the hippocampus and the striatum to simple association and frequency-based learning

Using fMRI and a learning paradigm, this study examined the independent contributions of the hippocampus and striatum to simple association and frequency-based learning. We scanned 10 right-handed young adult subjects using a spiral in/out sequence on a GE 3.0 T scanner during performance of the learning paradigm. The paradigm consisted of 2 cues that predicted each of 3 targets with varying probabilities. Simultaneously, we varied the frequency with which each target was presented throughout the task, independent of cue associations. Subjects had shorter response latencies to frequently occurring and highly associated target stimuli and longer response latencies to infrequent target stimuli, indicating learning. Imaging results showed increased caudate activity to infrequent relative to frequent targets and increased hippocampal activity to infrequent relative to frequent cue-target associations. This work provides evidence of different neural mechanisms underlying learning based on simple frequencies versus associations within a single paradigm.

Brain correlates of sentence translation in Finnish???Norwegian bilinguals

We measured brain activation with functional magnetic resonance imaging (fMRI) while Finnish-Norwegian bilinguals silently translated sentences from Finnish into Norwegian and decided whether a later presented probe sentence was a correct translation of the original sentence. The control task included silent sentence reading and probe sentence decision within a single language, Finnish. The translation minus control task contrast activated the left inferior frontal gyrus (Brodmann's area 47) and the left basal ganglia. The left inferior frontal activation appears to be related to active semantic retrieval and the basal ganglia activation to a general action control function that works by suppressing competing responses.

Learning-induced activation of transcription factors among multiple memory systems

Experimental evidence for multiple memory systems grew initially from reports that integrity of the medial temporal lobes is necessary for some, but not all, types of memory formation. A primary inference from many studies of multiple memory systems is that they operate independently during encoding, storage, and retrieval of information. An accumulation of recent evidence, however, suggests that multiple memory systems may interact under some conditions. At the cellular level of analysis, it is accepted widely that protein synthesis is necessary for the formation of long-term memory and recent efforts have focused on the mechanisms by which learning-induced gene transcription and translation are regulated. The present review examines learning-induced activation of transcription factors among multiple memory systems. The results indicate that studies of transcriptional regulation, in conjunction with other experimental approaches, can provide complementary lines of evidence to further understanding of the extent to which multiple memory systems are independent or interactive.

Computational correlates of consciousness

Over the past few years numerous proposals have appeared that attempt to characterize consciousness in terms of what could be called its computational correlates: Principles of information processing with which to characterize the differences between conscious and unconscious processing. Proposed computational correlates include architectural specialization (such as the involvement of specific regions of the brain in conscious processing), properties of representations (such as their stability in time or their strength), and properties of specific processes (such as resonance, synchrony, interactivity, or information integration). In exactly the same way as one can engage in a search for the neural correlates of consciousness, one can thus search for the computational correlates of consciousness. The most direct way of doing is to contrast models of conscious versus unconscious information processing. In this paper, I review these developments and illustrate how computational modeling of specific cognitive processes can be useful in exploring and in formulating putative computational principles through which to capture the differences between conscious and unconscious cognition. What can be gained from such approaches to the problem of consciousness is an understanding of the function it plays in information processing and of the mechanisms that subtend it. Here, I suggest that the central function of consciousness is to make it possible for cognitive agents to exert flexible, adaptive control over behavior. From this perspective, consciousness is best characterized as involving (1) a graded continuum defined over quality of representation, such that availability to consciousness and to cognitive control correlates with properties of representation, and (2) the implication of systems of meta-representations.

A comparison and evaluation of the predictions of relational and conjunctive accounts of hippocampal function

Relational and conjunctive memory theory each postulate that the hippocampus participates in the formation of long-term memory representations comprised of associations between multiple elements. The goals of the current work were to clarify and contrast these theories by outlining the nature of the representations that are spared vs. impaired following hippocampal damage according to each theoretical perspective. Relational theory predicts that hippocampal lesions will impair performance on tasks that require the formation of new long-term representations in which distinct elements must be regarded in relation to all other elements. Representations that remain intact despite hippocampal damage include separate representations of distinct individual elements or multiple stimuli fused into a static "blend" such as several elements viewed from one vantage point. Additionally, the relational account predicts that rapid incidental online processing of the relations can be achieved through structures other than the hippocampus, but this information will not be stored. In contrast, conjunctive theory predicts that hippocampal damage will impair the rapid formation of unitary representations that contain features of elements and their relative relationships bound in an inflexible manner. Deficits in the rapid formation of these conjunctive representations result in impaired performance on tasks that require rapid incidental stimulus binding. However, intact formation of conjunctive representations can occur over multiple trials in the service of problem solving. Using these theoretical frameworks, recent findings from the human and nonhuman animal literature are reexamined in order to determine whether one theory better accounts for current findings. We discuss empirical studies that serve as "critical experiments" in addressing the relational vs. conjunctive debate, and find that the predictions of relational theory are supported by existing findings over those from the conjunctive account.