A neurocomputational theory of the dopaminergic modulation of working memory functions

Glutamate, obsessive-compulsive disorder, schizophrenia, and the stability of cortical attractor neuronal networks

A computational neuroscience approach to the symptoms of obsessive-compulsive disorder based on a stochastic neurodynamical framework is described. An increased depth in the basins of attraction of attractor neuronal network states in the brain makes each state too stable, so that it tends to remain locked in that state, and cannot easily be moved on to another state. It is suggested that the different symptoms that may be present in obsessive--compulsive disorder could be related to changes of this type in different brain regions. In integrate-and-fire network simulations, an increase in the NMDA and/or AMPA receptor conductances, which increases the depth of the attractor basins, increases the stability of attractor networks, and makes them less easily moved on to another state by a new stimulus. Increasing GABA-receptor activated currents can partly reverse this overstability. There is now some evidence for overactivity in glutamate transmitter systems in obsessive-compulsive disorder, and the hypothesis presented here shows how some of the symptoms of obsessive-compulsive disorder could be produced by the increase in the stability of attractor networks that is produced by increased glutamatergic activity. In schizophrenia, a reduction of the firing rates of cortical neurons caused for example by reduced NMDA receptor function, present in schizophrenia, can lead to instability of the high firing rate attractor states that normally implement short-term memory and attention, contributing to the cognitive and negative symptoms of schizophrenia. Reduced cortical inhibition caused by a reduction of GABA neurotransmission, present in schizophrenia, can lead to instability of the spontaneous firing states of cortical networks, leading to a noise-induced jump to a high firing rate attractor state even in the absence of external inputs, contributing to the positive symptoms of schizophrenia.

Parkinson's disease and healthy aging: independent and interacting effects on action selection

Functional reorganization of the motor system occurs in response to both aging and Parkinson's disease (PD). Since PD typically develops in older adults, disease progression and the effects of treatment may interact with normal aging. Using event-related functional magnetic resonance imaging, we studied patients "on" and "off" their normal dopaminergic medication, age-matched controls and younger adults on tasks of action and action selection. For manual movements, aging increased activity in bilateral motor, premotor and cingulate cortex. Activation in the premotor regions of "on" patients was higher relative to age-matched controls. However, in contrast to controls and "off" patients, the activations for patients when "on" decreased with age. Voluntary selection of actions was associated with activation in a bilateral network of fronto-parietal cortex. Within this network, advancing severity of PD was associated with decreased activity particularly in premotor and ventrolateral prefrontal cortex. Together, these results reveal very different patterns of age-related changes in health and PD. Younger patients are able to exert greater compensatory activity in premotor cortex than older patients, even after correction for disease severity. This effect is dopamine dependant, and may in part explain the clinical observation of reduced dopamine responsiveness in older patients with PD.

Computational models of performance monitoring and cognitive control

The medial prefrontal cortex (mPFC) has been the subject of intense interest as a locus of cognitive control. Several computational models have been proposed to account for a range of effects, including error detection, conflict monitoring, error likelihood prediction, and numerous other effects observed with single-unit neurophysiology, fMRI, and lesion studies. Here, we review the state of computational models of cognitive control and offer a new theoretical synthesis of the mPFC as signaling response–outcome predictions. This new synthesis has two interacting components. The first component learns to predict the various possible outcomes of a planned action, and the second component detects discrepancies between the actual and intended responses; the detected discrepancies in turn update the outcome predictions. This single construct is consistent with a wide array of performance monitoring effects in mPFC and suggests a unifying account of the cognitive role of medial PFC in performance monitoring.

A computational neuroscience approach to schizophrenia and its onset

Computational neuroscience integrate-and-fire attractor network models can be used to understand the factors that alter the stability of cortical networks in the face of noise caused for example by neuronal spiking times. A reduction of the firing rates of cortical neurons caused for example by reduced NMDA receptor function (present in schizophrenia) can lead to instability of the high firing rate attractor states that normally implement short-term memory and attention, contributing to the cognitive and negative symptoms of schizophrenia. Reduced cortical inhibition caused by a reduction of GABA neurotransmission (present in schizophrenia) can lead to instability of the spontaneous firing states of cortical networks, leading to a noise-induced jump to a high firing rate attractor state even in the absence of external inputs, contributing to the positive symptoms of schizophrenia. We consider how effects occurring at the time of late adolescence including synaptic pruning, decreases in grey matter volume, and changes in GABA-mediated inhibition and dopamine may contribute to the onset in some individuals of schizophrenia at this time.

Dopamine modulates persistent synaptic activity and enhances the signal-to-noise ratio in the prefrontal cortex

Background

The importance of dopamine (DA) for prefrontal cortical (PFC) cognitive functions is widely recognized, but its mechanisms of action remain controversial. DA is thought to increase signal gain in active networks according to an inverted U dose-response curve, and these effects may depend on both tonic and phasic release of DA from midbrain ventral tegmental area (VTA) neurons.

Methodology/Principal Findings

We used patch-clamp recordings in organotypic co-cultures of the PFC, hippocampus and VTA to study DA modulation of spontaneous network activity in the form of Up-states and signals in the form of synchronous EPSP trains. These cultures possessed a tonic DA level and stimulation of the VTA evoked DA transients within the PFC. The addition of high (≥1 µM) concentrations of exogenous DA to the cultures reduced Up-states and diminished excitatory synaptic inputs (EPSPs) evoked during the Down-state. Increasing endogenous DA via bath application of cocaine also reduced Up-states. Lower concentrations of exogenous DA (0.1 µM) had no effect on the up-state itself, but they selectively increased the efficiency of a train of EPSPs to evoke spikes during the Up-state. When the background DA was eliminated by depleting DA with reserpine and alpha-methyl-p-tyrosine, or by preparing corticolimbic co-cultures without the VTA slice, Up-states could be enhanced by low concentrations (0.1–1 µM) of DA that had no effect in the VTA containing cultures. Finally, in spite of the concentration-dependent effects on Up-states, exogenous DA at all but the lowest concentrations increased intracellular current-pulse evoked firing in all cultures underlining the complexity of DA's effects in an active network.

Conclusions/Significance

Taken together, these data show concentration-dependent effects of DA on global PFC network activity and they demonstrate a mechanism through which optimal levels of DA can modulate signal gain to support cognitive functioning.

Implications of synaptic biophysics for recurrent network dynamics and active memory

In cortical networks, synaptic excitation is mediated by AMPA- and NMDA-type receptors. NMDA differ from AMPA synaptic potentials with regard to peak current, time course, and a strong voltage-dependent nonlinearity. Here we illustrate based on empirical and computational findings that these specific biophysical properties may have profound implications for the dynamics of cortical networks, and via dynamics on cognitive functions like active memory. The discussion will be led along a minimal set of neural equations introduced to capture the essential dynamics of the various phenomena described. NMDA currents could establish cortical bistability and may provide the relatively constant synaptic drive needed to robustly maintain enhanced levels of activity during working memory epochs, freeing fast AMPA currents for other computational purposes. Perhaps more importantly, variations in NMDA synaptic input-due to their biophysical particularities-control the dynamical regime within which single neurons and networks reside. By provoking bursting, chaotic irregularity, and coherent oscillations their major effect may be on the temporal pattern of spiking activity, rather than on average firing rate. During active memory, neurons may thus be pushed into a spiking regime that harbors complex temporal structure, potentially optimal for the encoding and processing of temporal sequence information. These observations provide a qualitatively different view on the role of synaptic excitation in neocortical dynamics than entailed by many more abstract models. In this sense, this article is a plead for taking the specific biophysics of real neurons and synapses seriously when trying to account for the neurobiology of cognition.

Quantification of biogenic amines by microchip electrophoresis with chemiluminescence detection

A highly sensitive microchip electrophoresis (MCE) method with chemiluminescence (CL) detection was developed for the determination of biogenic amines including agmatine (Agm), epinephrine (E), dopamine (DA), tyramine, and histamine in human urine samples. To achieve a high assay sensitivity, the targeted analytes were pre-column labeled by a CL tagging reagent, N-(4-aminobutyl)-N-ethylisoluminol (ABEI). ABEI-tagged biogenic amines after MCE separation reacted with hydrogen peroxide in the presence of horseradish peroxidase (HRP), producing CL emission. Since no CL reagent was added to the running buffer, the background of the CL detection was extremely low, resulting in a significant improvement in detection sensitivity. Detection limits (S/N=3) were in the range from 5.9x10(-8) to 7.7x10(-8) M for the biogenic amines tested, which were at least 10 times lower than those of the MCE-CL methods previously reported. Separation of a urine sample on a 7 cm glass/poly(dimethylsiloxane) (PDMS) microchip channel was completed within 3 min. Analysis of human urine samples found that the levels of Agm, E and DA were in the ranges of 2.61x10(-7) to 4.30x10(-7) M, 0.81x10(-7) to 1.12x10(-7) M, and 8.76x10(-7) to 11.21x10(-7) M (n=4), respectively.

The role of attention in sexual arousal: implications for treatment of sexual dysfunction

The literature on the role of attention in sexual arousal is reviewed, especially that which has implications for noninvasive treatment of sexual dysfunction. Findings suggest that voluntary control of sexual arousal can be achieved through attentional focus on nonsexual cognitions or sexual fantasy. Cognitive biases may direct attention and thus facilitate or impede sexual arousal. Sexual arousal may be influenced by directed attentional focus, and preliminary evidence suggests that mindfulness techniques may result in longer-term changes in attentional focus; these changes, in turn, may improve sexual response. Information-processing models of sexual arousal developed in light of such findings are discussed. This research establishes the central role of attentional processes in facilitating physiological and, especially, subjective sexual arousal. Implementing approaches that capitalize on attentional processes could advance noninvasive treatment of sexual dysfunction. Future avenues of research might investigate how play, mammalian play circuits, and flow states are relevant to sexual response and satisfaction.

Sustained activities and retrieval in a computational model of the perirhinal cortex

The perirhinal cortex is involved not only in object recognition and novelty detection but also in multimodal integration, reward association, and visual working memory. We propose a computational model that focuses on the role of the perirhinal cortex in working memory, particularly with respect to sustained activities and memory retrieval. This model describes how different partial informations are integrated into assemblies of neurons that represent the identity of an object. Through dopaminergic modulation, the resulting clusters can retrieve the global information with recurrent interactions between neurons. Dopamine leads to sustained activities after stimulus disappearance that form the basis of the involvement of the perirhinal cortex in visual working memory processes. The information carried by a cluster can also be retrieved by a partial thalamic or prefrontal stimulation. Thus, we suggest that areas involved in planning and memory coordination encode a pointer to access the detailed information encoded in the associative cortex such as the perirhinal cortex.

Tolcapone enhances food-evoked dopamine efflux and executive memory processes mediated by the rat prefrontal cortex

BACKGROUND AND RATIONALE

Genetic variations in catechol-O-methyl transferase (COMT) or administration of COMT inhibitors have a robust impact on cognition and executive function in humans. The COMT enzyme breaks down extracellular dopamine (DA) and has a particularly important role in the prefrontal cortex (PFC) where DA transporters are sparse. As such, the beneficial cognitive effects of the COMT inhibitor tolcapone are postulated to be the result of increased bioavailability of DA in the PFC. Furthermore, it has been shown previously that COMT inhibitors increase pharmacologically evoked DA but do not affect basal levels in the PFC.

OBJECTIVES

The current study characterized the ability of tolcapone to increase DA release in response to behaviorally salient stimuli and improve performance of the delayed spatial win-shift (DSWSh) task.

RESULTS AND CONCLUSIONS

Tolcapone enhanced PFC DA efflux associated with the anticipation and consumption of food when compared to saline controls. Chronic and acute treatment with tolcapone also reduced the number of errors committed during acquisition of the DSWSh. However, no dissociable effects were observed in experiments designed to selectively assay encoding or recall in well-trained animals, as both experiments showed improvement with tolcapone treatment. Taken together, these data suggest a generalized positive influence on cognition. Furthermore, these data support the conclusion of Apud and Weinberger (CNS Drugs 21:535-557, 2007) that agents which selectively potentiate PFC DA release may confer cognitive enhancement without the unwanted side effects produced by drugs that increase basal DA levels in cortical and subcortical brain regions.

Temporal dynamics of basal ganglia under-recruitment in Parkinson's disease: transient caudate abnormalities during updating of working memory

Using hybrid-blocked/event-related fMRI and the 2-back task we aimed to decompose tonic and phasic temporal dynamics of basal ganglia response abnormalities in working memory associated with early untreated Parkinson's disease. In view of the tonic/phasic dopamine hypothesis, which posits a functional division between phasic D(2)-dependent striatal updating processes and tonic D(1)-dependent prefrontal context-maintenance processes, we predicted that newly diagnosed, drug-naïve Parkinson's disease patients, with selective striatal dopamine deprivation, would demonstrate transient rather than sustained activation changes in the basal ganglia during 2-back performance. Task-related activation patterns within discrete basal ganglia structures were directly compared between patients and healthy elderly controls. The obtained results yielded uniquely transient underactivation foci in caudate nuclei, putamen and globus pallidus in Parkinson's disease patients, which indicates suboptimal phasic implementation of striatal D(2)-dependent gating mechanisms during updating. Sustained underactivation was only seen in the anterior putamen, which may reflect initial signs of tonic control impairment. No significant changes were exhibited in prefrontal cortex. The present findings resonate well with the tonic/phasic dopamine account and suggest that basal ganglia under-recruitment associated with executive dysfunction in early Parkinson's disease might predominantly stem from deficiencies in phasic executive components subserved by striatum.

The dual-state theory of prefrontal cortex dopamine function with relevance to catechol-o-methyltransferase genotypes and schizophrenia

There is now general consensus that at least some of the cognitive deficits in schizophrenia are related to dysfunctions in the prefrontal cortex (PFC) dopamine (DA) system. At the cellular and synaptic level, the effects of DA in PFC via D1- and D2-class receptors are highly complex, often apparently opposing, and hence difficult to understand with regard to their functional implications. Biophysically realistic computational models have provided valuable insights into how the effects of DA on PFC neurons and synaptic currents as measured in vitro link up to the neural network and cognitive levels. They suggest the existence of two discrete dynamical regimes, a D1-dominated state characterized by a high energy barrier among different network patterns that favors robust online maintenance of information and a D2-dominated state characterized by a low energy barrier that is beneficial for flexible and fast switching among representational states. These predictions are consistent with a variety of electrophysiological, neuroimaging, and behavioral results in humans and nonhuman species. Moreover, these biophysically based models predict that imbalanced D1:D2 receptor activation causing extremely low or extremely high energy barriers among activity states could lead to the emergence of cognitive, positive, and negative symptoms observed in schizophrenia. Thus, combined experimental and computational approaches hold the promise of allowing a detailed mechanistic understanding of how DA alters information processing in normal and pathological conditions, thereby potentially providing new routes for the development of pharmacological treatments for schizophrenia.

A unified framework for addiction: vulnerabilities in the decision process

The understanding of decision-making systems has come together in recent years to form a unified theory of decision-making in the mammalian brain as arising from multiple, interacting systems (a planning system, a habit system, and a situation-recognition system). This unified decision-making system has multiple potential access points through which it can be driven to make maladaptive choices, particularly choices that entail seeking of certain drugs or behaviors. We identify 10 key vulnerabilities in the system: (1) moving away from homeostasis, (2) changing allostatic set points, (3) euphorigenic "reward-like" signals, (4) overvaluation in the planning system, (5) incorrect search of situation-action-outcome relationships, (6) misclassification of situations, (7) overvaluation in the habit system, (8) a mismatch in the balance of the two decision systems, (9) over-fast discounting processes, and (10) changed learning rates. These vulnerabilities provide a taxonomy of potential problems with decision-making systems. Although each vulnerability can drive an agent to return to the addictive choice, each vulnerability also implies a characteristic symptomology. Different drugs, different behaviors, and different individuals are likely to access different vulnerabilities. This has implications for an individual's susceptibility to addiction and the transition to addiction, for the potential for relapse, and for the potential for treatment.

An attractor hypothesis of obsessive-compulsive disorder

We propose a top-down approach to the symptoms of obsessive-compulsive disorder (OCD) based on a statistical dynamical framework. An increased depth in the basins of attraction of attractor network states in the brain makes each state too stable, so that it tends to remain locked in that state and can not easily be moved on to another state. We suggest that the different symptoms that may be present in OCD could be related to changes of this type in different brain regions. For example, the difficulty in attentional and cognitive set switching could be related to networks operating in this way in the prefrontal cortex. Repetitive actions and a difficulty in moving to new actions could be related to overstability in networks in the higher order motor, including cingulate areas. In integrate-and-fire network simulations, an increase in the N-methyl-d-aspartate (NMDA) and/or alpha-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) receptor conductances, which increases the depth of the attractor basins, increases the stability of attractor networks, and makes them less easily moved on to another state by a new stimulus. Increasing gamma-aminobutyric acid (GABA)-receptor activated currents can partly reverse this overstability. There is now some evidence for overactivity in glutamate transmitter systems in OCD, and the hypothesis presented here shows how some of the symptoms of OCD could be produced by the increase in the stability of attractor networks that is produced by increased glutamatergic activity.

Mental fatigue: costs and benefits

A framework for mental fatigue is proposed, that involves an integrated evaluation of both expected rewards and energetical costs associated with continued performance. Adequate evaluation of predicted rewards and potential risks of actions is essential for successful adaptive behaviour. However, while both rewards and punishments can motivate to engage in activities, both types of motivated behaviour are associated with energetical costs. We will review findings that suggest that the nucleus accumbens, orbitofrontal cortex, amygdala, insula and anterior cingulate cortex are involved evaluating both the potential rewards associated with performing a task, as well as assessing the energetical demands involved in task performance. Behaviour will only proceed if this evaluation turns out favourably towards spending (additional) energy. We propose that this evaluation of predicted rewards and energetical costs is central to the phenomenon of mental fatigue: people will no longer be motivated to engage in task performance when energetical costs are perceived to outweigh predicted rewards.

Short-term memory trace in rapidly adapting synapses of inferior temporal cortex

Visual short-term memory tasks depend upon both the inferior temporal cortex (ITC) and the prefrontal cortex (PFC). Activity in some neurons persists after the first (sample) stimulus is shown. This delay-period activity has been proposed as an important mechanism for working memory. In ITC neurons, intervening (nonmatching) stimuli wipe out the delay-period activity; hence, the role of ITC in memory must depend upon a different mechanism. Here, we look for a possible mechanism by contrasting memory effects in two architectonically different parts of ITC: area TE and the perirhinal cortex. We found that a large proportion (80%) of stimulus-selective neurons in area TE of macaque ITCs exhibit a memory effect during the stimulus interval. During a sequential delayed matching-to-sample task (DMS), the noise in the neuronal response to the test image was correlated with the noise in the neuronal response to the sample image. Neurons in perirhinal cortex did not show this correlation. These results led us to hypothesize that area TE contributes to short-term memory by acting as a matched filter. When the sample image appears, each TE neuron captures a static copy of its inputs by rapidly adjusting its synaptic weights to match the strength of their individual inputs. Input signals from subsequent images are multiplied by those synaptic weights, thereby computing a measure of the correlation between the past and present inputs. The total activity in area TE is sufficient to quantify the similarity between the two images. This matched filter theory provides an explanation of what is remembered, where the trace is stored, and how comparison is done across time, all without requiring delay period activity. Simulations of a matched filter model match the experimental results, suggesting that area TE neurons store a synaptic memory trace during short-term visual memory.

To know whether one is looking at an object seen a few seconds ago or not depends on visual short-term memory. To study short-term memory, we recorded single neuronal activity from two brain areas of monkeys, the TE and the perirhinal cortex of the temporal lobe, known to be important in visual pattern recognition and memory. The monkeys performed a short-term visual memory task, a sequential match-to-sample. The monkeys had to signal when a sample stimulus reappeared in a short sequence of stimuli. In area TE only, small fluctuations occurring for a sample-elicited response were correlated with the responses when a match stimulus reappeared, as if a snapshot of the sample-induced response was stored and recalled. In our modeling, we propose that each TE neuron stores and compares the signals during short-term memory by storing the response to the sample in local and rapidly adapting synapses. Subsequent stimulus-elicited responses are then automatically multiplied by the locally stored signal. Here, we show that the match can be detected when the sum of the outputs of the population of TE neurons crosses a threshold. Correlated fluctuations will be a signature this type of local memory storage wherever it occurs in the brain.

Insight without cortex: lessons from the avian brain

Insight is a cognitive feature that is usually regarded as being generated by the neocortex and being present only in humans and possibly some closely related primates. In this essay we show that especially corvids display behavioral skills within the domains of object permanence, episodic memory, theory of mind, and tool use/causal reasoning that are insightful. These similarities between humans and corvids at the behavioral level are probably the result of a convergent evolution. Similarly, the telencephalic structures involved in higher cognitive functions in both species show a high degree of similarity, although the forebrain of birds has no cortex-like lamination. The neural substrate for insight-related cognitive functions in mammals and birds is thus not necessarily based on a laminated cortical structure but can be generated by differently organized forebrains. Hence, neither is insight restricted to mammals, as predicted from a "scala naturae", nor is the laminated cortex a prerequisite for the highest cognitive functions.

Sustained activation and executive control in the avian prefrontal cortex

We review our studies examining neural correlates of directed forgetting and executive control in the avian prefrontal cortex. One of the fundamental forms of executive control is the ability to selectively filter information, retaining that which is critical for the current purposes and discarding that which is not. In our first experiment, we trained birds on a directed-forgetting version of a delayed matching-to-sample task. Following a sample stimulus, a bird heard either a remember tone indicating that a memory test would follow, or a forget tone indicating that no memory test would be given. We found that neural activity in the avian prefrontal cortex increased when the bird was told to remember, and decreased when the bird was told to forget. Behavioral probe tests confirmed that the animals were forgetting on forget trials. Although the sustained activation observed on remember trials and the absence of such activation on forget trials could be a code of remembering and forgetting the sample stimulus, it could also be a code of the possibility of obtaining a reward. To address this issue we conducted a second study in which we used three cues: remember, forget, and forget-reward. The forget-reward cue instructed the subject to forget the sample yet at the same time provided a free reward. Neural activity on forget-reward trials matched that on remember trials tentatively indicating that the sustained activation on remember trials might be a reward code rather than a sample stimulus code. Behavioral probe tests, however, failed to indicate that the animals were forgetting on forget-reward trials, and hence it still is possible that the sustained activation could be a code for memory of the sample stimulus.

A dynamical systems hypothesis of schizophrenia

We propose a top-down approach to the symptoms of schizophrenia based on a statistical dynamical framework. We show that a reduced depth in the basins of attraction of cortical attractor states destabilizes the activity at the network level due to the constant statistical fluctuations caused by the stochastic spiking of neurons. In integrate-and-fire network simulations, a decrease in the NMDA receptor conductances, which reduces the depth of the attractor basins, decreases the stability of short-term memory states and increases distractibility. The cognitive symptoms of schizophrenia such as distractibility, working memory deficits, or poor attention could be caused by this instability of attractor states in prefrontal cortical networks. Lower firing rates are also produced, and in the orbitofrontal and anterior cingulate cortex could account for the negative symptoms, including a reduction of emotions. Decreasing the GABA as well as the NMDA conductances produces not only switches between the attractor states, but also jumps from spontaneous activity into one of the attractors. We relate this to the positive symptoms of schizophrenia, including delusions, paranoia, and hallucinations, which may arise because the basins of attraction are shallow and there is instability in temporal lobe semantic memory networks, leading thoughts to move too freely round the attractor energy landscape.

One of the hallmarks of schizophrenia is the complexity and heterogeneity of the illness. We propose that part of the reason for the inconsistent symptoms may be a reduced signal-to-noise ratio and increased statistical fluctuations in different cortical brain networks. The novelty of the approach described here is that instead of basing our hypothesis purely on biological mechanisms, we develop a top-down approach based on the different types of symptoms and relate them to instabilities in attractor neural networks. Schizophrenia is characterized by cognitive, negative, and positive symptoms. We propose which characteristic effects in a dynamical system could cause these symptoms, and investigate our hypothesis in a computational model. We implement an integrate-and-fire network model and focus on the alterations of synaptic channels activated via NMDA and GABA receptors. We found that a decrease in the NMDA receptor conductance could contribute to both the cognitive and negative symptoms by reducing the neuronal firing rates and the stability of the attractor states. A reduction of both NMDA and GABA conductance causes an instability of the attractor states related to the positive symptoms. Overall, we provide a framework to discuss schizophrenia in a dynamical system framework.

Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system

The prefrontal cortex (PFC) has long been thought to serve as an 'executive' that controls the selection of actions and cognitive functions more generally. However, the mechanistic basis of this executive function has not been clearly specified often amounting to a homunculus. This paper reviews recent attempts to deconstruct this homunculus by elucidating the precise computational and neural mechanisms underlying the executive functions of the PFC. The overall approach builds upon existing mechanistic models of the basal ganglia (BG) and frontal systems known to play a critical role in motor control and action selection, where the BG provide a 'Go' versus 'NoGo' modulation of frontal action representations. In our model, the BG modulate working memory representations in prefrontal areas to support more abstract executive functions. We have developed a computational model of this system that is capable of developing human-like performance on working memory and executive control tasks through trial-and-error learning. This learning is based on reinforcement learning mechanisms associated with the midbrain dopaminergic system and its activation via the BG and amygdala. Finally, we briefly describe various empirical tests of this framework.

Differential increase of extracellular dopamine and serotonin in the ‘prefrontal cortex’ and striatum of pigeons during working memory

Monoamines, such as dopamine (DA) and serotonin (5-HT), play a central role in the modulation of cognitive processes at the forebrain level. Experimental and clinical studies based on dopaminergic pathology, depletion or medication indicate that DA, in particular, is involved in working memory (WM). However, it is unclear whether DA is indeed related to WM, whether its function is specific to the prefrontal cortex (PFC), and whether other modulators, such as 5-HT, might have similar functions. Therefore, the aims of this study were threefold. First, we analysed whether increased prefrontal DA release is related to WM in general or only to its short-term memory component. Second, we examined whether the DA release during cognitive tasks is specific to prefrontal areas or also occurs in the striatum. Third, we analysed whether prefrontal or striatal 5-HT release accompanies working and short-term memory. We approached these questions by using in vivo microdialysis to analyse the extracellular DA and 5-HT release in the pigeons' 'PFC' and striatum during matching-to-sample tasks with or without a delay. Here, we show that DA has no unitary function but is differentially released during working as well as short-term memory in the pigeons' 'prefrontal' cortex. Striatal DA shows an increased efflux only during WM that involves a delay component. WM is also accompanied by a 'prefrontal' but not a striatal release of 5-HT, whose efflux pattern is thus partly different to that of DA. Our findings thus show a triple dissociation between transmitters, structures and tasks within the avian 'prefronto'-striatal system.

Dissociating effects of cocaine and d-amphetamine on dopamine and serotonin in the perirhinal, entorhinal, and prefrontal cortex of freely moving rats

RATIONALE

Neuroimaging studies with humans showed widespread activation of the cortex in response to psychostimulant drugs. However, the neurochemical nature of these brain activities is not characterized.

OBJECTIVE

The aim of the present study was to investigate the effects of cocaine and d-amphetamine on dopamine (DA) and serotonin (5-HT) in cortical areas of the hippocampal network in comparison to the prefrontal cortex (PFC).

MATERIALS AND METHODS

We conducted in vivo microdialysis experiments in behaving rats measuring DA and 5-HT in the perirhinal cortex (PRC), entorhinal cortex (EC), and PFC, after application of cocaine (0, 5, 10, 20 mg/kg; i.p.) or d-amphetamine (0, 0.5, 1.0, 2.5 mg/kg; i.p.).

RESULTS

Cocaine and d-amphetamine dose-dependently increased DA and 5-HT levels in the PRC, EC, and PFC. A predominant DA response to d-amphetamine was only found in the PFC, but not in the PRC and EC. Cocaine increased DA and 5-HT to an equal extent in the PFC and PRC but induced a predominant 5-HT response in the EC. When comparing the neurochemical responses between the drugs at an equal level of behavioral activation, cocaine was more potent than d-amphetamine in increasing 5-HT in the PFC, while no differences were found in the PRC or EC or in the DA responses in all three cortical areas.

CONCLUSIONS

We conclude that cocaine and d-amphetamine increase DA and 5-HT levels in PRC and EC largely to the same extent as in the PFC.

Some conceptual problems with the classical theory of behaviour

Behaviour is usually assumed to depend on the reach of a critical intensity--termed reactivity threshold--by its motivation. This view represents a simple, predictive theoretical framework in ethology and animal psychology. However, it is here argued that only the influence of an isolated motivation on behaviour can be explained that way; that such a view fails to account for behaviour when several motivations are jointly activated. Upon analysis, the classical theory of behaviour (CTB) proves to be under-specified and thus leads to three conceptual problems that make it logically inconsistent for the study of multiple motivations. A revision of the CTB, called anticipatory dynamics model (ADM), is then developed in order to bring a theoretical solution to these conceptual problems. The ADM hypothesizes that an organism's motivational interactions are due to the limitation of the organism's attentional resources.

Boosting Focally-Induced Brain Plasticity by Dopamine

Dopamine (DA) simultaneously produces both excitation and inhibition in the human cortex. In order to shed light on the functional significance of these seemingly opposing effects, we administered the DA precursor levodopa (L-dopa) to healthy subjects in conjunction with 2 neuroplasticity-inducing motor cortex stimulation protocols. Transcranial direct current stimulation (tDCS) induces cortical excitability enhancement by anodal and depression by cathodal brain polarization, which is not restricted to specific subgroups of synapses. In contrast, paired associative stimulation (PAS) induces focal excitability enhancements of somatosensory and motor cortical neuronal synaptic connections. Here, we show that administering L-dopa turns the unspecific excitability enhancement caused by anodal tDCS into inhibition and prolongs the cathodal tDCS-induced excitability diminution. Conversely, it stabilizes the PAS-induced synapse-specific excitability increase. Most importantly, it prolongs all of these aftereffects by a factor of about 20. Hereby, DA focuses synapse-specific excitability-enhancing neuroplasticity in human cortical networks.

Implicit learning, executive function and hedonic activity in chronic polydrug abusers, currently abstinent polydrug abusers and controls

AIMS

The study seeks to evaluate impairments of implicit learning and executive function in chronic polydrug abusers. It was hypothesized that implicit learning and executive function correlate with anhedonia.

DESIGN

A cross-sectional group comparison.

SETTINGS

Department of Psychiatry, University of Tübingen, Germany. Participants A total of 25 male polydrug abusers with opiate dependence, n = 26 polydrug abusers abstinent for more than 3 months and n = 26 non-drug-using healthy males.

SETTING

Abstinent polydrug abusers were recruited from a community treatment centre, current polydrug abusers from local drug counselling services and controls through advertisements.

MEASUREMENTS

A psychological battery assessing implicit learning (serial reaction-time task), various executive functions (latent inhibition, delayed matching-to-sample, Trail Making Test, acquisition and modification of conditioned responses, figural reasoning) and verbal logic memory was administered. Hedonic thoughts and activities as well as depressive symptoms were assessed through questionnaires.

FINDINGS

In chronic polydrug abusers, there were moderate impairments of implicit learning, of acquisition, reversal and extinction of conditioned responses, of latent inhibition as well as anhedonia, while working memory was spared. In the abstinent group, cognitive performance was normal except for latent inhibition and more anhedonia and depression than in controls.

CONCLUSIONS

The findings suggest that current polydrug abusers suffer from impairment of many cognitive functions and from anhedonia. During abstinence, there is near normal cognitive function but still anhedonia. Anhedonia was correlated with implicit learning but not with executive function.

Dopamine activity in the occipital and temporal cortices of rats: dissociating effects of sensory but not pharmacological stimulation

Little is known about the functional role of DA in other cortical areas than in the prefrontal cortex. Here we demonstrate that visual stimulation increases DA activity as measured by in vivo microdialysis in the occipital, but not in the temporal cortex of freely moving rats. Auditory stimulation neither acutely affected DA in the occipital nor in the temporal cortex. A pharmacological challenge with cocaine (0, 5, 10, 20 mg/kg; i.p.) increased DA levels in both areas dose-dependently. These data suggest that DA might play a role in visual processing selectively in the occipital cortex. Furthermore, the results indicate that DA in both regions may contribute to cocaine's behavioral effects.

Unity and diversity of tonic and phasic executive control components in episodic and working memory

The present study aimed to delineate the extent to which unitary executive functions might be shared across the separate domains of episodic and working memory. A mixed blocked/event-related functional magnetic resonance imaging (fMRI) design was employed to assess sustained (tonic control) and transient (phasic control) brain responses arising from incrementing executive demand (source versus item episodic memory - vis-à-vis - two-back versus one-back working memory) using load-dependent activation overlaps as indices of common components. Although an extensive portion of the regional load effects constituted differential control modulations in both sustained and transient responses, commonalities were also found implicating a subset of executive core mechanisms consistent with unitary or domain general control. 'Unitary' control modulations were temporally dissociated into (1) shared tonic components involving medial and lateral prefrontal cortex, striatum, cerebellum and superior parietal cortex, assumed to govern enhanced top-down context processing, monitoring and sustained attention throughout task periods and (2) stimulus-synchronous phasic components encompassing posterior intraparietal sulcus, hypothesized to support dynamic shifting of the 'focus of attention' among internal representations. Taken together, these results converge with theoretical models advocating both unity and diversity among executive control processes.

The ability of the mesocortical dopamine system to operate in distinct temporal modes

BACKGROUND

This review discusses evidence that cells in the mesocortical dopamine (DA) system influence information processing in target areas across three distinct temporal domains.

DISCUSSIONS

Phasic bursting of midbrain DA neurons may provide temporally precise information about the mismatch between expected and actual rewards (prediction errors) that has been hypothesized to serve as a learning signal in efferent regions. However, because DA acts as a relatively slow modulator of cortical neurotransmission, it is unclear whether DA can indeed act to precisely transmit prediction errors to prefrontal cortex (PFC). In light of recent physiological and anatomical evidence, we propose that corelease of glutamate from DA and/or non-DA neurons in the VTA could serve to transmit this temporally precise signal. In contrast, DA acts in a protracted manner to provide spatially and temporally diffuse modulation of PFC pyramidal neurons and interneurons. This modulation occurs first via a relatively rapid depolarization of fast-spiking interneurons that acts on the order of seconds. This is followed by a more protracted modulation of a variety of other ionic currents on timescales of minutes to hours, which may bias the manner in which cortical networks process information. However, the prolonged actions of DA may be curtailed by counteracting influences, which likely include opposing actions at D1 and D2-like receptors that have been shown to be time- and concentration-dependent. In this way, the mesocortical DA system optimizes the characteristics of glutamate, GABA, and DA neurotransmission both within the midbrain and cortex to communicate temporally precise information and to modulate network activity patterns on prolonged timescales.

Dopaminergic modulation of local network activity in rat prefrontal cortex

Dopamine modulates prefrontal cortex excitability in complex ways. Dopamine's net effect on local neuronal networks is therefore difficult to predict based on studies on pharmacologically isolated excitatory or inhibitory connections. In the present work, we have studied the effects of dopamine on evoked activity in acute rat brain slices when both excitation and inhibition are intact. Whole cell recordings from layer II/III pyramidal cells under conditions of normal synaptic transmission showed that bath-applied dopamine (30 microM) increased the outward inhibitory component of composite postsynaptic currents, whereas inward excitatory currents were not significantly affected. Optical imaging with the voltage-sensitive dye N-(3-(triethylammonium)propyl)-4-(4-(p-diethylaminophenyl)buta-dienyl)pyridinium dibromide revealed that bath application of dopamine significantly decreased the amplitude, duration, and lateral spread of activity in local cortical networks. This effect of dopamine was observed both with single and train (5 at 20 Hz) stimuli. The effect was mimicked by the D1-like receptor agonistR(+)-6-chloro-7,8-dihydroxy-1-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrobromide (1 microM) and was blocked by R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (10 microM), a selective antagonist for D1-like receptors. The D2-like receptor agonist quinpirole (10 microM) had no significant effect on evoked dye signals. Our results suggest that dopamine's effect on inhibition dominates over that on excitation under conditions of normal synaptic transmission. Such neuromodulation by dopamine may be important for maintenance of stability in local neuronal networks in the prefrontal cortex.

COMT genotype predicts BOLD signal and noise characteristics in prefrontal circuits

OBJECTIVE

Prefrontal dopamine (DA) is catabolized by the COMT (catechol-O-methyltransferase) enzyme. Literature suggests that the Val/Met single nucleotide polymorphism (SNP) in the COMT gene predicts executive cognition in humans with Val carriers showing poorer performance due to less available synaptic DA. Recent fMRI studies are thought to agree with these studies having demonstrated prefrontal hyperactivation during n-back and attention-requiring tasks. This was interpreted as "less efficient" processing due to impaired signal-to-noise ratio (SNR) of neuronal activity. However, electrophysiological studies of neuronal SNR in primates and humans imply that prefrontal cortex should show a diminished prefrontal BOLD response in Val carriers. In the present study, we addressed the question of whether the prefrontal SNR of the BOLD response is decreased in Val carriers using a visual oddball task and an approach to analysis of fMRI data that maximizes noise characterization.

METHODS

We investigated N=17 homozygous Met carriers compared with N=24 Val carriers matched for age, sex, education, IQ, reaction time (variability) and head motion. Event-related fMRI was conducted presenting 160 visual stimuli (40 targets, checkerboard reversal). Subjects had to respond as quickly as possible to targets by button press. In the fMRI GLM [y(t)=beta*x(t)+c+e(t)] analysis, voxel-by-voxel 'activation' [y(t)] as well as residual noise variance [e(t)=sigma2] were calculated using a conservative full-width half maximum (FWHM=6 mm).

RESULTS

As compared to Val carriers, we observed a stronger and more extended BOLD responses in homozygous Met carriers in left supplementary motor area (SMA) extending to ACC and dorsolateral prefrontal cortex. Vice versa, increased levels of noise were seen in Val carriers surrounding the peak activation maximum.

DISCUSSION

In line with our expectations from prior electrophysiological studies, we observed a diminished BOLD response and increased noise in Val carriers. This suggests that the DA stabilizes cortical microcircuits by sharpening the signal and suppressing surrounding noise.

Impact of complex genetic variation in COMT on human brain function

Catechol-O-methyltransferase (COMT) has been shown to be critical for prefrontal dopamine flux, prefrontal cortex-dependent cognition and activation. Several potentially functional variants in the gene have been identified, but considerable controversy exists regarding the contribution of individual alleles and haplotypes to risk for schizophrenia, partly because clinical phenotypes are ill-defined and preclinical studies are limited by lack of adequate models. Here, we propose a neuroimaging approach to overcome these limitations by characterizing the functional impact of ambiguous haplotypes on a neural system-level intermediate phenotype in humans. Studying 126 healthy control subjects during a working-memory paradigm, we find that a previously described risk variant in a functional Val158Met (rs4680) polymorphism interacts with a P2 promoter region SNP (rs2097603) and an SNP in the 3' region (rs165599) in predicting inefficient prefrontal working memory response. We report evidence that the nonlinear response of prefrontal neurons to dopaminergic stimulation is a neural mechanism underlying these nonadditive genetic effects. This work provides an in vivo approach to functional validation in brain of the biological impact of complex genetic variations within a gene that may be critical for its clinical association.

DRD4 and DAT1 polymorphisms modulate human gamma band responses

Gamma oscillations (30-80 Hz) have been demonstrated to be important for perceptual and cognitive processes. Animal and in vitro studies have revealed possible underlying generation mechanisms of the gamma rhythm. However, little is known about the neurochemical modulation of these oscillations during human cognition. Schizophrenia and Attention Deficit Hyperactivity Disorder, which lead to failure of attentional modulation and working memory, introduce significant changes in gamma responses and have significant associations with genetic polymorphisms of dopamine receptor D4 (DRD4), dopamine transporter (DAT), and catechol-O-methyltransferase (COMT). Therefore, the presence of direct relations between these polymorphisms and gamma oscillations was investigated in human subjects using an auditory target detection paradigm. The 7-repeat isoform of the DRD4 polymorphism that produces a subsensitive variant of the D4 receptor enhanced the auditory evoked and induced gamma responses to both standard and target stimuli. The 10/10 genotype of the DAT1 polymorphism, which reduces DAT expression and hence yields an increase in extracellular dopamine, specifically enhanced evoked gamma responses to target stimuli. The COMT polymorphism did not significantly change gamma responses. It seems plausible to assume that the modulation pattern of the evoked gamma response by DRD4 polymorphism relates to reduced inhibition via the D4 receptor, whereas the DAT1 effect is related to the target detection mechanism probably mediated by the D1 receptor.

Beyond bistability: Biophysics and temporal dynamics of working memory

Working memory has often been modeled and conceptualized as a kind of binary (bistable) memory switch, where stimuli turn on plateau-like persistent activity in subsets of cells, in line with many in vivo electrophysiological reports. A potentially related form of bistability, termed up- and down-states, has been studied with regard to its synaptic and ionic basis in vivo and in reduced cortical preparations. Also single cell mechanisms for producing bistability have been proposed and investigated in brain slices and computationally. Recently, however, it has been emphasized that clear plateau-like bistable activity is rather rare during working memory tasks, and that neurons exhibit a multitude of different temporally unfolding activity profiles and temporal structure within their spiking dynamics. Hence, working memory seems to be a highly dynamical neural process with yet unknown mappings from dynamical to computational properties. Empirical findings on ramping activity profiles and temporal structure will be reviewed, as well as neural models that attempt to account for it and its computational significance. Furthermore, recent in vivo, neural culture, and in vitro preparations will be discussed that offer new possibilities for studying the biophysical mechanisms underlying computational processes during working memory. These preparations have revealed additional evidence for temporal structure and spatio-temporally organized attractor states in cortical networks, as well as for specific computational properties that may characterize synaptic processing during high-activity states as during working memory. Together such findings may lay the foundations for highly dynamical theories of working memory based on biophysical principles.

Dopaminergic control of working memory and its relevance to schizophrenia: A circuit dynamics perspective

This article argues how dopamine controls working memory and how the dysregulation of the dopaminergic system is related to schizophrenia. In the dorsolateral prefrontal cortex, which is the principal part of the working memory system, recurrent excitation is subtly balanced with intracortical inhibition. A potent controller of the dorsolateral prefrontal cortical circuit is the mesocortical dopaminergic system. To understand the characteristics of the dopaminergic control of working memory, the stability of the circuit dynamics under the influence of dopamine has been studied. Recent computational studies suggest that the hyperdopaminergic state is usually stable but the hypodopaminergic state tends to be unstable. The stability also depends on the efficacy of the glutamatergic transmission in the corticomesencephalic projections to dopamine neurons. When this cortical feedback is hypoglutamatergic, the circuit of the dorsolateral prefrontal cortex tends to be unstable, such that a slight increase in dopamine releasability causes a catastrophic jump of the dorsolateral prefrontal cortex activity from a low to a high level. This may account for the seemingly paradoxical overactivation of the dorsolateral prefrontal cortex observed in schizophrenic patients. Given that dopamine transmission is abnormal in the brains of patients with schizophrenia and working memory deficit is a core dysfunction in schizophrenia, the concept of circuit stability would be useful not only for understanding the mechanisms of working memory processing but for developing therapeutic strategies to enhance cognitive functions in schizophrenia.

Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia

The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and executive functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the mechanistic basis of executive function remains elusive, often amounting to a homunculus. This article presents an attempt to deconstruct this homunculus through powerful learning mechanisms that allow a computational model of the prefrontal cortex to control both itself and other brain areas in a strategic, task-appropriate manner. These learning mechanisms are based on subcortical structures in the midbrain, basal ganglia, and amygdala, which together form an actor-critic architecture. The critic system learns which prefrontal representations are task relevant and trains the actor, which in turn provides a dynamic gating mechanism for controlling working memory updating. Computationally, the learning mechanism is designed to simultaneously solve the temporal and structural credit assignment problems. The model's performance compares favorably with standard backpropagation-based temporal learning mechanisms on the challenging 1-2-AX working memory task and other benchmark working memory tasks.

Methylphenidate increases cortical excitability via activation of alpha-2 noradrenergic receptors

Although methylphenidate (MPH), a catecholaminergic reuptake blocker, is prescribed for attention-deficit/hyperactivity disorder, there is a dearth of information regarding the cellular basis of its actions. To address this issue, we used whole-cell patch-clamp recordings to investigate the roles of various catecholamine receptors in MPH-induced changes in cortical neuron excitability. We bath-applied dopamine or noradrenaline receptor antagonists in combination with MPH to pyramidal cells located in deep layers of the infralimbic and prelimbic prefrontal cortices. Application of MPH (10 microM) by itself increased cortical cell excitability in slices obtained from juvenile rats. This MPH-mediated increase in excitability was lost when catecholamines were depleted with reserpine prior to recording, demonstrating the requirement for a presynaptic monoamine component. Antagonist studies further revealed that stimulation of alpha-2 noradrenergic receptors mediates the MPH-induced increase in intrinsic excitability. Dopamine D1 receptors played no observable role in the actions of MPH. We therefore propose that MPH is acting to increase catecholaminergic tone in the PFC, and thereby increases cortical excitability by mediating the disinhibition of pyramidal cells through mechanisms that may include activation of alpha-2 adrenoreceptors located in interneurons.

Role of the Prefrontal Cortex in Attentional Control over Bistable Vision

The primary source of top-down attentional control in object perception is the prefrontal cortex. This region is involved in the maintenance of goal-related information as well as in attentional selection and set shifting. Recent approaches have emphasized the role of top-down processes during elementary visual processes as exemplified in bistable vision where perception oscillates automatically between two mutually exclusive states. The prefrontal cortex might influence this process either by maintaining the dominant pattern while protecting it against the competing representation, or by facilitating perceptual switches between the two competing representations. To address this issue, we investigated reported perceptual reversals in patients with circumscribed lesions of the prefrontal cortex and healthy control participants in three experimental conditions: hold (maintaining the dominant view), speed (inducing as many perceptual switches as possible), and neutral (no intervention). Results indicated that although the patients showed normal switching rates in the neutral condition and were able to control perceptual switches in the hold condition as much as control subjects were, they were less able to facilitate reversals specifically in the speed condition. These results suggest that the prefrontal cortex is necessary to bias the selection of visual representations in accord with current goals, but is less essential for maintaining selected information active that is continuously available in the environment. As for attentional selection, the present results suggest that the prefrontal cortex initiates perceptual reversals by withdrawing top-down support from the dominant representation without (or prior to) boosting the suppressed view.

A Large-scale Neurocomputational Model of Task-oriented Behavior Selection and Working Memory in Prefrontal Cortex

The prefrontal cortex (PFC) is crucially involved in the executive component of working memory, representation of task state, and behavior selection. This article presents a large-scale computational model of the PFC and associated brain regions designed to investigate the mechanisms by which working memory and task state interact to select adaptive behaviors from a behavioral repertoire. The model consists of multiple brain regions containing neuronal populations with realistic physiological and anatomical properties, including extrastriate visual cortical regions, the inferotemporal cortex, the PFC, the striatum, and midbrain dopamine (DA) neurons. The onset of a delayed match-to-sample or delayed nonmatch-to-sample task triggers tonic DA release in the PFC causing a switch into a persistent, stimulus-insensitive dynamic state that promotes the maintenance of stimulus representations within prefrontal networks. Other modeled prefrontal and striatal units select cognitive acceptance or rejection behaviors according to which task is active and whether prefrontal working memory representations match the current stimulus. Working memory task performance and memory fields of prefrontal delay units are degraded by extreme elevation or depletion of tonic DA levels. Analyses of cellular and synaptic activity suggest that hyponormal DA levels result in increased prefrontal activation, whereas hypernormal DA levels lead to decreased activation. Our simulation results suggest a range of predictions for behavioral, single-cell, and neuroimaging response data under the proposed task set and under manipulations of DA concentration.