Neuronal coding of prediction errors

Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex

Emotions are multifaceted, but a key aspect of emotion involves the assessment of the value of environmental stimuli. This article reviews the many psychological representations, including representations of stimulus value, which are formed in the brain during Pavlovian and instrumental conditioning tasks. These representations may be related directly to the functions of cortical and subcortical neural structures. The basolateral amygdala (BLA) appears to be required for a Pavlovian conditioned stimulus (CS) to gain access to the current value of the specific unconditioned stimulus (US) that it predicts, while the central nucleus of the amygdala acts as a controller of brainstem arousal and response systems, and subserves some forms of stimulus-response Pavlovian conditioning. The nucleus accumbens, which appears not to be required for knowledge of the contingency between instrumental actions and their outcomes, nevertheless influences instrumental behaviour strongly by allowing Pavlovian CSs to affect the level of instrumental responding (Pavlovian-instrumental transfer), and is required for the normal ability of animals to choose rewards that are delayed. The prelimbic cortex is required for the detection of instrumental action-outcome contingencies, while insular cortex may allow rats to retrieve the values of specific foods via their sensory properties. The orbitofrontal cortex, like the BLA, may represent aspects of reinforcer value that govern instrumental choice behaviour. Finally, the anterior cingulate cortex, implicated in human disorders of emotion and attention, may have multiple roles in responding to the emotional significance of stimuli and to errors in performance, preventing responding to inappropriate stimuli.

Functional aspects of dopamine metabolism in the putative prefrontal cortex analogue and striatum of pigeons (Columba livia)

Dopamine (DA) in mammalian associative structures, such as the prefrontal cortex (PFC), plays a prominent role in learning and memory processes, and its homeostasis differs from that of DA in the striatum, a sensorimotor region. The neostriatum caudolaterale (NCL) of birds resembles the mammalian PFC according to connectional, electrophysiological, and behavioral data. In the present study, DA regulation in the associative NCL and the striatal lobus parolfactorius (LPO) of pigeons was compared to uncover possible differences corresponding to those between mammalian PFC and striatum. Extracellular levels of DA and its metabolites (homovanillic acid [HVA], dihydroxyphenylacetic acid [DOPAC]) and the serotonin metabolite 5-hydroxyindoleacetic acid (5-HIAA) were investigated by in vivo microdialysis of urethane-anesthetized pigeons under basal conditions and after systemic administration of D-amphetamine. DA was reliably determined only in LPO dialysates, and DA metabolite levels were significantly higher in LPO than in NCL. The HVA/DOPAC ratio, indicating extracellular lifetime of DA, was more than twice as high in NCL than in LPO dialysates. After amphetamine, DA increased in LPO while still being undetectable in NCL, and DA metabolites decreased in both regions. 5-HIAA slightly decreased in NCL dialysates. Amphetamine effects were delayed in NCL compared with the striatum. In conclusion, effects of amphetamine on the pigeon's ascending monoamine systems resemble those found in mammals, suggesting similar regulatory properties. The neurochemical differences between NCL and LPO parallel those between associative regions, such as PFC and dorsal striatum in mammals. They may reflect weaker regulation of extracellular DA, favoring DAergic volume transmission, in associative than striatal forebrain regions.

Early life environment modulates 'handedness' in rats

Right handedness is one of the most prominent markers of human functional brain asymmetry. Deviation from this norm appears to be associated with certain developmental disorders. While many studies have dealt with the genetic contribution to the determination of handedness, few have examined whether environmental factors that are subtler than forced hand switching can modulate the development of handedness. In this study, we exposed rats to a novel environment for 3 min daily during their first 3 weeks of life and found that their paw preferences during both infancy and adulthood showed a leftward shift compared with the controls. This result suggests that 'handedness' can be modified by rather subtle early environmental manipulation. Since exposure to a novel environment does not involve a direct asymmetric activation of the sensory--motor system underlying paw-use, mechanisms beyond this paw-specific system must exist to mediate the observed modulation of 'handedness'.

Inhibition of climbing fibres is a signal for the extinction of conditioned eyelid responses

A fundamental tenet of cerebellar learning theories asserts that climbing fibre afferents from the inferior olive provide a teaching signal that promotes the gradual adaptation of movements. Data from several forms of motor learning provide support for this tenet. In pavlovian eyelid conditioning, for example, where a tone is repeatedly paired with a reinforcing unconditioned stimulus like periorbital stimulation, the unconditioned stimulus promotes acquisition of conditioned eyelid responses by activating climbing fibres. Climbing fibre activity elicited by an unconditioned stimulus is inhibited during the expression of conditioned responses-consistent with the inhibitory projection from the cerebellum to inferior olive. Here, we show that inhibition of climbing fibres serves as a teaching signal for extinction, where learning not to respond is signalled by presenting a tone without the unconditioned stimulus. We used reversible infusion of synaptic receptor antagonists to show that blocking inhibitory input to the climbing fibres prevents extinction of the conditioned response, whereas blocking excitatory input induces extinction. These results, combined with analysis of climbing fibre activity in a computer simulation of the cerebellar-olivary system, suggest that transient inhibition of climbing fibres below their background level is the signal that drives extinction.

Dopamine and glutamate release in the nucleus accumbens and ventral tegmental area of rat following lateral hypothalamic self-stimulation

Rewarding hypothalamic brain stimulation is thought to depend on trans-synaptic activation of high-threshold (and thus rarely directly depolarized by rewarding stimulation) dopaminergic fibers of the medial forebrain bundle. We used in vivo microdialysis and high-performance liquid chromatography coupled with electrochemical or fluorometric detection to investigate the concurrent release of dopamine and glutamate in the nucleus accumbens septi and in the ventral tegmental area, as a function of lateral hypothalamic self-stimulation.Self-stimulation at a variety of stimulation frequencies and pulse widths increased levels of dopamine and its primary metabolites, dihydroxyphenylacetic acid and homovanillic acid in the nucleus accumbens. Lateral hypothalamic self-stimulation also induced significant increases in ventral tegmental area dopamine and metabolite levels, and the percentage increase of dopamine was higher in this region than in the nucleus accumbens. Local perfusion with the dopamine uptake inhibitor nomifensine (10 microM) increased dopamine levels in the nucleus accumbens about three-fold and potentiated the increase of dopamine levels induced by self-stimulation. Nomifensine perfusion also induced a delayed decrease in nucleus accumbens glutamate levels, and self-stimulation did not modify this effect of the drug. Local perfusion with the D2-type dopamine receptor antagonist raclopride significantly increased both basal and self-stimulation induced dopamine release in the nucleus accumbens. Neither nomifensine nor raclopride perfusion significantly affected the maximal rates of self-stimulation. Perfusion with tetrodotoxin (2 microM) into nucleus accumbens significantly decreased basal and prevented stimulation-induced increases in accumbens dopamine levels but only slightly decreased the rate of self-stimulation. In contrast, perfusion of tetrodotoxin (0.5 microM) into the ventral tegmental area decreased basal and blocked stimulation-induced increases in both nucleus accumbens and ventral tegmental area dopamine levels; this treatment also blocked or strongly inhibited self-stimulation. While it had no effect on glutamate levels in the nucleus accumbens, lateral hypothalamic self-stimulation induced a significant and tetrodotoxin-sensitive increase in glutamate levels in the ventral tegmental area. Taken together, the present results indicate that, across a broad range of stimulation parameters, rewarding lateral hypothalamus stimulation causes major and persistent activation of the mesolimbic dopamine system, and suggest descending glutamatergic fibers in the medial forebrain bundle as a candidate for the directly activated descending pathway in lateral hypothalamus brain stimulation reward.

Brain and body hyperthermia associated with heroin self-administration in rats

Intravenous heroin self-administration in trained rats was accompanied by robust brain hyperthermia (+2.0-2.5 degrees C); parallel changes were found in the dorsal and ventral striatum, mediodorsal thalamus, and deep temporal muscle. Temperature began to increase at variable latency after a signal of drug availability, increased reliably (approximately 0.4 degrees C) before the first lever press for heroin, increased further (approximately 1.2 degrees C) after the first heroin injection, and rose more slowly after the second and third injections to stabilize at an elevated plateau (39-40 degrees C) for the remainder of the session. Brain and body temperature declined slowly when drug self-administration was terminated; naloxone precipitated a much more rapid decrease to baseline levels. Changes in temperature were similar across repeated daily sessions, except for the increase associated with the first self-administration of each session, which had progressively shorter latency and greater acceleration. Despite consistent biphasic fluctuations in movement activity associated with heroin self-administrations (gradual increase preceding the lever press, followed by an abrupt hypodynamia after drug infusion), mean brain temperature was very stable at an elevated plateau. Only mean muscle temperature showed evidence of biphasic fluctuations (+/-0.2 degrees C) that were time locked to and correlated with lever pressing and associated movements. Drug- and behavior-related changes in brain temperature thus appear to reflect some form of neuronal activation, and, because temperature is a factor capable of affecting numerous neural functions, it may be an important variable in the control of behavior by drugs of abuse.

Role of human frontal and supplementary eye fields in double step saccades

Transcranial magnetic stimulation (TMS) was applied over the frontal eye field (FEF) and the supplementary eye field (SEF) regions before execution of the first saccade in a double step task. When applied over the FEF, stimulation increased the percentage error in amplitude of the contralateral second saccade as compared to no stimulation. This was due to an interference with retinotopic but not craniotopic gain calculation. Stimulation of the SEF region interfered with saccade ordering. Thus, FEF might participate in target memorization whereas SEF is confirmed to code order information for sequential saccades even in this paradigm of only two consecutive movements.

Prefrontal neurons and the genetics of schizophrenia

This article reviews prefrontal cortical biology as it relates to pathophysiology and genetic risk for schizophrenia. Studies of prefrontal neurocognition and functional neuroimaging of prefrontal information processing consistently reveal abnormalities in patients with schizophrenia. Abnormalities of prefrontal information processing also are found in unaffected individuals who are genetically at risk for schizophrenia, suggesting that genetic polymorphisms affecting prefrontal function may be susceptibility alleles for schizophrenia. One such candidate is a functional polymorphism in the catechol-o-methyl transferase (COMT) gene that markedly affects enzyme activity and that appears to uniquely impact prefrontal dopamine. The COMT genotype predicts performance on prefrontal executive cognition and working memory tasks. Functional magnetic resonance imaging confirms that COMT genotype affects prefrontal physiology during working memory. Family-based association studies have revealed excessive transmission to schizophrenic offspring of the allele (val) related to poorer prefrontal function. These various data provide convergent evidence that the COMT val allele increases risk for schizophrenia by virtue of its effect on dopamine-mediated prefrontal information processing-the first plausible mechanism for a genetic effect on normal human cognition and risk for mental illness.

Beautiful faces have variable reward value: fMRI and behavioral evidence

The brain circuitry processing rewarding and aversive stimuli is hypothesized to be at the core of motivated behavior. In this study, discrete categories of beautiful faces are shown to have differing reward values and to differentially activate reward circuitry in human subjects. In particular, young heterosexual males rate pictures of beautiful males and females as attractive, but exert effort via a keypress procedure only to view pictures of attractive females. Functional magnetic resonance imaging at 3 T shows that passive viewing of beautiful female faces activates reward circuitry, in particular the nucleus accumbens. An extended set of subcortical and paralimbic reward regions also appear to follow aspects of the keypress rather than the rating procedures, suggesting that reward circuitry function does not include aesthetic assessment.

Making choices: the neurophysiology of visual-saccadic decision making

Imagine the decisions you might make while playing a simple game like 'matching pennies'. At each play, you and your opponent, say the mathematician John vonNeumann, each lay down a penny heads or tails up. If both pennies show the same side, vonNeumann wins, if not, you win. Before each play, you have the subjective experience of deciding what to do: of choosing whether to play heads or tails. Although decisions like these are not yet understood at a physiological level, progress has been made towards understanding simple decision making in at least one model system: the primate neural architecture that uses visual data and prior knowledge about patterns in the environment to select and execute saccades. Both the visual system and the brainstem circuits that control saccadic eye movements are particularly well understood, making it possible for physiologists to begin to study the connections between these sensory and motor processes at a level of complexity that would be impossible in other less well understood systems.

Role of tonically active neurons in primate caudate in reward-oriented saccadic eye movement

Recent studies have suggested that the basal ganglia are essential for reward-oriented behavior. A popular proposal is that the interaction between sensorimotor and reward-related signals occurs in the striatal projection neurons. However, the role of interneurons remains unclear. Using the one-direction-rewarded version of the memory-guided saccade task (1DR), we examined the activity of tonically active neurons (TANs), presumed cholinergic interneurons, in the caudate. Many TANs (73/155, 47.1%) responded, usually with a pause, to a visual cue that indicated both the saccade goal and the presence or absence of reward. For most TANs (44/73, 60.3%), the response was spatially selective (contralateral dominant), but was not modulated by the reward significance. TANs are thus distinct from caudate projection neurons, which have responses to the cue that are both spatially selective and reward contingent, and from midbrain dopamine neurons, which have cue responses that are spatially nonselective and reward contingent. TANs were nonetheless sensitive to the reward schedule: in the all-directions-rewarded version (ADR) compared with 1DR, the cue responses of TANs were smaller, less frequent, and less spatially selective. In 1DR, it would first be detected that reward is not given regularly, and this process would then promote discrimination of individual stimuli in relation to reward. We propose that TANs would contribute to the detection of the context that requires discrimination, whereas dopamine neurons would contribute to the stimulus discrimination. These features of TANs might be explained by their cytoarchitecture, namely, as large aspiny neurons.

The neuropsychological basis of addictive behaviour

The argument advanced in this review is that drug addiction can be understood in terms of normal learning and memory systems of the brain which, through the actions of chronically self-administered drugs, are pathologically subverted, thereby leading to the establishment of compulsive drug-seeking habits, strengthened by the motivational impact of drug-associated stimuli and occurring at the expense of other sources of reinforcement. We review data from our studies that have utilized procedures which reveal the various influences of pavlovian stimuli on goal-directed behaviour, namely discriminated approach, pavlovian-to-instrumental transfer and conditioned reinforcement, in order to demonstrate their overlapping and also unique neural bases. These fundamental studies are also reviewed in the context of the neural and psychological mechanisms underlying drug-seeking behaviour that is under the control of drug-associated environmental stimuli. The ways in which such drug-seeking behaviour becomes compulsive and habitual, as well as the propensity for relapse to drug-seeking even after long periods of relapse, are discussed in terms of the aberrant learning set in train by the effects of self-administered drugs on plastic processes in limbic cortical-ventral striatal systems.

Responses of human frontal cortex to surprising events are predicted by formal associative learning theory

Learning depends on surprise and is not engendered by predictable occurrences. In this functional magnetic resonance imaging (fMRI) study of causal associative learning, we show that dorsolateral prefrontal cortex (DLPFC) is associated specifically with the adjustment of inferential learning on the basis of unpredictability. At the outset, when all associations were unpredictable, DLPFC activation was maximal. This response attenuated with learning but, subsequently, activation here was evoked by surprise violations of the learned association. Furthermore, the magnitude of DLPFC response to a surprise event was sensitive to the relationship that had been learned and was predictive of subsequent behavioral change. In short, the physiological response properties of right DLPFC satisfied specific predictions made by associative learning theory.

The context of uncertainty modulates the subcortical response to predictability

Implicit motor learning tasks typically involve comparisons of subject responses during a sequence versus a random condition. In neuroimaging, brain regions that are correlated with a sequence are described, but the temporal relationship of sequence versus nonsequence conditions is often not explored. We present a functional magnetic resonance imaging (fMRI) study describing activation related to sequential predictability in an implicit sensorimotor learning task and the history (context) dependence of these effects. Participants regarded four squares displayed horizontally across a screen and pressed a button when any one of the four targets was illuminated in a particular color. A repeating spatial sequence with varying levels of predictability was embedded within a random color presentation. Both the right dorsolateral prefrontal cortex (R DLPFC) and right caudate displayed a positive correlation to increasing predictability, whereas the left posterior parietal cortex (L PPC) displayed a negative correlation. However, the activation changes within the caudate were significant when transitioning from high predictability to low predictability but not for the reverse case, suggesting a sensitivity not only to predictability but to order effects as well. These results support the hypothesized relationship between basal ganglia and visuomotor sequential learning, but demonstrate the importance of context upon sequence learning.

An integrative theory of prefrontal cortex function

The prefrontal cortex has long been suspected to play an important role in cognitive control, in the ability to orchestrate thought and action in accordance with internal goals. Its neural basis, however, has remained a mystery. Here, we propose that cognitive control stems from the active maintenance of patterns of activity in the prefrontal cortex that represent goals and the means to achieve them. They provide bias signals to other brain structures whose net effect is to guide the flow of activity along neural pathways that establish the proper mappings between inputs, internal states, and outputs needed to perform a given task. We review neurophysiological, neurobiological, neuroimaging, and computational studies that support this theory and discuss its implications as well as further issues to be addressed

Effect of levodopa in combination with physiotherapy on functional motor recovery after stroke: a prospective, randomised, double-blind study

BACKGROUND

Functional disability is generally caused by hemiplegia after stroke. Physiotherapy used to be the only way of improving motor function in such patients. However, administration of amphetamines in addition to exercise improves motor recovery in animals, probably by increasing the concentration of norepinephrine in the central nervous system. Our aim was to ascertain whether levodopa could enhance the efficacy of physiotherapy after hemiplegia.

METHODS

We did a prospective, randomised, placebo-controlled, double-blind study in which we enrolled 53 primary stroke patients. For the first 3 weeks patients received single doses of levodopa 100 mg or placebo daily in combination with physiotherapy. For the second 3 weeks patients had only physiotherapy. We quantitatively assessed motor function every week with Rivermead motor assessment (RMA).

FINDINGS

Six patients were excluded from analyses because of non-neurological complications. Motor recovery was significantly improved after 3 weeks of drug intervention in those on levodopa (RMA improved by 6.4 points) compared with placebo (4.1), and the result was independent of initial degree of impairment (p<0.004). The advantage of the levodopa group was maintained at study endpoint 3 weeks after levodopa was stopped. At the end of the study the total RMA score gain for the levodopa group was 8.2 points compared with 5.7 in the placebo group (p=0.020).

INTERPRETATION

A single dose of levodopa is well tolerated and, when given in combination with physiotherapy, enhances motor recovery in patients with hemiplegia. In view of its minimal side-effects, levodopa will be a possible add- on during stroke rehabilitation.

CONDITIONING WITH COMPOUND STIMULI IN DROSOPHILA MELANOGASTER IN THE FLIGHT SIMULATOR

Short-term memory in Drosophila melanogaster operant visual learning in the flight simulator is explored using patterns and colours as a compound stimulus. Presented together during training, the two stimuli accrue the same associative strength whether or not a prior training phase rendered one of the two stimuli a stronger predictor for the reinforcer than the other(no blocking). This result adds Drosophila to the list of other invertebrates that do not exhibit the robust vertebrate blocking phenomenon. Other forms of higher-order learning, however, were detected: a solid sensory preconditioning and a small second-order conditioning effect imply that associations between the two stimuli can be formed, even if the compound is not reinforced.

Reward signaling by dopamine neurons

Dopamine projections from the midbrain to the striatum and frontal cortex are involved in behavioral reactions controlled by rewards, as inferred from deficits in parkinsonism, schizophrenia, and drug addiction. Recent experiments have shown that dopamine neurons are not directly modulated in relation to movements. Rather, they appear to code the rewarding aspects of environmental stimuli. They show short, phasic increases of activity following primary food and liquid rewards ("unconditioned stimuli") and conditioned, reward-predicting stimuli of visual, auditory, and somatosensory modalities. They also display smaller activation-depression sequences after stimuli resembling rewards and after novel or particularly intense stimuli. Rewards are only reported as far as they occur differently than predicted. According to learning theories, a "prediction error" message may constitute a powerful teaching signal for behavior and learning. The phasic reward message is different from the more tonic enabling function of dopamine that is deficient in Parkinson's disease, indicating that dopamine neurons subserve different functions at different time scales. Neurons in other brain structures, such as the striatum, orbitofrontal cortex, and amygdala, code the quality, quantity, and preference of rewards. The dopamine reward prediction error signal may cooperate with these reward perception signals during the learning and performance of behavioral reactions to motivating environmental stimuli.

Dopamine responses comply with basic assumptions of formal learning theory

According to contemporary learning theories, the discrepancy, or error, between the actual and predicted reward determines whether learning occurs when a stimulus is paired with a reward. The role of prediction errors is directly demonstrated by the observation that learning is blocked when the stimulus is paired with a fully predicted reward. By using this blocking procedure, we show that the responses of dopamine neurons to conditioned stimuli was governed differentially by the occurrence of reward prediction errors rather than stimulus-reward associations alone, as was the learning of behavioural reactions. Both behavioural and neuronal learning occurred predominantly when dopamine neurons registered a reward prediction error at the time of the reward. Our data indicate that the use of analytical tests derived from formal behavioural learning theory provides a powerful approach for studying the role of single neurons in learning.

Cortical remodelling induced by activity of ventral tegmental dopamine neurons

Representations of sensory stimuli in the cerebral cortex can undergo progressive remodelling according to the behavioural importance of the stimuli. The cortex receives widespread projections from dopamine neurons in the ventral tegmental area (VTA), which are activated by new stimuli or unpredicted rewards, and are believed to provide a reinforcement signal for such learning-related cortical reorganization. In the primary auditory cortex (AI) dopamine release has been observed during auditory learning that remodels the sound-frequency representations. Furthermore, dopamine modulates long-term potentiation, a putative cellular mechanism underlying plasticity. Here we show that stimulating the VTA together with an auditory stimulus of a particular tone increases the cortical area and selectivity of the neural responses to that sound stimulus in AI. Conversely, the AI representations of nearby sound frequencies are selectively decreased. Strong, sharply tuned responses to the paired tones also emerge in a second cortical area, whereas the same stimuli evoke only poor or non-selective responses in this second cortical field in naive animals. In addition, we found that strong long-range coherence of neuronal discharge emerges between AI and this secondary auditory cortical area.

Limbic-cortical-ventral striatal activation during retrieval of a discrete cocaine-associated stimulus: a cellular imaging study with gamma protein kinase C expression

We investigated the neuronal activation associated with reexposure to a discrete cocaine-associated stimulus using in situ hybridization to quantify the expression of the plasticity-regulated gene, gamma protein kinase C (gamma PKC), in the limbic-cortical-ventral striatal system. Groups of rats were trained to self-administer cocaine paired with a light stimulus (Paired) or paired with an auditory stimulus but also receiving light presentations yoked to those in the Paired group (Unpaired). Additional groups received noncontingent cocaine-light pairings (Pavlovian) or saline-light pairings (Saline) that were yoked to the Paired group. After acquisition of self-administration by the Paired and Unpaired groups, all groups had a 3 d drug- and training-free period before being reexposed to noncontingent presentations of the light conditioning stimulus during a 5 min test session in the training context. There were four major patterns of results for regional gamma PKC expression 2 hr later. (1) Changes occurred only in groups in which the light was predictive of cocaine. (2) Increases were seen in the amygdala, but decreases were seen in the medial prefrontal cortex. (3) No changes were seen in the hippocampus. (4) Although changes were observed in the basal and central nuclei of the amygdala and the prelimbic cortex in both the Paired and Pavlovian groups, additional changes were observed in the nucleus accumbens core, lateral amygdala, and anterior cingulate cortex in the Pavlovian group. These results suggest not only that regionally selective alterations in gamma PKC expression are an index of the retrieval of Pavlovian associations formed between a drug and a discrete stimulus, but also that a distinct neural circuitry may underlie Pavlovian stimulus-reward associations in cocaine-experienced rats.

Dopamine production in the caudate putamen restores feeding in dopamine-deficient mice

Dopamine-deficient (DD) mice cannot synthesize dopamine (DA) in dopaminergic neurons due to selective inactivation of the tyrosine hydroxylase gene in those neurons. These mice become hypoactive and hypophagic and die of starvation by 4 weeks of age. We used gene therapy to ascertain where DA replacement in the brain restores feeding and other behaviors in DD mice. Restoration of DA production within the caudate putamen restores feeding on regular chow and nest-building behavior, whereas restoration of DA production in the nucleus accumbens restores exploratory behavior. Replacement of DA to either region restores preference for sucrose or a palatable diet without fully rescuing coordination or initiation of movement. These data suggest that a fundamental difference exists between feeding for sustenance and the ability to prefer rewarding substances.

The cognitive neuroscience of sustained attention: where top-down meets bottom-up

The psychological construct 'sustained attention' describes a fundamental component of attention characterized by the subject's readiness to detect rarely and unpredictably occurring signals over prolonged periods of time. Human imaging studies have demonstrated that activation of frontal and parietal cortical areas, mostly in the right hemisphere, are associated with sustained attention performance. Animal neuroscientific research has focused on cortical afferent systems, particularly on the cholinergic inputs originating in the basal forebrain, as crucial components of the neuronal network mediating sustained attentional performance. Sustained attention performance-associated activation of the basal forebrain corticopetal cholinergic system is conceptualized as a component of the 'top-down' processes initiated by activation of the 'anterior attention system' and designed to mediate knowledge-driven detection and selection of target stimuli. Activated cortical cholinergic inputs facilitate these processes, particularly under taxing attentional conditions, by enhancing cortical sensory and sensory-associational information processing, including the filtering of noise and distractors. Collectively, the findings from human and animal studies provide the basis for a relatively precise description of the neuronal circuits mediating sustained attention, and the dissociation between these circuits and those mediating the 'arousal' components of attention.

Flexibility in a Single Behavioral Variable of Drosophila

The flexibility of behavior is so rich, and its components are so exquisitely interwoven, that one may be well advised to turn to an isolated behavioral module for study. Gill withdrawal inAplysia, the proboscis extension reflex in the honeybee, and lid closure in mammals are such examples. We have chosen yawing, a single component of flight orientation in Drosophila melanogaster, for this approach. A specialty of this preparation is that the behavioral output can be reduced beyond the single module by one further step. It can be studied in tethered animals in which all turns are blocked while the differentially beating wings still provide the momentum. These intended yaw turns are measured by a torque meter to which the fly is hooked. The fly is held horizontally as if cruising at high speed. The head is glued to the thorax. It can bend its abdomen, extend its proboscis, and move its legs but cannot shift its direction of gaze or its orientation in space. Evidently, a fly hardly ever encounters this bizarre situation in the wild. We describe here the flexibility in this single behavioral variable. It provides insights into the relation between classical and operant conditioning, the processing of and interactions between the conditioned visual stimuli, early visual memory, visual pattern recognition, selective attention, and several other experience-dependent properties of visual orientation behavior. We start with a brief summary of visual flight control at the torque meter.

Performance monitoring by the supplementary eye field

Intelligent behaviour requires self-control based on the consequences of actions. The countermanding task is designed to study self-control; it requires subjects to withhold planned movements in response to an imperative stop signal, which they can do with varying success. In humans, the medial frontal cortex has been implicated in the supervisory control of action. In monkeys, the supplementary eye field in the dorsomedial frontal cortex is involved in producing eye movements, but its precise function has not been clarified. To investigate the role of the supplementary eye field in the control of eye movements, we recorded neural activity in macaque monkeys trained to perform an eye movement countermanding task. Distinct groups of neurons were active after errors, after successful withholding of a partially prepared movement, or in association with reinforcement. These three forms of activation could not be explained by sensory or motor factors. Our results lead us to put forward the hypothesis that the supplementary eye field contributes to monitoring the context and consequences of eye movements.

Multiple reward signals in the brain

The fundamental biological importance of rewards has created an increasing interest in the neuronal processing of reward information. The suggestion that the mechanisms underlying drug addiction might involve natural reward systems has also stimulated interest. This article focuses on recent neurophysiological studies in primates that have revealed that neurons in a limited number of brain structures carry specific signals about past and future rewards. This research provides the first step towards an understanding of how rewards influence behaviour before they are received and how the brain might use reward information to control learning and goal-directed behaviour.

Learning and selective attention

Selective attention involves the differential processing of different stimuli, and has widespread psychological and neural consequences. Although computational modeling should offer a powerful way of linking observable phenomena at different levels, most work has focused on the relatively narrow issue of constraints on processing resources. By contrast, we consider statistical and informational aspects of selective attention, divorced from resource constraints, which are evident in animal conditioning experiments involving uncertain predictions and unreliable stimuli. Neuromodulatory systems and limbic structures are known to underlie attentional effects in such tasks.

Is heterosynaptic modulation essential for stabilizing Hebbian plasticity and memory?

In 1894, Ramón y Cajal first proposed that memory is stored as an anatomical change in the strength of neuronal connections. For the following 60 years, little evidence was recruited in support of this idea. This situation changed in the middle of the twentieth century with the development of cellular techniques for the study of synaptic connections and the emergence of new formulations of synaptic plasticity that redefined Ramón y Cajal's idea, making it more suitable for testing. These formulations defined two categories of plasticity, referred to as homosynaptic or Hebbian activity-dependent, and heterosynaptic or modulatory input-dependent. Here we suggest that Hebbian mechanisms are used primarily for learning and for short-term memory but often cannot, by themselves, recruit the events required to maintain a long-term memory. In contrast, heterosynaptic plasticity commonly recruits long-term memory mechanisms that lead to transcription and to synpatic growth. When jointly recruited, homosynaptic mechanisms assure that learning is effectively established and heterosynaptic mechanisms ensure that memory is maintained.

The prefrontal cortex and cognitive control

One of the enduring mysteries of brain function concerns the process of cognitive control. How does complex and seemingly willful behaviour emerge from interactions between millions of neurons? This has long been suspected to depend on the prefrontal cortex--the neocortex at the anterior end of the brain--but now we are beginning to uncover its neural basis. Nearly all intended behaviour is learned and so depends on a cognitive system that can acquire and implement the 'rules of the game' needed to achieve a given goal in a given situation. Studies indicate that the prefrontal cortex is central in this process. It provides an infrastructure for synthesizing a diverse range of information that lays the foundation for the complex forms of behaviour observed in primates.