Position sensitivity in phasically discharging nucleus accumbens neurons of rats alternating between tasks requiring complementary types of spatial cues

Nucleus Accumbens Shell Neurons Encode the Kinematics of Reward Approach Locomotion

The nucleus accumbens (NAc) is considered an interface between motivation and action, with NAc neurons playing an important role in promoting reward approach. However, the encoding by NAc neurons that contributes to this role remains unknown. We recorded 62 NAc neurons in male Wistar rats (n = 5) running towards rewarded locations in an 8-arm radial maze. Variables related to locomotor approach kinematics were the best predictors of the firing rate for most NAc neurons. Nearly 18% of the recorded neurons were inhibited during the entire approach run (locomotion-off cells), suggesting that reduction in firing of these neurons promotes initiation of locomotor approach. 27% of the neurons presented a peak of activity during acceleration followed by a valley during deceleration (acceleration-on cells). Together, these neurons accounted for most of the speed and acceleration encoding identified in our analysis. In contrast, a further 16% of neurons presented a valley during acceleration followed by a peak just prior to or after reaching reward (deceleration-on cells). These findings suggest that these three classes of NAc neurons influence the time course of speed changes during locomotor approach to reward.

Activation of prefrontal cortex and striatal regions in rats after shifting between rules in a T-maze

Prefrontal cortical and striatal areas have been identified by inactivation or lesion studies to be required for behavioral flexibility, including selecting and processing of different types of information. In order to identify these networks activated selectively during the acquisition of new reward contingency rules, rats were trained to discriminate orientations of bars presented in pseudorandom sequence on two video monitors positioned behind the goal sites on a T-maze with return arms. A second group already trained in the visual discrimination task learned to alternate left and right goal arm visits in the same maze while ignoring the visual cues still being presented. In each experimental group, once the rats reached criterion performance, the brains were prepared after a 90-min delay for later processing for c-fos immunohistochemistry. While both groups extinguished a prior strategy and acquired a new rule, they differed by the identity of the strategies and previous learning experience. Among the 28 forebrain areas examined, there were significant increases in the relative density of c-fos immunoreactive cell bodies after learning the second rule in the prefrontal cortex cingulate, the prelimbic and infralimbic areas, the dorsomedial striatum and the core of the nucleus accumbens, the ventral subiculum, and the central nucleus of the amygdala. These largely correspond to structures previously identified in inactivation studies, and their neurons fire synchronously during learning and strategy shifts. The data suggest that this dynamic network may underlie reward-based selection for action-a type of cognitive flexibility.

Distributed cell assemblies spanning prefrontal cortex and striatum

Highly synchronous neuronal assembly activity is deemed essential for cognitive brain function. In theory, such synchrony could coordinate multiple brain areas performing complementary processes. However, cell assemblies have been observed only in single structures, typically cortical areas, and little is known about their synchrony with downstream subcortical structures, such as the striatum. Here, we demonstrate distributed cell assemblies activated at high synchrony (∼10 ms) spanning prefrontal cortex and striatum. In addition to including neurons at different brain hierarchical levels, surprisingly, they synchronized functionally distinct limbic and associative sub-regions. These assembly activations occurred when members shifted their firing phase relative to ongoing 4 Hz and theta rhythms, in association with high gamma oscillations. This suggests that these rhythms could mediate the emergence of cross-structural assemblies. To test for the role of assemblies in behavior, we trained the rats to perform a task requiring cognitive flexibility, alternating between two different rules in a T-maze. Overall, assembly activations were correlated with task-relevant parameters, including impending choice, reward, rule, or rule order. Moreover, these behavioral correlates were more robustly expressed by assemblies than by their individual member neurons. Finally, to verify whether assemblies can be endogenously generated, we found that they were indeed spontaneously reactivated during sleep and quiet immobility. Thus, cell assemblies are a more general coding mechanism than previously envisioned, linking distributed neocortical and subcortical areas at high synchrony.

Flexible use of allocentric and egocentric spatial memories activates differential neural networks in mice

Goal-directed navigation can be based on world-centered (allocentric) or body-centered (egocentric) representations of the environment, mediated by a wide network of interconnected brain regions, including hippocampus, striatum and prefrontal cortex. The relative contribution of these regions to navigation from novel or familiar routes, that demand a different degree of flexibility in the use of the stored spatial representations, has not been completely explored. To address this issue, we trained mice to find a reward relying on allocentric or egocentric information, in a modified version of the cross-maze task. Then we used Zif268 expression to map brain activation when well-trained mice were required to find the goal from a novel or familiar location. Successful navigation was correlated with the activation of CA1, posterior-dorsomedial striatum, nucleus accumbens core and infralimbic cortex when allocentric-trained mice needed to use a novel route. Allocentric navigation from a familiar route activated dorsomedial striatum, nucleus accumbens, prelimbic and infralimbic cortex. None of the structures analyzed was significantly activated in egocentric-trained mice, irrespective of the starting position. These data suggest that a flexible use of stored allocentric information, that allows goal finding even from a location never explored during training, induces a shift from fronto-striatal to hippocampal circuits.

Cerebral perfusion mapping during retrieval of spatial memory in rats

Studies of brain functional activation during spatial navigation using electrophysiology and immediate-early gene responses have typically targeted a limited number of brain regions. Our study provides the first whole brain analysis of cerebral activation during retrieval of spatial memory in the freely-moving rat. Rats (LEARNERS) were trained in the Barnes maze, an allocentric spatial navigation task, while CONTROLS received passive exposure. After 19 days, functional brain mapping was performed during recall by bolus intravenous injection of [¹⁴C]-iodoantipyrine using a novel subcutaneous minipump triggered by remote activation. Regional cerebral blood flow (rCBF)-related tissue radioactivity was analyzed by statistical parametric mapping from autoradiographic images of the three-dimensionally reconstructed brains. Functional connectivity was examined between regions of the spatial navigation circuit through interregional correlation analysis. Significant rCBF increases were noted in LEARNERS compared to CONTROLS broadly across the spatial navigation circuit, including the hippocampus (anterior dorsal CA1, posterior ventral CA1-3), subiculum, thalamus, striatum, medial septum, cerebral cortex, with decreases noted in the mammillary nucleus, amygdala and insula. LEARNERS showed a significantly greater positive correlation of rCBF of the ventral hippocampus with retrosplenial, lateral orbital, parietal and primary visual cortex, and a significantly more negative correlation with the mammillary nucleus, amygdala, posterior entorhinal cortex, and anterior thalamic nucleus. The complex sensory component of the spatial navigation task was underscored by broad activation across visual, somatosensory, olfactory, auditory and vestibular circuits which was enhanced in LEARNERS. Brain mapping facilitated by an implantable minipump represents a powerful tool for evaluation of mammalian behaviors dependent on locomotion.

Synchronicity: The Role of Midbrain Dopamine in Whole-Brain Coordination

Midbrain dopamine seems to play an outsized role in motivated behavior and learning. Widely associated with mediating reward-related behavior, decision making, and learning, dopamine continues to generate controversies in the field. While many studies and theories focus on what dopamine cells encode, the question of how the midbrain derives the information it encodes is poorly understood and comparatively less addressed. Recent anatomical studies suggest greater diversity and complexity of afferent inputs than previously appreciated, requiring rethinking of prior models. Here, we elaborate a hypothesis that construes midbrain dopamine as implementing a Bayesian selector in which individual dopamine cells sample afferent activity across distributed brain substrates, comprising evidence to be evaluated on the extent to which stimuli in the on-going sensorimotor stream organizes distributed, parallel processing, reflecting implicit value. To effectively generate a temporally resolved phasic signal, a population of dopamine cells must exhibit synchronous activity. We argue that synchronous activity across a population of dopamine cells signals consensus across distributed afferent substrates, invigorating responding to recognized opportunities and facilitating further learning. In framing our hypothesis, we shift from the question of how value is computed to the broader question of how the brain achieves coordination across distributed, parallel processing. We posit the midbrain is part of an “axis of agency” in which the prefrontal cortex (PFC), basal ganglia (BGS), and midbrain form an axis mediating control, coordination, and consensus, respectively.

Reassessing wanting and liking in the study of mesolimbic influence on food intake

Humans and animals such as rats and mice tend to overconsume calorie-dense foods, a phenomenon that likely contributes to obesity. One often-advanced explanation for why we preferentially consume sweet and fatty foods is that they are more "rewarding" than low-calorie foods. "Reward" has been subdivided into three interdependent psychological processes: hedonia (liking a food), reinforcement (formation of associations among stimuli, actions, and/or the food), and motivation (wanting the food). Research into these processes has focused on the mesolimbic system, which comprises both dopamine neurons in the ventral tegmental area and neurons in their major projection target, the nucleus accumbens. The mesolimbic system and closely connected structures are commonly referred to as the brain's "reward circuit." Implicit in this title is the assumption that "rewarding" experiences are generally the result of activity in this circuit. In this review, I argue that food intake and the preference for calorie-dense foods can be explained without reference to subjective emotions. Furthermore, the contribution of mesolimbic dopamine to food intake and preference may not be a general one of promoting or coordinating behaviors that result in the most reward or caloric intake but may instead be limited to the facilitation of a specific form of neural computation that results in conditioned approach behavior. Studies on the neural mechanisms of caloric intake regulation must address how sensory information about calorie intake affects not just the mesolimbic system but also many other forms of computation that govern other types of food-seeking and food-oriented behaviors.

Ramping single unit activity in the medial prefrontal cortex and ventral striatum reflects the onset of waiting but not imminent impulsive actions

The medial prefrontal cortex (mPFC) and ventral striatum (VS), including the nucleus accumbens, are key forebrain regions involved in regulating behaviour for future rewards. Dysfunction of these regions can result in impulsivity, characterized by actions that are mistimed and executed without due consideration of their consequences. Here we recorded the activity of single neurons in the mPFC and VS of 16 rats during performance on a five-choice serial reaction time task of sustained visual attention and impulsivity. Impulsive responses were assessed by the number of premature responses made before target stimuli were presented. We found that the majority of cells signalled trial outcome after an action was made (both rewarded and unrewarded). Positive and negative ramping activity was a feature of population activity in the mPFC and VS (49.5 and 50.4% of cells, respectively). This delay-related activity increased at the same rate and reached the same maximum (or minimum) for trials terminated by either correct or premature responses. However, on premature trials, the ramping activity started earlier and coincided with shorter latencies to begin waiting. For all trial types the pattern of ramping activity was unchanged when the pre-stimulus delay period was made variable. Thus, premature responses may result from a failure in the timing of the initiation of a waiting process, combined with a reduced reliance on external sensory cues, rather than a primary failure in delay activity. Our findings further show that the neural locus of this aberrant timing signal may emanate from structures outside the mPFC and VS.

Neuronal activity in the nucleus accumbens and hippocampus in rats during formation of seeking behavior in a radial maze

Seeking behavior of rats in a radial maze with asymmetric reward was studied by means of synchronous recording of cell activity in the hippocampus and ventral striatum. The synchrony of cell activity in the hippocampus and nucleus accumbens was modulated by spatial position and reward; the important role in this synchronization can be played by theta rhythm. This is in line with the anatomical and physiological data on the convergence of hippocampal spatially organized positional and reward value information inputs from the amygdala and ventral segmental area to n. accumbens.

Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies

Behavior in spatial navigation is often organized into map-based (place-driven) vs. map-free (cue-driven) strategies; behavior in operant conditioning research is often organized into goal-directed vs. habitual strategies. Here we attempt to unify the two. We review one powerful theory for distinct forms of learning during instrumental conditioning, namely model-based (maintaining a representation of the world) and model-free (reacting to immediate stimuli) learning algorithms. We extend these lines of argument to propose an alternative taxonomy for spatial navigation, showing how various previously identified strategies can be distinguished as “model-based” or “model-free” depending on the usage of information and not on the type of information (e.g., cue vs. place). We argue that identifying “model-free” learning with dorsolateral striatum and “model-based” learning with dorsomedial striatum could reconcile numerous conflicting results in the spatial navigation literature. From this perspective, we further propose that the ventral striatum plays key roles in the model-building process. We propose that the core of the ventral striatum is positioned to learn the probability of action selection for every transition between states of the world. We further review suggestions that the ventral striatal core and shell are positioned to act as “critics” contributing to the computation of a reward prediction error for model-free and model-based systems, respectively.

Reward cues in space: commonalities and differences in neural coding by hippocampal and ventral striatal ensembles

Forming place-reward associations critically depends on the integrity of the hippocampal-ventral striatal system. The ventral striatum (VS) receives a strong hippocampal input conveying spatial-contextual information, but it is unclear how this structure integrates this information to invigorate reward-directed behavior. Neuronal ensembles in rat hippocampus (HC) and VS were simultaneously recorded during a conditioning task in which navigation depended on path integration. In contrast to HC, ventral striatal neurons showed low spatial selectivity, but rather coded behavioral task phases toward reaching goal sites. Outcome-predicting cues induced a remapping of firing patterns in the HC, consistent with its role in episodic memory. VS remapped in conjunction with the HC, indicating that remapping can take place in multiple brain regions engaged in the same task. Subsets of ventral striatal neurons showed a "flip" from high activity when cue lights were illuminated to low activity in intertrial intervals, or vice versa. The cues induced an increase in spatial information transmission and sparsity in both structures. These effects were paralleled by an enhanced temporal specificity of ensemble coding and a more accurate reconstruction of the animal's position from population firing patterns. Altogether, the results reveal strong differences in spatial processing between hippocampal area CA1 and VS, but indicate similarities in how discrete cues impact on this processing.

Fast-spiking interneurons of the rat ventral striatum: temporal coordination of activity with principal cells and responsiveness to reward

Although previous in vitro studies revealed inhibitory synaptic connections of fast-spiking interneurons to principal cells in the striatum, uncertainty remains about the nature of the behavioural events that correlate with changes in interneuron activity and about the temporal coordination of interneuron firing with spiking of principal cells under natural conditions. Using in vivo tetrode recordings from the ventral striatum in freely moving rats, fast-spiking neurons were distinguished from putative medium-sized spiny neurons on the basis of their spike waveforms and rates. Cross-correlograms of fast-spiking and putative medium-sized spiny neuron firing patterns revealed a variety of temporal relationships, including peaks of concurrent firing and transient decrements in medium-sized spiny neuron spiking around fast-spiking unit activity. Notably, the onset of these decrements was mostly in advance of the fast-spiking unit firing. Many of these temporal relationships were dependent on the sleep-wake state. Coordinated activity was also found amongst pairs of the same phenotype, both fast-spiking units and putative medium-sized spiny neurons, which was often marked by a broad peak of concurrent firing. When studying fast-spiking neurons in a reward-searching task, they generally showed a pre-reward ramping increment in firing rate but a decrement specifically when the rat received reward. In conclusion, our data indicate that various forms of temporally coordinated activity exist amongst ventral striatal interneurons and principal cells, which cannot be explained by feed-forward inhibitory circuits alone. Furthermore, firing patterns of ventral striatal fast-spiking interneurons do not merely correlate with the general arousal state of the animal but display distinct reward-related changes in firing rate.

The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward

The basal ganglia are often conceptualised as three parallel domains that include all the constituent nuclei. The 'ventral domain' appears to be critical for learning flexible behaviours for exploration and foraging, as it is the recipient of converging inputs from amygdala, hippocampal formation and prefrontal cortex, putatively centres for stimulus evaluation, spatial navigation, and planning/contingency, respectively. However, compared to work on the dorsal domains, the rich potential for quantitative theories and models of the ventral domain remains largely untapped, and the purpose of this review is to provide the stimulus for this work. We systematically review the ventral domain's structures and internal organisation, and propose a functional architecture as the basis for computational models. Using a full schematic of the structure of inputs to the ventral striatum (nucleus accumbens core and shell), we argue for the existence of many identifiable processing channels on the basis of unique combinations of afferent inputs. We then identify the potential information represented in these channels by reconciling a broad range of studies from the hippocampal, amygdala and prefrontal cortex literatures with known properties of the ventral striatum from lesion, pharmacological, and electrophysiological studies. Dopamine's key role in learning is reviewed within the three current major computational frameworks; we also show that the shell-based basal ganglia sub-circuits are well placed to generate the phasic burst and dip responses of dopaminergic neurons. We detail dopamine's modulation of ventral basal ganglia's inputs by its actions on pre-synaptic terminals and post-synaptic membranes in the striatum, arguing that the complexity of these effects hint at computational roles for dopamine beyond current ideas. The ventral basal ganglia are revealed as a constellation of multiple functional systems for the learning and selection of flexible behaviours and of behavioural strategies, sharing the common operations of selection-by-disinhibition and of dopaminergic modulation.

Distinct subsets of nucleus accumbens neurons encode operant responding for ethanol versus water

Subsets of nucleus accumbens (NAc) neurons process information about operant responses for drugs as well as natural rewards (food and water) by excitations and inhibitions in firing rate time-locked to the operant response. The degree to which ensembles of neurons exhibit similar firing patterns when encoding cues and operant responses across different reinforcer conditions will provide critical information regarding the functional organization of this nucleus. The present experiment evaluated the relative contribution of subsets of accumbens neurons that encode distinct features of lever press responding for ethanol vs. water. Electrophysiological recordings (n = 153 neurons) were made in the accumbens of rats trained on concurrent reinforcement schedules for ethanol and water throughout a self-administration session. During operant responding, 52% of neurons exhibited patterned discharges characterized by significant increases or decreases in firing rate of +/- 1 s relative to lever presses for ethanol and/or water. Of these phasic cells, 85% discriminated between presses for ethanol and water (i.e. exhibited firing patterns unique to one reinforcer type), while 15% exhibited identical firing patterns relative to lever presses for both reinforcers. Notably, the data revealed that both high ethanol preference and spatially distinct lever positions contributed to the reinforcer specificity. Together, these data demonstrate that subsets of NAc neurons encode conditioned and instrumental aspects of ethanol vs. water reinforcement in well-trained rats, and that reinforcer preference and spatial cues are important components of this differential information processing.

Anticipatory reward signals in ventral striatal neurons of behaving rats

It has been proposed that the striatum plays a crucial role in learning to select appropriate actions, optimizing rewards according to the principles of 'Actor-Critic' models of trial-and-error learning. The ventral striatum (VS), as Critic, would employ a temporal difference (TD) learning algorithm to predict rewards and drive dopaminergic neurons. This study examined this model's adequacy for VS responses to multiple rewards in rats. The respective arms of a plus-maze provided rewards of varying magnitudes; multiple rewards were provided at 1-s intervals while the rat stood still. Neurons discharged phasically prior to each reward, during both initial approach and immobile waiting, demonstrating that this signal is predictive and not simply motor-related. In different neurons, responses could be greater for early, middle or late droplets in the sequence. Strikingly, this activity often reappeared after the final reward, as if in anticipation of yet another. In contrast, previous TD learning models show decremental reward-prediction profiles during reward consumption due to a temporal-order signal introduced to reproduce accurate timing in dopaminergic reward-prediction error signals. To resolve this inconsistency in a biologically plausible manner, we adapted the TD learning model such that input information is nonhomogeneously distributed among different neurons. By suppressing reward temporal-order signals and varying richness of spatial and visual input information, the model reproduced the experimental data. This validates the feasibility of a TD-learning architecture where different groups of neurons participate in solving the task based on varied input information.

Striatal versus hippocampal representations during win-stay maze performance

The striatum and hippocampus are widely held to be components of distinct memory systems that can guide competing behavioral strategies. However, some electrophysiological studies have suggested that neurons in both structures encode spatial information and may therefore make similar contributions to behavior. In rats well trained to perform a win-stay radial maze task, we recorded simultaneously from dorsal hippocampus and from multiple striatal subregions, including both lateral areas implicated in motor responses to cues and medial areas that work cooperatively with hippocampus in cognitive operations. In each brain region, movement through the maze was accompanied by the continuous sequential activation of sets of projection neurons. Hippocampal neurons overwhelmingly were active at a single spatial location (place cells). Striatal projection neurons were active at discrete points within the progression of every trial-especially during choices or following reward delivery-regardless of spatial position. Place-cell-type firing was not observed even for medial striatal cells entrained to the hippocampal theta rhythm. We also examined neural coding in earlier training sessions, when rats made use of spatial working memory to guide choices, and again found that striatal cells did not show place-cell-type firing. Prospective or retrospective encoding of trajectory was not observed in either hippocampus or striatum, at either training stage. Our results indicate that, at least in this task, dorsal hippocampus uses a spatial foundation for information processing that is not substantially modulated by spatial working memory demands. By contrast, striatal cells do not use such a spatial foundation, even in medial subregions that cooperate with hippocampus in the selection of spatial strategies. The progressive dominance of a striatum-dependent strategy does not appear to be accompanied by large changes in striatal or hippocampal single-cell representations, suggesting that the conflict between strategies may be resolved elsewhere.

Lesion of the ventral and intermediate hippocampus abolishes anticipatory activity in the medial prefrontal cortex of the rat

The medial prefrontal cortex (mPFC) of the rat receives a prominent input from the ventral two thirds of the hippocampus, a structure important for spatial awareness, working memory and motivation. We recently found [Hok V, Lenck-Santini PP, Roux S, Save E, Muller RU, Poucet B. Goal-related activity in hippocampal place cells. J Neurosci 2007;27:472-82.] that neurones in the dorsal hippocampus exhibit anticipatory firing prior to the release of a food pellet on an operant task. Here we look for similar activity in the mPFC on the same task and test whether this activity is dependent on the hippocampus. Rats were trained to navigate to a goal zone, wait for the release of a food pellet and then forage for the pellet while unit activity was recorded in the prelimbic and infralimbic areas of the mPFC. Two 16 min sessions were conducted per day, one session with the goal delimited by a cue disc, the second without the cue. In controls, a large proportion of mPFC neurones exhibited activity similar to that seen in the hippocampus while the animal was stationary at the goal. Over half exhibited the same activity regardless of goal location. Anticipatory activity was largely abolished in animals with bilateral lesions of the ventral and intermediate hippocampus, both in cued and uncued sessions. Even though lesioned animals continued to perform the task, they tended to leave the goal zone prematurely. We suggest that the anticipatory activity in the mPFC is dependent on similar activity in the hippocampus and that both structures have a role in either impulse control or reward expectation.

Learning processing in the basal ganglia: a mosaic of broken mirrors

In the present review we propose a model to explain the role of the basal ganglia in sensorimotor and cognitive functions based on a growing body of behavioural, anatomical, physiological, and neurochemical evidence accumulated over the last decades. This model proposes that the body and its surrounding environment are represented in the striatum in a fragmented and repeated way, like a mosaic consisting of the fragmented images of broken mirrors. Each fragment forms a functional unit representing articulated parts of the body with motion properties, objects of the environment which the subject can approach or manipulate, and locations the subject can move to. These units integrate the sensory properties and movements related to them. The repeated and widespread distribution of such units amplifies the combinatorial power of the associations among them. These associations depend on the phasic release of dopamine in the striatum triggered by the saliency of stimuli and will be reinforced by the rewarding consequences of the actions related to them. Dopamine permits synaptic plasticity in the corticostriatal synapses. The striatal units encoding the same stimulus/action send convergent projections to the internal segment of the globus pallidus (GPi) and to the substantia nigra pars reticulata (SNr) that stimulate or hold the action through a thalamus-frontal cortex pathway. According to this model, this is how the basal ganglia select actions based on environmental stimuli and store adaptive associations as nondeclarative memories such as motor skills, habits, and memories formed by Pavlovian and instrumental conditioning.

Preferential reactivation of motivationally relevant information in the ventral striatum

Spontaneous "off-line" reactivation of neuronal activity patterns may contribute to the consolidation of memory traces. The ventral striatum exhibits reactivation and has been implicated in the processing of motivational information. It is unknown, however, whether reactivating neuronal ensembles specifically recapitulate information relating to rewards that were encountered during wakefulness. We demonstrate a prolonged reactivation in rat ventral striatum during quiet wakefulness and slow-wave but not rapid eye movement sleep. Reactivation of reward-related information processed in this structure was particularly prominent, and this was primarily attributable to spike trains temporally linked to reward sites. It was accounted for by small, strongly correlated subgroups in recorded cell assemblies and can thus be characterized as a sparse phenomenon. Our results indicate that reactivated memory traces may not only comprise feature- and context-specific information but also contain a value component.

Encoding of action history in the rat ventral striatum

In a dynamic environment, animals need to update information about the rewards expected from their alternative actions continually to make optimal choices for its survival. Because the reward resulting from a given action can be substantially delayed, the process of linking a reward to its causative action would be facilitated by memory signals related to the animal's previous actions. Although the ventral striatum has been proposed to play a key role in updating the information about the rewards expected from specific actions, it is not known whether the signals related to previous actions exist in the ventral striatum. In the present study, we recorded neuronal ensemble activity in the rat ventral striatum during a visual discrimination task and investigated whether neuronal activity in the ventral striatum encoded signals related to animal's previous actions. The results show that many neurons modulated their activity according to the animal's goal choice in the previous trial, indicating that memory signals for previous actions are available in the ventral striatum. In contrast, few neurons conveyed signals on impending goal choice of the animal, suggesting the absence of decision signals in the ventral striatum. Memory signals for previous actions might contribute to the process of updating the estimates of rewards expected from alternative actions in the ventral striatum.

The nucleus accumbens and reward: neurophysiological investigations in behaving animals

The nucleus accumbens (Acb) is a crucial component of the brain reward system. This report reviews electrophysiological studies that examined Acb cell firing during goal-directed behaviors for natural reinforcers (food, water, sucrose) and drugs of abuse (cocaine, heroin, ethanol). Studies that examined the role of environmental stimuli and operant contingencies on Acb activity during behavior are also explored. Given the extensive literature that links dopamine in the Acb with drug reinforcement, experiments are considered that examined the influence of dopamine in modulating Acb cell firing during drug-seeking behaviors. Finally, because the Acb is one neural substrate of a larger brain reward circuit, the influence of afferent input (hippocampus and prefrontal cortex) on Acb cell firing during behavior is also discussed. These findings provide a unique insight into the cellular mechanisms underlying reward-related processing and goal-directed behaviors and reveal a level of functional organization in the Acb not identified by other experimental approaches.

Rat Nucleus Accumbens Neurons Persistently Encode Locations Associated With Morphine Reward

When rats and mice are free to explore a familiar environment they spend more time in a previously rewarded location. This conditioned place preference (CPP) results from an increased probability of initiating transitions from an unrewarded location to one previously paired with reward. We recorded nucleus accumbens (NAc) neurons while rats explored a three-room in-line apparatus. Before place conditioning, approximately equal proportions of NAc neurons show excitations or inhibitions when the rat is in each of the rooms (morphine paired, center or saline paired). Conditioning increased the proportion of neurons inhibited while the rat was in the morphine room and neurons excited in the saline room. Many of the neurons in these two groups responded during room transitions. Furthermore, the postconditioning increase in the population of neurons with room-selective responding persisted for several weeks after the last morphine treatment. This long-lasting change in population responses of NAc neurons to initially neutral locations is a neural correlate of the change in location preference manifest as CPP.

39th Annual European Brain and Behaviour Society Abstracts

The EUROPEAN BRAIN AND BEHAVIOUR SOCIETY has held its 39th Annual General Meeting in Trieste, in the campus next to the Miramare castle and its park, co-hosted by SISSA, the International School for Advanced Studies, and ICTP, the Abdus Salam International Centre for Theoretical Physics. Alessandro Treves (SISSA) was the head and inspiration of the Local Organizing committee, supported by P. Battaglini, L. Chelazzi, M. Diamond and G. Vallortigara. All approaches relating brain and behaviour were represented at the meeting, which aimed to further expand the wide spectrum of previous EBBS AGMs, and to bring together integrative, system, cognitive, computational neuroscientists.

See also the societies home page: http://www.ebbs-science.org/.

Space and context in the temporal cortex

The hippocampus has a critical role in certain kinds of spatial memory processes. Hippocampal "place" cells, fire selectively when an animal is in a particular location within the environment. It is thought that this activity underlies a representation of the environment that can be used for memory-based spatial navigation. But how is this representation constructed and how is it "read"? A simple mechanism, based on place field density across an environment, is described that could allow hippocampal representations to be "read" by other brain regions for the purpose of navigation. The possible influence of activity in neighboring brain regions such as the perirhinal cortex, and pre- and para-subiculum on the construction of the hippocampal spatial representation is then discussed.

How prior reward experience biases exploratory movements: a probabilistic model

Animals return to rewarded locations. An example of this is conditioned place preference (CPP), which is widely used in studies of drug reward. Although CPP is expressed as increased time spent in a previously rewarded location, the behavioral strategy underlying this change is unknown. We continuously monitored rats (n = 22) in a three-room in-line configuration, before and after morphine conditioning in one end room. Although sequential room visit durations were variable, their probability distribution was exponential, indicating that the processes controlling visit durations can be modeled by instantaneous room exit probabilities. Further analysis of room transitions and computer simulations of probabilistic models revealed that the exploratory bias toward the morphine room is best explained by an increase in the probability of a subset of rapid, direct transitions from the saline- to the morphine-paired room by the central room. This finding sharply delineates and constrains possible neural mechanisms for a class of self-initiated, goal-directed behaviors toward previously rewarded locations.

Neural representations of location outside the hippocampus

Place cells of the rat hippocampus are a dominant model system for understanding the role of the hippocampus in learning and memory at the level of single-unit and neural ensemble responses. A complete understanding of the information processing and computations performed by the hippocampus requires detailed knowledge about the properties of the representations that are present in hippocampal afferents and efferents in order to decipher the transformations that occur to these representations in the hippocampal circuitry. Neural recordings in behaving rats have revealed a number of brain areas that contain place-related firing properties in the parahippocampal regions and in other brain regions that are thought to interact with the hippocampus in certain behavioral tasks. Although investigators have just begun to scratch the surface in terms of understanding these properties, differences in the precise nature of the spatial firing between the hippocampus and these other regions promise to reveal important clues regarding the exact role of the hippocampus in learning and memory and the nature of its interactions with other brain systems to support adaptive behavior.

The dynamic nature of spatial encoding in the hippocampus

Many hippocampal neurons (place cells) appear to represent a particular location within an environment (their place field). This property would appear to be central to hippocampal involvement in navigation based on spatial memory. Although a navigationally useful representation might also include information about distal goals, having a place field and being able to represent a distal goal would appear to be mutually exclusive place cell properties. Our simulations demonstrate, however, that information about goal direction can be simply derived from the changes in place field density that occur when place fields shift location in a goal-directed manner. Previous reports that place fields respond dynamically to shifts in goal location may, therefore, represent the operation of such a system.

A robot model of the basal ganglia: Behavior and intrinsic processing

The existence of multiple parallel loops connecting sensorimotor systems to the basal ganglia has given rise to proposals that these nuclei serve as a selection mechanism resolving competitions between the alternative actions available in a given context. A strong test of this hypothesis is to require a computational model of the basal ganglia to generate integrated selection sequences in an autonomous agent, we therefore describe a robot architecture into which such a model is embedded, and require it to control action selection in a robotic task inspired by animal observations. Our results demonstrate effective action selection by the embedded model under a wide range of sensory and motivational conditions. When confronted with multiple, high salience alternatives, the robot also exhibits forms of behavioral disintegration that show similarities to animal behavior in conflict situations. The model is shown to cast light on recent neurobiological findings concerning behavioral switching and sequencing.

A study on the role of the dorsal striatum and the nucleus accumbens in allocentric and egocentric spatial memory consolidation

There is now accumulating evidence that the striatal complex in its two major components, the dorsal striatum and the nucleus accumbens, contributes to spatial memory. However, the possibility that different striatal subregions might modulate specific aspects of spatial navigation has not been completely elucidated. Therefore, in this study, two different learning procedures were used to determine whether the two striatal components could be distinguished on the basis of their involvement in spatial learning using different frames of reference: allocentric and egocentric. The task used involved the detection of a spatial change in the configuration of four objects placed in an arena, after the mice had had the opportunity to experience the objects in a constant position for three previous sessions. In the first part of the study we investigated whether changes in the place where the animals were introduced into the arena during habituation and testing could induce a preferential use of an egocentric or an allocentric frame of reference. In the second part of the study we performed focal injections of the N-methyl-d-aspartate (NMDA) receptors' antagonist, AP-5, within the two subregions immediately after training. The results indicate that using the two behavioral procedures, the animals rely on an egocentric and an allocentric spatial frame of reference. Furthermore, they demonstrate that AP-5 (37.5, 75, and 150 ng/side) injections into the dorsal striatum selectively impaired consolidation of spatial information in the egocentric but not in the allocentric procedure. Intra-accumbens AP-5 administration, instead, impaired animals trained using both procedures.

The ventral striatum in off-line processing: ensemble reactivation during sleep and modulation by hippocampal ripples

Previously it has been shown that the hippocampus and neocortex can spontaneously reactivate ensemble activity patterns during post-behavioral sleep and rest periods. Here we examined whether such reactivation also occurs in a subcortical structure, the ventral striatum, which receives a direct input from the hippocampal formation and has been implicated in guidance of consummatory and conditioned behaviors. During a reward-searching task on a T-maze, flanked by sleep and rest periods, parallel recordings were made from ventral striatal ensembles while EEG signals were derived from the hippocampus. Statistical measures indicated a significant amount of reactivation in the ventral striatum. In line with hippocampal data, reactivation was especially prominent during post-behavioral slow-wave sleep, but unlike the hippocampus, no decay in pattern recurrence was visible in the ventral striatum across the first 40 min of post-behavioral rest. We next studied the relationship between ensemble firing patterns in ventral striatum and hippocampal ripples-sharp waves, which have been implicated in pattern replay. Firing rates were significantly modulated in close temporal association with hippocampal ripples in 25% of the units, showing a marked transient enhancement in the average response profile. Strikingly, ripple-modulated neurons in ventral striatum showed a clear reactivation, whereas nonmodulated cells did not. These data suggest, first, the occurrence of pattern replay in a subcortical structure implied in the processing and prediction of reward and, second, a functional linkage between ventral striatal reactivation and a specific type of high-frequency population activity associated with hippocampal replay.

Spatially selective reward site responses in tonically active neurons of the nucleus accumbens in behaving rats

To study how hippocampal output signals conveying spatial and other contextual information might be integrated in the nucleus accumbens, tonically active accumbens neurons were recorded in three unrestrained rats as they performed spatial orientation tasks on an elevated round rotatable platform with four identical reward boxes symmetrically placed around the edge. The partially water-deprived rats were required to shuttle either between the pair of reward boxes indicated by beacon cues (lights in the boxes) or between the pair of boxes occupying particular locations in relation to environmental landmark cues. In 43/82 neurons, behaviorally correlated phasic modulations in discharge activity occurred, primarily prior to or after water was provided at the reward boxes. Twenty-two had inhibitory modulation, 12 excitatory, and nine were mixed excitatory and inhibitory. Although tonically active neurons (TANs) have rarely been reported in the rodent, the inhibitory and mixed responses correspond to previously reports in the macaque accumbens of tonically active neurons with activity correlated with reward delivery and, following conditioning, to sensory stimuli associated with rewards. Eighteen of the 43 tonically active accumbens neurons showed spatial selectivity, i.e., behaviorally correlated increases or decreases in firing rate were of different magnitudes at the respective reward boxes. This is the first demonstration that the configuration of environmental sensory cues associated with reward sites are also an effective stimulus for these neurons and that different neurons are selective for different places. These results are consistent with a role for the nucleus accumbens in the initiation of goal-directed displacement behaviors.

Dynamics of neural coding in the accumbens during extinction and reinstatement of rewarded behavior

Neural correlates of reward-seeking behavior are observed in the nucleus accumbens (NAC). The dependence of these correlates upon the presence of a reward was studied by comparing the behavioral correlates observed when the presence of the reward was manipulated within a single behavioral session. Rats were well-trained on a continuous reinforcement instrumental task reinforced by 0.1 ml drops of 5% sucrose. Extracellular single-unit neural activity was recorded from electrode arrays implanted into the NAC when instrumental behavior was and then was not reinforced with sucrose (within-session extinction). A variable delay between the instrumental response and the sucrose delivery allowed for separation of neural activity related to these task events. A spike activity increase around the time of the instrumental response was the most common behavioral correlate, while a decrease in spike activity upon sucrose delivery was the second most common behavioral correlate. Following removal of the reinforcer, subjects continued to perform the instrumental response, allowing for the examination of response-related spike activity under extinction conditions in which the response was no longer reinforced by sucrose. A majority of the response-related neural activity patterns were lost when sucrose was no longer available. New neural responses also were detected during this period. For some subjects, the reinforcer was again made available during the same session. Encoding of the primary behavioral events during this period of reinstated reinforcer was similar, but not identical, to that observed during the first period of reinforced responding. These findings reveal that instrumental task-associated spike activity within the NAC is partially dependent upon the presence of the reinforcer, and that encoding across the population is distinct under reinforced and extinction conditions.

Putting a spin on the dorsal-ventral divide of the striatum

Since its conception three decades ago, the idea that the striatum consists of a dorsal sensorimotor part and a ventral portion processing limbic information has sparked a quest for functional correlates and anatomical characteristics of the striatal divisions. But this classic dorsal-ventral distinction might not offer the best view of striatal function. Anatomy and neurophysiology show that the two striatal areas have the same basic structure and that sharp boundaries are absent. Behaviorally, a distinction between dorsolateral and ventromedial seems most valid, in accordance with a mediolateral functional zonation imposed on the striatum by its excitatory cortical, thalamic and amygdaloid inputs. Therefore, this review presents a synthesis between the dorsal-ventral distinction and the more mediolateral-oriented functional striatal gradient.

Different representation of forthcoming reward in nucleus accumbens and medial prefrontal cortex

We recorded single units in the rat nucleus accumbens (NAcc) and medial prefrontal cortex (mPFC) to investigate activity related to reward mediation and anticipation during execution of an alternating reward task. NAcc and mPFC neurons showed increased activity differently during an interposed delay preceding reward delivery. Some NAcc neurons increased their activity specifically during the delay period before reward presentation, discriminating forthcoming food or water presentation. A subset of neurons in the mPFC similarly discriminated between food and water reward during the delay; however, about half did not discriminate reward qualities. These results show that the NAcc and the mPFC contribute differently to the organization and execution of goal-directed behavior.

Neurons in hippocampal afferent zones of rat striatum parse routes into multi-pace segments during maze navigation

Hippocampal 'place' neurons discharge when rats occupy specific regions within an environment. This finding is a cornerstone of the theory of the hippocampus as a cognitive map of space. But for navigation, representations of current position must be implemented by signals concerning where to go next, and how to get there. In recordings in hippocampal output structures associated with the motor system (nucleus accumbens and ventromedial caudate nucleus) in rats solving a plus-maze, neurons fired continuously from the moment the rat left one location until it arrived at the next goal site, or at an intermediate place, such as the maze centre. While other studies have shown discharges during reward approach behaviours, this is the first demonstration of activity corresponding to the parsing of complex routes into sequences of movements between landmarks, similar to the lists of instructions we often employ to communicate directions to follow between points on a map. As these cells fired during a series of several paces or re-orientation movements, perhaps this is homologous to 'chunking'. The temporal overlaps in the activity profiles of the individual neurons provide a possible substrate to successively trigger movements required to arrive at the goal. These hippocampally informed, and in some cases, spatially selective responses support the view of the ventral striatum as an interface between limbic and motor systems, permitting contextual representations to have an impact on fundamental action sequences for goal-directed behaviour.

Cue-Evoked Firing of Nucleus Accumbens Neurons Encodes Motivational Significance During a Discriminative Stimulus Task

The nucleus accumbens (NAc) has long been thought of as a limbic-motor interface. Despite behavioral and anatomical evidence in favor of this idea, little is known about how NAc neurons encode information about motivationally relevant environmental cues and use this information to affect motor action. We therefore investigated the firing of these neurons during the performance of a discriminative stimulus (DS) task using simultaneous multiple single-unit recordings in rats. In this task, two stimuli are randomly presented to the animal: a DS, which signals the availability of a sucrose reward contingent on an operant response, and a similar but nonrewarded stimulus (NS). Subpopulations of NAc neurons increased or decreased their firing in association with several distinct components of the task. In this paper, we investigate cue- and operant-responsive neurons. Neurons excited and inhibited by cues showed larger firing changes in response to the DS than the NS and larger changes when the animal made an operant response to the cue than when the animal failed to respond. Excitations during operant responding were not modulated by the information contained by the cue, whereas inhibitions during operant responding were somewhat larger if the operant response occurred during the DS and somewhat smaller if they occurred in the absence of a cue. These results are consistent with the hypothesis that the firing of subpopulations of NAc neurons encode both the predictive value of environmental stimuli and the specific motor behaviors required to respond to them.

Firing of nucleus accumbens neurons during the consummatory phase of a discriminative stimulus task depends on previous reward predictive cues

The nucleus accumbens (NAc) plays an important role in both appetitive and consummatory behavior. To examine how NAc neurons encode information during reward consumption, we recorded the firing activity of rat NAc neurons during the performance of a discriminative stimulus task. In this task, the animal must make an operant response to an intermittently presented cue to obtain a sucrose reward delivered in a reward receptacle. Uncued entries to the receptacle were not rewarded. Both excitations and inhibitions during reward consumption were observed, but substantially more neurons were inhibited than excited. These excitations and inhibitions began when the animal entered the reward receptacle and ended when the animal exited the receptacle. Both excitations and inhibitions were much smaller or nonexistent when the animal made uncued entries into the reward receptacle. In one set of experiments, we randomly withheld the reward in some cued trials that would otherwise have been rewarded. Excitations and inhibitions were of similar magnitude whether or not the reward was delivered. This indicates that the sensory stimulus of reward does not drive these phasic responses; instead, the reward-associated responses may be driven by the conditioned stimuli associated with reward, or they may encode information about consummatory motor activity. Another population of NAc neurons was excited on exit from the reward receptacle. Many of these excitations persisted for tens of seconds after the receptacle exit and showed a significant inverse correlation with the rate of uncued operant responding. These findings are consistent with a contribution of NAc neurons to both reward consummatory and reward seeking behavior.

Tonically active neurons in the primate striatum and their role in the processing of information about motivationally relevant events

Analysis of recordings of single neuronal activity in the striatum of monkeys engaged in behavioural tasks has shown that tonically active neurons (TANs) can be distinguished by their distinct spontaneous firing and functional properties. As TANs are assumed to be cholinergic interneurons, the study of their physiological characteristics allows us to gain an insight into the role of a particular type of local-circuit neuron in the processing of information at the striatal level. In monkeys performing various behavioural tasks, the change in the activity of TANs, unlike the diversity of task-related activations exhibited by the phasically active population of striatal neurons, involves a transient depression of the tonic firing related to environmental events of motivational significance. Such events include primary rewards and stimuli that have acquired a reward value during associative learning. These neurons also respond to an aversive air puff, indicating that their responsiveness is not restricted to appetitive conditions. Another striking feature of the TANs is that their responses can be modulated by predictions about stimulus timing. Temporal variations in event occurrence have been found to favour the responses of TANs, whereas the responses are diminished or abolished in the presence of external cues that predict the time at which events will occur. These data suggest that the TANs respond as do detectors of motivationally relevant events, but they also demonstrate that these neurons are influenced by predictive information based on past experience with a given temporal context. TANs represent a unique subset of striatal neurons that might serve a modulatory function, monitoring for temporal relationships between environmental events.