An integrate-and-fire model of prefrontal cortex neuronal activity during performance of goal-directed decision making

An Integrate-and-Fire Spiking Neural Network Model Simulating Artificially Induced Cortical Plasticity

We describe an integrate-and-fire (IF) spiking neural network that incorporates spike-timing-dependent plasticity (STDP) and simulates the experimental outcomes of four different conditioning protocols that produce cortical plasticity. The original conditioning experiments were performed in freely moving non-human primates (NHPs) with an autonomous head-fixed bidirectional brain-computer interface (BCI). Three protocols involved closed-loop stimulation triggered from (1) spike activity of single cortical neurons, (2) electromyographic (EMG) activity from forearm muscles, and (3) cycles of spontaneous cortical beta activity. A fourth protocol involved open-loop delivery of pairs of stimuli at neighboring cortical sites. The IF network that replicates the experimental results consists of 360 units with simulated membrane potentials produced by synaptic inputs and triggering a spike when reaching threshold. The 240 cortical units produce either excitatory or inhibitory postsynaptic potentials (PSPs) in their target units. In addition to the experimentally observed conditioning effects, the model also allows computation of underlying network behavior not originally documented. Furthermore, the model makes predictions about outcomes from protocols not yet investigated, including spike-triggered inhibition, γ-triggered stimulation and disynaptic conditioning. The success of the simulations suggests that a simple voltage-based IF model incorporating STDP can capture the essential mechanisms mediating targeted plasticity with closed-loop stimulation.

A network model of behavioural performance in a rule learning task

Humans demonstrate differences in performance on cognitive rule learning tasks which could involve differences in properties of neural circuits. An example model is presented to show how gating of the spread of neural activity could underlie rule learning and the generalization of rules to previously unseen stimuli. This model uses the activity of gating units to regulate the pattern of connectivity between neurons responding to sensory input and subsequent gating units or output units. This model allows analysis of network parameters that could contribute to differences in cognitive rule learning. These network parameters include differences in the parameters of synaptic modification and presynaptic inhibition of synaptic transmission that could be regulated by neuromodulatory influences on neural circuits. Neuromodulatory receptors play an important role in cognitive function, as demonstrated by the fact that drugs that block cholinergic muscarinic receptors can cause cognitive impairments. In discussions of the links between neuromodulatory systems and biologically based traits, the issue of mechanisms through which these linkages are realized is often missing. This model demonstrates potential roles of neural circuit parameters regulated by acetylcholine in learning context-dependent rules, and demonstrates the potential contribution of variation in neural circuit properties and neuromodulatory function to individual differences in cognitive function.This article is part of the theme issue 'Diverse perspectives on diversity: multi-disciplinary approaches to taxonomies of individual differences'.

Neural circuits for learning context-dependent associations of stimuli

The use of reinforcement learning combined with neural networks provides a powerful framework for solving certain tasks in engineering and cognitive science. Previous research shows that neural networks have the power to automatically extract features and learn hierarchical decision rules. In this work, we investigate reinforcement learning methods for performing a context-dependent association task using two kinds of neural network models (using continuous firing rate neurons), as well as a neural circuit gating model. The task allows examination of the ability of different models to extract hierarchical decision rules and generalize beyond the examples presented to the models in the training phase. We find that the simple neural circuit gating model, trained using response-based regulation of Hebbian associations, performs almost at the same level as a reinforcement learning algorithm combined with neural networks trained with more sophisticated back-propagation of error methods. A potential explanation is that hierarchical reasoning is the key to performance and the specific learning method is less important.

Goal-Directed Decision Making with Spiking Neurons

Behavioral and neuroscientific data on reward-based decision making point to a fundamental distinction between habitual and goal-directed action selection. The formation of habits, which requires simple updating of cached values, has been studied in great detail, and the reward prediction error theory of dopamine function has enjoyed prominent success in accounting for its neural bases. In contrast, the neural circuit mechanisms of goal-directed decision making, requiring extended iterative computations to estimate values online, are still unknown. Here we present a spiking neural network that provably solves the difficult online value estimation problem underlying goal-directed decision making in a near-optimal way and reproduces behavioral as well as neurophysiological experimental data on tasks ranging from simple binary choice to sequential decision making. Our model uses local plasticity rules to learn the synaptic weights of a simple neural network to achieve optimal performance and solves one-step decision-making tasks, commonly considered in neuroeconomics, as well as more challenging sequential decision-making tasks within 1 s. These decision times, and their parametric dependence on task parameters, as well as the final choice probabilities match behavioral data, whereas the evolution of neural activities in the network closely mimics neural responses recorded in frontal cortices during the execution of such tasks. Our theory provides a principled framework to understand the neural underpinning of goal-directed decision making and makes novel predictions for sequential decision-making tasks with multiple rewards.

SIGNIFICANCE STATEMENT Goal-directed actions requiring prospective planning pervade decision making, but their circuit-level mechanisms remain elusive. We show how a model circuit of biologically realistic spiking neurons can solve this computationally challenging problem in a novel way. The synaptic weights of our network can be learned using local plasticity rules such that its dynamics devise a near-optimal plan of action. By systematically comparing our model results to experimental data, we show that it reproduces behavioral decision times and choice probabilities as well as neural responses in a rich set of tasks. Our results thus offer the first biologically realistic account for complex goal-directed decision making at a computational, algorithmic, and implementational level.

A model of hippocampal spiking responses to items during learning of a context-dependent task

Single unit recordings in the rat hippocampus have demonstrated shifts in the specificity of spiking activity during learning of a contextual item-reward association task. In this task, rats received reward for responding to different items dependent upon the context an item appeared in, but not dependent upon the location an item appears at. Initially, neurons in the rat hippocampus primarily show firing based on place, but as the rat learns the task this firing became more selective for items. We simulated this effect using a simple circuit model with discrete inputs driving spiking activity representing place and item followed sequentially by a discrete representation of the motor actions involving a response to an item (digging for food) or the movement to a different item (movement to a different pot for food). We implemented spiking replay in the network representing neural activity observed during sharp-wave ripple events, and modified synaptic connections based on a simple representation of spike-timing dependent synaptic plasticity. This simple network was able to consistently learn the context-dependent responses, and transitioned from dominant coding of place to a gradual increase in specificity to items consistent with analysis of the experimental data. In addition, the model showed an increase in specificity toward context. The increase of selectivity in the model is accompanied by an increase in binariness of the synaptic weights for cells that are part of the functional network.

Deep belief networks learn context dependent behavior

With the goal of understanding behavioral mechanisms of generalization, we analyzed the ability of neural networks to generalize across context. We modeled a behavioral task where the correct responses to a set of specific sensory stimuli varied systematically across different contexts. The correct response depended on the stimulus (A,B,C,D) and context quadrant (1,2,3,4). The possible 16 stimulus-context combinations were associated with one of two responses (X,Y), one of which was correct for half of the combinations. The correct responses varied symmetrically across contexts. This allowed responses to previously unseen stimuli (probe stimuli) to be generalized from stimuli that had been presented previously. By testing the simulation on two or more stimuli that the network had never seen in a particular context, we could test whether the correct response on the novel stimuli could be generated based on knowledge of the correct responses in other contexts. We tested this generalization capability with a Deep Belief Network (DBN), Multi-Layer Perceptron (MLP) network, and the combination of a DBN with a linear perceptron (LP). Overall, the combination of the DBN and LP had the highest success rate for generalization.

A wavelet-based neural model to optimize and read out a temporal population code

It has been proposed that the dense excitatory local connectivity of the neo-cortex plays a specific role in the transformation of spatial stimulus information into a temporal representation or a temporal population code (TPC). TPC provides for a rapid, robust, and high-capacity encoding of salient stimulus features with respect to position, rotation, and distortion. The TPC hypothesis gives a functional interpretation to a core feature of the cortical anatomy: its dense local and sparse long-range connectivity. Thus far, the question of how the TPC encoding can be decoded in downstream areas has not been addressed. Here, we present a neural circuit that decodes the spectral properties of the TPC using a biologically plausible implementation of a Haar transform. We perform a systematic investigation of our model in a recognition task using a standardized stimulus set. We consider alternative implementations using either regular spiking or bursting neurons and a range of spectral bands. Our results show that our wavelet readout circuit provides for the robust decoding of the TPC and further compresses the code without loosing speed or quality of decoding. We show that in the TPC signal the relevant stimulus information is present in the frequencies around 100 Hz. Our results show that the TPC is constructed around a small number of coding components that can be well decoded by wavelet coefficients in a neuronal implementation. The solution to the TPC decoding problem proposed here suggests that cortical processing streams might well consist of sequential operations where spatio-temporal transformations at lower levels forming a compact stimulus encoding using TPC that are subsequently decoded back to a spatial representation using wavelet transforms. In addition, the results presented here show that different properties of the stimulus might be transmitted to further processing stages using different frequency components that are captured by appropriately tuned wavelet-based decoders.

Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates

Recent work has given rise to the view that reward-based decision making is governed by two key controllers: a habit system, which stores stimulus-response associations shaped by past reward, and a goal-oriented system that selects actions based on their anticipated outcomes. The current literature provides a rich body of computational theory addressing habit formation, centering on temporal-difference learning mechanisms. Less progress has been made toward formalizing the processes involved in goal-directed decision making. We draw on recent work in cognitive neuroscience, animal conditioning, cognitive and developmental psychology, and machine learning to outline a new theory of goal-directed decision making. Our basic proposal is that the brain, within an identifiable network of cortical and subcortical structures, implements a probabilistic generative model of reward, and that goal-directed decision making is effected through Bayesian inversion of this model. We present a set of simulations implementing the account, which address benchmark behavioral and neuroscientific findings, and give rise to a set of testable predictions. We also discuss the relationship between the proposed framework and other models of decision making, including recent models of perceptual choice, to which our theory bears a direct connection.

Spatial learning and action planning in a prefrontal cortical network model

The interplay between hippocampus and prefrontal cortex (PFC) is fundamental to spatial cognition. Complementing hippocampal place coding, prefrontal representations provide more abstract and hierarchically organized memories suitable for decision making. We model a prefrontal network mediating distributed information processing for spatial learning and action planning. Specific connectivity and synaptic adaptation principles shape the recurrent dynamics of the network arranged in cortical minicolumns. We show how the PFC columnar organization is suitable for learning sparse topological-metrical representations from redundant hippocampal inputs. The recurrent nature of the network supports multilevel spatial processing, allowing structural features of the environment to be encoded. An activation diffusion mechanism spreads the neural activity through the column population leading to trajectory planning. The model provides a functional framework for interpreting the activity of PFC neurons recorded during navigation tasks. We illustrate the link from single unit activity to behavioral responses. The results suggest plausible neural mechanisms subserving the cognitive “insight” capability originally attributed to rodents by Tolman & Honzik. Our time course analysis of neural responses shows how the interaction between hippocampus and PFC can yield the encoding of manifold information pertinent to spatial planning, including prospective coding and distance-to-goal correlates.

We study spatial cognition, a high-level brain function based upon the ability to elaborate mental representations of the environment supporting goal-oriented navigation. Spatial cognition involves parallel information processing across a distributed network of interrelated brain regions. Depending on the complexity of the spatial navigation task, different neural circuits may be primarily involved, corresponding to different behavioral strategies. Navigation planning, one of the most flexible strategies, is based on the ability to prospectively evaluate alternative sequences of actions in order to infer optimal trajectories to a goal. The hippocampal formation and the prefrontal cortex are two neural substrates likely involved in navigation planning. We adopt a computational modeling approach to show how the interactions between these two brain areas may lead to learning of topological representations suitable to mediate action planning. Our model suggests plausible neural mechanisms subserving the cognitive spatial capabilities attributed to rodents. We provide a functional framework for interpreting the activity of prefrontal and hippocampal neurons recorded during navigation tasks. Akin to integrative neuroscience approaches, we illustrate the link from single unit activity to behavioral responses while solving spatial learning tasks.

Mechanisms Gating the Flow of Information in the Cortex: What They Might Look Like and What Their Uses may be

The notion of gating as a mechanism capable of controlling the flow of information from one set of neurons to another, has been studied in many regions of the central nervous system. In the nucleus accumbens, where evidence is especially clear, gating seems to rely on the action of bistable neurons, i.e., of neurons that oscillate between a quiescent "down" state and a firing "up" state, and that act as AND-gates relative to their entries. Independently from these observations, a growing body of evidence now indicates that bistable neurons are also quite abundant in the cortex, although their exact functions in the dynamics of the brain remain to be determined. Here, we propose that at least some of these bistable cortical neurons are part of circuits devoted to gating information flow within the cortex. We also suggest that currently available structural, electrophysiological, and imaging data support the existence of at least three different types of gating architectures. The first architecture involves gating directly by the cortex itself. The second architecture features circuits spanning the cortex and the thalamus. The third architecture extends itself through the cortex, the basal ganglia, and the thalamus. These propositions highlight the variety of mechanisms that could regulate the passage of action potentials between cortical neurons sets. They also suggest that gating mechanisms require larger-scale neural circuitry to control the state of the gates themselves, in order to fit in the overall wiring of the brain and complement its dynamics.

A biologically based model for the integration of sensory-motor contingencies in rules and plans: a prefrontal cortex based extension of the Distributed Adaptive Control architecture

Intelligence depends on the ability of the brain to acquire and apply rules and representations. At the neuronal level these properties have been shown to critically depend on the prefrontal cortex. Here we present, in the context of the Distributed Adaptive Control architecture (DAC), a biologically based model for flexible control and planning based on key physiological properties of the prefrontal cortex, i.e. reward modulated sustained activity and plasticity of lateral connectivity. We test the model in a series of pertinent tasks, including multiple T-mazes and the Tower of London that are standard experimental tasks to assess flexible control and planning. We show that the model is both able to acquire and express rules that capture the properties of the task and to quickly adapt to changes. Further, we demonstrate that this biomimetic self-contained cognitive architecture generalizes to planning. In addition, we analyze the extended DAC architecture, called DAC 6, as a model that can be applied for the creation of intelligent and psychologically believable synthetic agents.

Working Memory Performance Correlates with Prefrontal-Hippocampal Theta Interactions but not with Prefrontal Neuron Firing Rates

Performance of memory tasks is impaired by lesions to either the medial prefrontal cortex (mPFC) or the hippocampus (HPC); although how these two areas contribute to successful performance is not well understood. mPFC unit activity is temporally affected by hippocampal-theta oscillations, with almost half the mPFC population entrained to theta in behaving animals, pointing to theta interactions as the mechanism enabling collaborations between these two areas. mPFC neurons respond to sensory stimuli and responses in working memory tasks, though the function of these correlated firing rate changes remains unclear because similar responses are reported during mPFC dependent and independent tasks. Using a DNMS task we compared error trials vs. correct trials and found almost all mPFC cells fired at similar rates during both error and correct trials (92%), however theta-entrainment of mPFC neurons declined during error performance as only 17% of cells were theta-entrained (during correct trials 46% of the population was theta-entrained). Across the population, error and correct trials did not differ in firing rate, but theta-entrainment was impaired. Periods of theta-entrainment and firing rate changes appeared to be independent variables, and only theta-entrainment was correlated with successful performance, indicating mPFC-HPC theta-range interactions are the key to successful DNMS performance.

Cooperative activity of neurons in the nucleus accumbens and frontal cortex in cats trained to select reinforcements of different value

Results obtained at the level of the organization of interneuronal interactions of cells in the nucleus accumbens and frontal cortex revealed the features of the involvement of this component in "impulsive" and "self-controlled" behavior, consisting of an increase in bidirectional interactions between the structures of interest, accompanied by simultaneous reductions in the regularity of interactions with increases in "impulsivity" and decreases in "self-control." Long-latency reactions appearing only in "impulsive" animals were associated with decreases in the control of frontal cortex cells by the nucleus accumbens during the signal period, which correlated with the low activity of the network activity of the nucleus accumbens in these animals. Comparison of the patterns of frontal-accumbens interactions as the animals performed a single type of activity demonstrated that the connections in neuron pairs during the presignal and signal periods were similar, while significant differences in patterns were seen during the performance of different types of activity.

Shaping embodied neural networks for adaptive goal-directed behavior

The acts of learning and memory are thought to emerge from the modifications of synaptic connections between neurons, as guided by sensory feedback during behavior. However, much is unknown about how such synaptic processes can sculpt and are sculpted by neuronal population dynamics and an interaction with the environment. Here, we embodied a simulated network, inspired by dissociated cortical neuronal cultures, with an artificial animal (an animat) through a sensory-motor loop consisting of structured stimuli, detailed activity metrics incorporating spatial information, and an adaptive training algorithm that takes advantage of spike timing dependent plasticity. By using our design, we demonstrated that the network was capable of learning associations between multiple sensory inputs and motor outputs, and the animat was able to adapt to a new sensory mapping to restore its goal behavior: move toward and stay within a user-defined area. We further showed that successful learning required proper selections of stimuli to encode sensory inputs and a variety of training stimuli with adaptive selection contingent on the animat's behavior. We also found that an individual network had the flexibility to achieve different multi-task goals, and the same goal behavior could be exhibited with different sets of network synaptic strengths. While lacking the characteristic layered structure of in vivo cortical tissue, the biologically inspired simulated networks could tune their activity in behaviorally relevant manners, demonstrating that leaky integrate-and-fire neural networks have an innate ability to process information. This closed-loop hybrid system is a useful tool to study the network properties intermediating synaptic plasticity and behavioral adaptation. The training algorithm provides a stepping stone towards designing future control systems, whether with artificial neural networks or biological animats themselves.

The ability of a brain to learn has been studied at various levels. However, a large gap exists between behavioral studies of learning and memory and studies of cellular plasticity. In particular, much remains unknown about how cellular plasticity scales to affect network population dynamics. In previous studies, we have addressed this by growing mammalian brain cells in culture and creating a long-term, two-way interface between a cultured network and a robot or an artificial animal. Behavior and learning could now be observed in concert with the detailed and long-term electrophysiology. In this work, we used modeling/simulation of living cortical cultures to investigate the network's capability to learn goal-directed behavior. A biologically inspired simulated network was used to determine an effective closed-loop training algorithm, and the system successfully exhibited multi-task goal-directed adaptive behavior. The results suggest that even though lacking the characteristic layered structure of a brain, the network still could be functionally shaped and showed meaningful behavior. Knowledge gained from working with such closed-loop systems could influence the design of future artificial neural networks, more effective neuroprosthetics, and even the use of living networks themselves as a biologically based control system.

Modeling the role of working memory and episodic memory in behavioral tasks

The mechanisms of goal-directed behavior have been studied using reinforcement learning theory, but these theoretical techniques have not often been used to address the role of memory systems in performing behavioral tasks. This work addresses this shortcoming by providing a way in which working memory (WM) and episodic memory may be included in the reinforcement learning framework, then simulating the successful acquisition and performance of six behavioral tasks, drawn from or inspired by the rat experimental literature, that require WM or episodic memory for correct performance. With no delay imposed during the tasks, simulations with WM can solve all of the tasks at above the chance level. When a delay is imposed, simulations with both episodic memory and WM can solve all of the tasks except a disambiguation of odor sequences task.

Dynamical model of salience gated working memory, action selection and reinforcement based on basal ganglia and dopamine feedback

A simple working memory model based on recurrent network activation is proposed and its application to selection and reinforcement of an action is demonstrated as a solution to the temporal credit assignment problem. Reactivation of recent salient cue states is generated and maintained as a type of salience gated recurrently active working memory, while lower salience distractors are ignored. Cue reactivation during the action selection period allows the cue to select an action while its reactivation at the reward period allows the reinforcement of the action selected by the reactivated state, which is necessarily the action which led to the reward being found. A down-gating of the external input during the reactivation and maintenance prevents interference. A double winner-take-all system which selects only one cue and only one action allows the targeting of the cue-action allocation to be modified. This targeting works both to reinforce a correct cue-action allocation and to punish the allocation when cue-action allocations change. Here we suggest a firing rate neural network implementation of this system based on the basal ganglia anatomy with input from a cortical association layer where reactivations are generated by signals from the thalamus. Striatum medium spiny neurons represent actions. Auto-catalytic feedback from a dopamine reward signal modulates three-way Hebbian long term potentiation and depression at the cortical-striatal synapses which represent the cue-action associations. The model is illustrated by the numerical simulations of a simple example--that of associating a cue signal to a correct action to obtain reward after a delay period, typical of primate cue reward tasks. Through learning, the model shows a transition from an exploratory phase where actions are generated randomly, to a stable directed phase where the animal always chooses the correct action for each experienced state. When cue-action allocations change, we show that this is noticed by the model, the incorrect cue-action allocations are punished and the correct ones discovered.

Consequences of parameter differences in a model of short-term persistent spiking buffers provided by pyramidal cells in entorhinal cortex

In previous simulations of hippocampus-dependent and prefrontal cortex-dependent tasks, we demonstrated the use of one-shot short-term buffering with time compression that may be achieved through persistent spiking activity during theta rhythm. A biophysically plausible implementation of such a first-in first-out buffer of short sequences of spike patterns includes noise and differences between the parameter values of individual model pyramidal cells. We show that a specific set of parameters determines model buffer capacity and buffer function, and individual differences can have consequences similar to those of noise. The set of parameters includes the frequency of network theta rhythm and the strength of recurrent inhibition (affecting capacity), as well as the time constants of the characteristic after-depolarizing response and the phase of afferent input during theta rhythm (affecting buffer function). Given a sufficient number of pyramidal cells in layer II of entorhinal cortex, and in each self-selected category of pyramidal cells with similar model parameters, buffer function within a category is reliable with category-specific properties. Properties include buffering of spikes in the order of inputs or in the reversed order. Multiple property sets may enable parallel buffers with different capacities, which may underlie differences of place field sizes and may interact with grid cell firing in a separate population of layer II stellate cells in the entorhinal cortex.

Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling

In Pavlovian and instrumental conditioning, reward typically comes seconds after reward-triggering actions, creating an explanatory conundrum known as "distal reward problem": How does the brain know what firing patterns of what neurons are responsible for the reward if 1) the patterns are no longer there when the reward arrives and 2) all neurons and synapses are active during the waiting period to the reward? Here, we show how the conundrum is resolved by a model network of cortical spiking neurons with spike-timing-dependent plasticity (STDP) modulated by dopamine (DA). Although STDP is triggered by nearly coincident firing patterns on a millisecond timescale, slow kinetics of subsequent synaptic plasticity is sensitive to changes in the extracellular DA concentration during the critical period of a few seconds. Random firings during the waiting period to the reward do not affect STDP and hence make the network insensitive to the ongoing activity-the key feature that distinguishes our approach from previous theoretical studies, which implicitly assume that the network be quiet during the waiting period or that the patterns be preserved until the reward arrives. This study emphasizes the importance of precise firing patterns in brain dynamics and suggests how a global diffusive reinforcement signal in the form of extracellular DA can selectively influence the right synapses at the right time.

Directional signals in the prefrontal cortex and in area MT during a working memory for visual motion task

Neurons in the middle temporal visual area (MT) have been implicated in the perception of visual motion, whereas prefrontal cortex (PFC) neurons have been linked to temporary storage of sensory signals, attentional and executive control of behavior. Using a task that placed demands on both sets of neurons, we investigated their contribution to working memory for visual motion. Monkeys compared the direction of two moving random-dot stimuli, sample and test, separated by a brief memory delay. Neurons in both areas showed robust direction-selective activity during all phases of the task. During the sample, approximately 60% of task-related PFC neurons were direction selective, and this selectivity emerged 40 ms later than in MT. Unlike MT, the PFC responses to sample did not correlate with behavioral choices, but their selectivity was modulated by task demands and diminished on error trials. Reliable directional signals were found in both areas during the memory delay, but these signals were transient rather than sustained by neurons of either area. Responses to the test in both areas were modulated by the remembered sample direction, decreasing when the test direction matched the sample. This decrease arose in the PFC 100 ms later than in MT and was predictive of the forthcoming decision. Our data suggest that neurons in the two regions are functionally connected and make unique contributions to different task components. PFC neurons reflect task-related information about visual motion and represent decisions that may be based, in part, on the comparison in MT between the remembered sample and test.

First-In-First-Out Item Replacement in a Model of Short-Term Memory Based on Persistent Spiking

Persistent neuronal firing has been modeled in relation to observed brain rhythms, especially to theta oscillations recorded in behaving animals. Models of short-term memory that are based on such persistent firing properties of specific neurons can meet the requirements of spike-timing-dependent potentiation of synaptic strengths during the encoding of a temporal sequence of spike patterns. We show that such a spiking buffer can be simulated with integrate-and-fire neurons that include a leak current even when different numbers of spikes represent successive items. We propose a mechanism that successfully replaces items in the buffer in first-in-first-out (FIFO) order when the distribution of spike density in a theta cycle is asymmetric, as found in experimental data. We predict effects on the function and capacity of the buffer model caused by changes in modeled theta cycle duration, the timing of input to the buffer, the strength of recurrent inhibition, and the strength and timing of after-hyperpolarization and after-depolarization (ADP). Shifts of input timing or changes in ADP parameters can enable the reverse-order buffering of items, with FIFO replacement in a full buffer. As noise increases, the simulated buffer provides robust output that may underlie episodic encoding.

Computational algorithms and neuronal network models underlying decision processes

Animals or humans often encounter such situations in which they must choose their behavioral responses to be made in the near or distant future. Such a decision is made through continuous and bidirectional interactions between the environment surrounding the brain and its internal state or dynamical processes. Therefore, decision making may provide a unique field of researches for studying information processing by the brain, a biological system open to information exchanges with the external world. To make a decision, the brain must analyze pieces of information given externally, past experiences in a similar situation, possible behavioral responses, and predicted outcomes of the individual responses. In this article, we review results of recent experimental and theoretical studies of neuronal substrates and computational algorithms for decision processes.

Representation of Spatial Goals in Rat Orbitofrontal Cortex

The orbitofrontal cortex (OFC) is thought to participate in making and evaluating goal-directed decisions. In rodents, spatial navigation is a major mode of goal-directed behavior, and anatomical and lesion studies implicate the OFC in spatial processing, but there is little direct evidence for coding of spatial or motor variables. Here, we recorded from ventrolateral and lateral OFC in an odor-cued two-alternative choice task requiring orientation and approach to spatial goal ports. In this context, over half of OFC neurons encoded choice direction or goal port location. A subset of neurons was jointly selective for the trial outcome and port location, information useful for the selection or evaluation of spatial goals. These observations show that the rodent OFC not only encodes information relating to general motivational significance, as shown previously, but also encodes spatiomotor variables needed to define specific behavioral goals and the locomotor actions required to attain them.

A model of prefrontal cortical mechanisms for goal-directed behavior

Many behavioral tasks require goal-directed actions to obtain delayed reward. The prefrontal cortex appears to mediate many aspects of goal-directed decision making. This article presents a model of prefrontal cortex function emphasizing the influence of goal-related activity on the choice of the next motor output. The model can be interpreted in terms of key elements of Reinforcement Learning Theory. Different neocortical minicolumns represent distinct sensory input states and distinct motor output actions. The dynamics of each minicolumn include separate phases of encoding and retrieval. During encoding, strengthening of excitatory connections forms forward and reverse associations between each state, the following action, and a subsequent state, which may include reward. During retrieval, activity spreads from reward states throughout the network. The interaction of this spreading activity with a specific input state directs selection of the next appropriate action. Simulations demonstrate how these mechanisms can guide performance in a range of goal-directed tasks, and provide a functional framework for some of the neuronal responses previously observed in the medial prefrontal cortex during performance of spatial memory tasks in rats.