Agents of the mind

Statistical mechanics of structural and temporal credit assignment effects on learning in neural networks

Neural networks can learn flexible input-output associations by changing their synaptic weights. The representational performance and learning dynamics of neural networks are intensively studied in several fields. Neural networks face the "credit assignment problem" in situations in which only incomplete performance evaluations are available. The credit assignment problem is that a network should assign credit or blame for its behaviors according to the contribution to the network performance. In reinforcement learning, a scalar evaluation signal is delivered to a network. The two main types of credit assignment problems in reinforcement learning are structural and temporal, that is, which parameters of the network (structural) and which past network activities (temporal) are related to an evaluation signal given from an environment. In this study, we apply statistical mechanical analysis to the learning processes in a simple neural network model to clarify the effects of two kinds of credit assignments and their interactions. Our model is based on node perturbation learning with eligibility trace. Node perturbation is a stochastic gradient learning method that can solve structural credit assignment problems by introducing a perturbation into the system output. The eligibility trace preserves the past network activities with a temporal credit to deal with the delay of an instruction signal. We show that both credit assignment effects mutually interact and the optimal time constant of the eligibility trace varies not only for the evaluation delay but also the network size.

Adaptation, expertise, and giftedness: towards an understanding of cortical, subcortical, and cerebellar network contributions

Current cortico-centric models of cognition lack a cohesive neuroanatomic framework that sufficiently considers overlapping levels of function, from "pathological" through "normal" to "gifted" or exceptional ability. While most cognitive theories presume an evolutionary context, few actively consider the process of adaptation, including concepts of neurodevelopment. Further, the frequent co-occurrence of "gifted" and "pathological" function is difficult to explain from a cortico-centric point of view. This comprehensive review paper proposes a framework that includes the brain's vertical organization and considers "giftedness" from an evolutionary and neurodevelopmental vantage point. We begin by discussing the current cortico-centric model of cognition and its relationship to intelligence. We then review an integrated, dual-tiered model of cognition that better explains the process of adaptation by simultaneously allowing for both stimulus-based processing and higher-order cognitive control. We consider the role of the basal ganglia within this model, particularly in relation to reward circuitry and instrumental learning. We review the important role of white matter tracts in relation to speed of adaptation and development of behavioral mastery. We examine the cerebellum's critical role in behavioral refinement and in cognitive and behavioral automation, particularly in relation to expertise and giftedness. We conclude this integrated model of brain function by considering the savant syndrome, which we believe is best understood within the context of a dual-tiered model of cognition that allows for automaticity in adaptation as well as higher-order executive control.

Differential effects of dopaminergic therapies on dorsal and ventral striatum in Parkinson's disease: implications for cognitive function

Cognitive abnormalities are a feature of Parkinson's disease (PD). Unlike motor symptoms that are clearly improved by dopaminergic therapy, the effect of dopamine replacement on cognition seems paradoxical. Some cognitive functions are improved whereas others are unaltered or even hindered. Our aim was to understand the effect of dopamine replacement therapy on various aspects of cognition. Whereas dorsal striatum receives dopamine input from the substantia nigra (SN), ventral striatum is innervated by dopamine-producing cells in the ventral tegmental area (VTA). In PD, degeneration of SN is substantially greater than cell loss in VTA and hence dopamine-deficiency is significantly greater in dorsal compared to ventral striatum. We suggest that dopamine supplementation improves functions mediated by dorsal striatum and impairs, or heightens to a pathological degree, operations ascribed to ventral striatum. We consider the extant literature in light of this principle. We also survey the effect of dopamine replacement on functional neuroimaging in PD relating the findings to this framework. This paper highlights the fact that currently, titration of therapy in PD is geared to optimizing dorsal striatum-mediated motor symptoms, at the expense of ventral striatum operations. Increased awareness of contrasting effects of dopamine replacement on dorsal versus ventral striatum functions will lead clinicians to survey a broader range of symptoms in determining optimal therapy, taking into account both those aspects of cognition that will be helped versus those that will be hindered by dopaminergic treatment.

Cortico-basal ganglia circuitry: a review of key research and implications for functional connectivity studies of mood and anxiety disorders

There is considerable evidence that dysfunction of the cortico-basal ganglia circuits may be associated with several mood and anxiety disorders. However, it is unclear whether circuit abnormalities contribute directly either to the neurobiology of these conditions or to the manifestation of symptoms. Understanding the role of these pathways in psychiatric illness has been limited by an incomplete characterization of normal function. In recent years, studies using animal models and human functional imaging have greatly expanded the literature describing normal cortico-basal ganglia circuit function. In this paper, recent key studies of circuit function using human and animal models are reviewed and integrated with findings from other studies conducted over the previous decades. The literature suggests several hypotheses of cortico-basal ganglia circuitry function in mood and anxiety disorders that warrant further exploration. Hypotheses are proposed herein based upon the cortico-basal ganglia mechanisms of: (1) feedforward and feedback control, (2) circuit integration and (3) emotional control. These are presented as models of circuit function, which may be particularly relevant to future investigations using neuroimaging and functional connectivity analyses.

A neural model of hippocampal-striatal interactions in associative learning and transfer generalization in various neurological and psychiatric patients

Building on our previous neurocomputational models of basal ganglia and hippocampal region function (and their modulation by dopamine and acetylcholine, respectively), we show here how an integration of these models can inform our understanding of the interaction between the basal ganglia and hippocampal region in associative learning and transfer generalization across various patient populations. As a common test bed for exploring interactions between these brain regions and neuromodulators, we focus on the acquired equivalence task, an associative learning paradigm in which stimuli that have been associated with the same outcome acquire a functional similarity such that subsequent generalization between these stimuli increases. This task has been used to test cognitive dysfunction in various patient populations with damages to the hippocampal region and basal ganglia, including studies of patients with Parkinson's disease (PD), schizophrenia, basal forebrain amnesia, and hippocampal atrophy. Simulation results show that damage to the hippocampal region-as in patients with hippocampal atrophy (HA), hypoxia, mild Alzheimer's (AD), or schizophrenia-leads to intact associative learning but impaired transfer generalization performance. Moreover, the model demonstrates how PD and anterior communicating artery (ACoA) aneurysm-two very different brain disorders that affect different neural mechanisms-can have similar effects on acquired equivalence performance. In particular, the model shows that simulating a loss of dopamine function in the basal ganglia module (as in PD) leads to slow acquisition learning but intact transfer generalization. Similarly, the model shows that simulating the loss of acetylcholine in the hippocampal region (as in ACoA aneurysm) also results in slower acquisition learning. We argue from this that changes in associative learning of stimulus-action pathways (in the basal ganglia) or changes in the learning of stimulus representations (in the hippocampal region) can have similar functional effects.

The basal ganglia communicate with the cerebellum

The basal ganglia and cerebellum are major subcortical structures that influence not only movement, but putatively also cognition and affect. Both structures receive input from and send output to the cerebral cortex. Thus, the basal ganglia and cerebellum form multisynaptic loops with the cerebral cortex. Basal ganglia and cerebellar loops have been assumed to be anatomically separate and to perform distinct functional operations. We investigated whether there is any direct route for basal ganglia output to influence cerebellar function that is independent of the cerebral cortex. We injected rabies virus (RV) into selected regions of the cerebellar cortex in cebus monkeys and used retrograde transneuronal transport of the virus to determine the origin of multisynaptic inputs to the injection sites. We found that the subthalamic nucleus of the basal ganglia has a substantial disynaptic projection to the cerebellar cortex. This pathway provides a means for both normal and abnormal signals from the basal ganglia to influence cerebellar function. We previously showed that the dentate nucleus of the cerebellum has a disynaptic projection to an input stage of basal ganglia processing, the striatum. Taken together these results provide the anatomical substrate for substantial two-way communication between the basal ganglia and cerebellum. Thus, the two subcortical structures may be linked together to form an integrated functional network.

Conditional routing of information to the cortex: a model of the basal ganglia's role in cognitive coordination

The basal ganglia play a central role in cognition and are involved in such general functions as action selection and reinforcement learning. Here, we present a model exploring the hypothesis that the basal ganglia implement a conditional information-routing system. The system directs the transmission of cortical signals between pairs of regions by manipulating separately the selection of sources and destinations of information transfers. We suggest that such a mechanism provides an account for several cognitive functions of the basal ganglia. The model also incorporates a possible mechanism by which subsequent transfers of information control the release of dopamine. This signal is used to produce novel stimulus-response associations by internalizing transferred cortical representations in the striatum. We discuss how the model is related to production systems and cognitive architectures. A series of simulations is presented to illustrate how the model can perform simple stimulus-response tasks, develop automatic behaviors, and provide an account of impairments in Parkinson's and Huntington's diseases.

Speaking words in two languages with one brain: neural overlap and dissociation

The present study investigated the neural overlap and dissociation underlying overt word production in the first language (L1) and second language (L2). Twenty-four Chinese-English bilinguals named pictures in either L1 or L2 while being scanned with functional magnetic resonance imaging (fMRI). When comparing picture naming in L2 to naming in L1, increased activity in the left inferior frontal gyrus, bilateral supplementary motor areas (SMA), left precentral gyrus, left lingual gyrus, left cuneus, bilateral putamen, bilateral globus pallidus, bilateral caudate and bilateral cerebellum were observed. This suggested that word production in L2 is less automatic and needs to recruit more neural resources for lexical retrieval, articulatory processing and cognitive control than in L1. In contrast, picture naming in L1 relative to picture naming in L2 revealed increased activity in the right putamen and right globus pallidus probably due to different phonological features between Chinese and English. In addition, the conjunction analysis, for the first time, revealed the common neural correlates underlying picture naming in L1 and L2.

The cerebellum and language: evidence from patients with cerebellar degeneration

Clinical and imaging studies suggest that the cerebellum is involved in language tasks, but the extent to which slowed language production in cerebellar patients contributes to their poor performance on these tasks is not clear. We explored this relationship in 18 patients with cerebellar degeneration and 16 healthy controls who completed measures of verbal fluency (phonemic and semantic), word stem completion, and oral naming speed. Cerebellar patients showed significantly slower response times when naming common nouns, which correlated with their degree of motor impairment. Patients were significantly impaired on both phonemic and semantic fluency measures compared to controls (p<0.001), even when naming speed was entered as a covariate (p=0.03). On the word stem completion task, patients were significantly less accurate (p<0.001), had more errors due to non-responses (p=0.008), and were slower to respond (p=0.036) than controls; group effects were significant for overall accuracy, but not response time, when the effects of naming speed were covaried (p=0.014). These findings suggest that cerebellar patients' poorer performance on language tasks cannot be explained solely by slower language production, and that the integrity of cerebellar-prefrontal loops might underlie poorer performance on measures of executive function in cerebellar patients.

Basal ganglia contribution to the initiation of corrective submovements

We investigated the neural processes, with a focus on subcortical circuits, which govern corrective submovements in visually targeted action. During event-related fMRI, subjects moved a cursor to capture targets presented at varying movement amplitudes. Movements were performed in a rehearsed null and a novel viscous (25% random trials) torque field. Movement error feedback was provided after each trial. The viscous field invoked a significantly larger error at the end of the primary movement. Subjects compensated by producing more corrections than they had in the null condition. Corrective submovements were appropriately scaled such that terminal error was similar between the two conditions. Parametric analysis identified two regions where the BOLD signal correlated with the number of submovements per trial: a cerebellar region similar to the one noted in the task contrast and the contralateral dorsal putamen. A separate parametric analysis identified brain regions where activity correlated with movement amplitude. This identified the same cerebellar region as above, bilateral parietal cortex, and left motor and premotor cortex. Our data indicate that the basal ganglia and cerebellum play complementary roles in regulating ongoing actions when precise updating is required. The basal ganglia have a key role in contextually-based motor decision-making, i.e. for deciding if and when to correct a given movement by initiating corrective submovements, and the cerebellum is more generally involved in amplifying and refining the command signals for movements of different amplitudes.

Thinking as the control of imagination: a conceptual framework for goal-directed systems

This paper offers a conceptual framework which (re)integrates goal-directed control, motivational processes, and executive functions, and suggests a developmental pathway from situated action to higher level cognition. We first illustrate a basic computational (control-theoretic) model of goal-directed action that makes use of internal modeling. We then show that by adding the problem of selection among multiple action alternatives motivation enters the scene, and that the basic mechanisms of executive functions such as inhibition, the monitoring of progresses, and working memory, are required for this system to work. Further, we elaborate on the idea that the off-line re-enactment of anticipatory mechanisms used for action control gives rise to (embodied) mental simulations, and propose that thinking consists essentially in controlling mental simulations rather than directly controlling behavior and perceptions. We conclude by sketching an evolutionary perspective of this process, proposing that anticipation leveraged cognition, and by highlighting specific predictions of our model.

Neurocomputational models of basal ganglia function in learning, memory and choice

The basal ganglia (BG) are critical for the coordination of several motor, cognitive, and emotional functions and become dysfunctional in several pathological states ranging from Parkinson's disease to Schizophrenia. Here we review principles developed within a neurocomputational framework of BG and related circuitry which provide insights into their functional roles in behavior. We focus on two classes of models: those that incorporate aspects of biological realism and constrained by functional principles, and more abstract mathematical models focusing on the higher level computational goals of the BG. While the former are arguably more "realistic", the latter have a complementary advantage in being able to describe functional principles of how the system works in a relatively simple set of equations, but are less suited to making specific hypotheses about the roles of specific nuclei and neurophysiological processes. We review the basic architecture and assumptions of these models, their relevance to our understanding of the neurobiological and cognitive functions of the BG, and provide an update on the potential roles of biological details not explicitly incorporated in existing models. Empirical studies ranging from those in transgenic mice to dopaminergic manipulation, deep brain stimulation, and genetics in humans largely support model predictions and provide the basis for further refinement. Finally, we discuss possible future directions and possible ways to integrate different types of models.

A dopaminergic basis for working memory, learning and attentional shifting in Parkinsonism

Parkinson's disease (PD) patients exhibit cognitive deficits, including reinforcement learning, working memory (WM) and set shifting. Computational models of the basal ganglia-frontal system posit similar mechanisms for these deficits in terms of reduced dynamic range of striatal dopamine (DA) signals in both medicated and non-medicated states. Here, we report results from the first study that tests PD patients on and off dopaminergic medications in a modified version of the AX continuous performance (AX-CPT) working memory task, consisting of three performance phases and one phase requiring WM associations to be learned via reinforcement feedback. Patients generally showed impairments relative to controls. Medicated patients showed deficits specifically when having to ignore distracting stimuli during the delay. Our models suggest that this impairment is due to medication causing excessive WM updating by enhancing striatal "Go" signals that facilitate such updating, while concurrently suppressing "NoGo" signals. In contrast, patients off medication showed deficits consistent with an overall reduction in striatal DA and associated Go updating signals. Supporting this dichotomy, patients on and off medication both showed attentional shifting deficits, but for different reasons. Deficits in non-medicated patients were consistent with an inability to update the new attentional set, whereas those in medicated patients were evident when having to ignore distractors that had previously been task relevant. Finally, in the feedback-based WM phase, medicated patients were better than unmedicated patients, suggesting a key role of striatal DA in using feedback to update information into WM. These results lend further insight into the role of basal ganglia dopamine in WM and broadly support predictions from neurocomputational models.

Action selection and refinement in subcortical loops through basal ganglia and cerebellum

Subcortical loops through the basal ganglia and the cerebellum form computationally powerful distributed processing modules (DPMs). This paper relates the computational features of a DPM's loop through the basal ganglia to experimental results for two kinds of natural action selection. First, functional imaging during a serial order recall task was used to study human brain activity during the selection of sequential actions from working memory. Second, microelectrode recordings from monkeys trained in a step-tracking task were used to study the natural selection of corrective submovements. Our DPM-based model assisted in the interpretation of puzzling data from both of these experiments. We come to posit that the many loops through the basal ganglia each regulate the embodiment of pattern formation in a given area of cerebral cortex. This operation serves to instantiate different kinds of action (or thought) mediated by different areas of cerebral cortex. We then use our findings to formulate a model of the aetiology of schizophrenia.

Understanding decision-making deficits in neurological conditions: insights from models of natural action selection

Models of natural action selection implicate fronto-striatal circuits in both motor and cognitive 'actions'. Dysfunction of these circuits leads to decision-making deficits in various populations. We review how computational models provide insights into the mechanistic basis for these deficits in Parkinson's patients and those with ventromedial frontal damage. We then consider implications of the models for understanding behaviour and cognition in attention-deficit/hyperactivity disorder (ADHD). Incorporation of cortical noradrenaline function into the model improves action selection in noisy environments and accounts for response variability in ADHD. We close with more general clinical implications.

What is reinforced by phasic dopamine signals?

The basal ganglia have been associated with processes of reinforcement learning. A strong line of supporting evidence comes from the recording of dopamine (DA) neurones in behaving monkeys. Unpredicted, biologically salient events, including rewards cause a stereotypic short-latency (70-100 ms), short-duration (100-200 ms) burst of DA activity - the phasic response. This response is widely considered to represent reward prediction errors used as teaching signals in appetitive learning to promote actions that will maximise future reward acquisition. For DA signalling to perform this function, sensory processing afferent to DA neurones should discriminate unpredicted reward-related events. However, the comparative response latencies of DA neurones and orienting gaze-shifts indicate that phasic DA responses are triggered by pre-attentive sensory processing. Consequently, in circumstances where biologically salient events are both spatially and temporally unpredictable, it is unlikely their identity will be known at the time of DA signalling. The limited quality of afferent sensory processing and the precise timing of phasic DA signals, suggests that they may play a less direct role in 'Law of Effect' appetitive learning. Rather, the 'time-stamp' nature of the phasic response, in conjunction with the other signals likely to be present in the basal ganglia at the time of phasic DA input, suggests it may reinforce the discovery of unpredicted sensory events for which the organism is responsible. Furthermore, DA-promoted repetition of preceding actions/movements should enable the system to converge on those aspects of context and behavioural output that lead to the discovery of novel actions.

Indirect biological measures of consciousness from field studies of brains as dynamical systems

Consciousness fully supervenes when the 1.5 kgm mass of protoplasm in the head directs the body into material and social environments and engages in reciprocity. While consciousness is not susceptible to direct measurement, a limited form exercised in animals and pre-lingual children can be measured indirectly with biological assays of arousal, intention and attention. In this essay consciousness is viewed as operating simultaneously in a field at all levels ranging from subatomic to social. The relations and transpositions between levels require sophisticated mathematical treatments that are largely still to be devised. In anticipation of those developments the available experimental data are reviewed concerning the state variables in several levels that collectively constitute the substrate of biological consciousness. The basic metaphors are described that represent the neural machinery of transposition in consciousness. The processes are sketched by which spatiotemporal neural activity patterns emerge as fields that may represent the contents of consciousness. The results of dynamical analysis are discussed in terms serving to distinguish between the neural point processes dictated by the neuron doctrine vs. continuously variable neural fields generated by neural masses in cortex.

Reinforcement learning: Computational theory and biological mechanisms

Reinforcement learning is a computational framework for an active agent to learn behaviors on the basis of a scalar reward signal. The agent can be an animal, a human, or an artificial system such as a robot or a computer program. The reward can be food, water, money, or whatever measure of the performance of the agent. The theory of reinforcement learning, which was developed in an artificial intelligence community with intuitions from animal learning theory, is now giving a coherent account on the function of the basal ganglia. It now serves as the "common language" in which biologists, engineers, and social scientists can exchange their problems and findings. This article reviews the basic theoretical framework of reinforcement learning and discusses its recent and future contributions toward the understanding of animal behaviors and human decision making.

Reinforcement learning: Computational theory and biological mechanisms

Reinforcement learning is a computational framework for an active agent to learn behaviors on the basis of a scalar reward signal. The agent can be an animal, a human, or an artificial system such as a robot or a computer program. The reward can be food, water, money, or whatever measure of the performance of the agent. The theory of reinforcement learning, which was developed in an artificial intelligence community with intuitions from animal learning theory, is now giving a coherent account on the function of the basal ganglia. It now serves as the "common language" in which biologists, engineers, and social scientists can exchange their problems and findings. This article reviews the basic theoretical framework of reinforcement learning and discusses its recent and future contributions toward the understanding of animal behaviors and human decision making.

Signaling patterns of globus pallidus internal segment neurons during forearm rotation

We recorded extracellular single unit discharges of globus pallidus internal segment (GPi) neurons in monkeys performing a visually driven forearm rotation movement task in order to quantify how discharge patterns changed in relation to kinematic parameters. Subjects grasped a handle that rotated about its axis while facing a video screen displaying visual targets. Continuous visual feedback of handle rotation position was provided. Monkeys generated forearm rotation movements of +/-35 degrees and +/-70 degrees amplitude in order to align the cursor and targets. Trial records were aligned to forearm rotation onset in order to compare the discharge patterns that were associated with movements of different amplitudes, velocities, and directions. In addition, we quantified the depth of modulation of neuronal discharge associated with movements generated in two different task phases. Comparisons of discharge patterns were made between the visually guided, rewarded phase ("cued movements") and the self-paced, unrewarded phase that returned the monkey to the task start position ("return movements") by quantifying the goodness of fit between neuronal discharge during cued and return movements. Our analyses revealed no systematic relationship between the depth of modulation of GPi neurons and forearm rotation amplitude, direction, or velocity. Furthermore, comparisons between the two behavioral contexts revealed a systematic attenuation of modulation that could not be attributed to differences in movement velocity. Collectively, these findings suggest that the GPi neurons that we studied were not significantly involved in mediating movement kinematics, but may have instead been instrumental in the processing of information about the behavioral context during which movements were generated.

A physiologically plausible model of action selection and oscillatory activity in the basal ganglia

The basal ganglia (BG) have long been implicated in both motor function and dysfunction. It has been proposed that the BG form a centralized action selection circuit, resolving conflict between multiple neural systems competing for access to the final common motor pathway. We present a new spiking neuron model of the BG circuitry to test this proposal, incorporating all major features and many physiologically plausible details. We include the following: effects of dopamine in the subthalamic nucleus (STN) and globus pallidus (GP), transmission delays between neurons, and specific distributions of synaptic inputs over dendrites. All main parameters were derived from experimental studies. We find that the BG circuitry supports motor program selection and switching, which deteriorates under dopamine-depleted and dopamine-excessive conditions in a manner consistent with some pathologies associated with those dopamine states. We also validated the model against data describing oscillatory properties of BG. We find that the same model displayed detailed features of both gamma-band (30-80 Hz) and slow (approximately 1 Hz) oscillatory phenomena reported by Brown et al. (2002) and Magill et al. (2001), respectively. Only the parameters required to mimic experimental conditions (e.g., anesthetic) or manipulations (e.g., lesions) were changed. From the results, we derive the following novel predictions about the STN-GP feedback loop: (1) the loop is functionally decoupled by tonic dopamine under normal conditions and recoupled by dopamine depletion; (2) the loop does not show pacemaking activity under normal conditions in vivo (but does after combined dopamine depletion and cortical lesion); (3) the loop has a resonant frequency in the gamma-band.

The place of 'codes' in nonlinear neurodynamics

A key problem in cognitive science is to explain the neural mechanisms of the rapid transposition between stimulus energy and abstract concept--between the specific and the generic--in both material and conceptual aspects, not between neural and psychic aspects. Three approaches by researchers to a solution in terms of neural codes are considered. Materialists seek rate and frequency codes in the interspike intervals of trains of action potentials induced by stimuli and carried by topologically organized axonal lines. Cognitivists refer to the symbol grounding problem and search for symbolic codes in firings of hierarchically organized feature-detector neurons of phonemes, lines, odorants, pressures, etc., that object-detector neurons bind into representations of probabilities of stimulus occurrence. Dynamicists seek neural correlates of stimuli and associated behaviors in spatial patterns of oscillatory fields of dendritic activity that self-organize and evolve as trajectories through high-dimensional brain state space; the codes are landscapes of chaotic attractors. Unlike codes in DNA and the periodic table, these codes have neither alphabet nor syntax. They are epistemological metaphors required by experimentalists to measure neural activity and by engineers to model brain functions. Here I review the central neural mechanisms of olfaction as a paradigm for use of codes to explain how brains create cortical activities that mediate sensation, perception, comprehension, prediction, decision, and action or inaction.

The role of the basal ganglia and cerebellum in language processing

The roles of the cerebellum and basal ganglia have typically been confined in the literature to motor planning and control. However, mounting evidence suggests that these structures are involved in more cognitive domains such as language processing. In the current study, we looked at effective connectivity (the influence that one brain region has on another) of the cerebellum and basal ganglia with regions thought to be involved in phonological processing, i.e. left inferior frontal gyrus and left lateral temporal cortex. We analyzed functional magnetic resonance imaging data (fMRI) obtained during a rhyming judgment task in adults using dynamic causal modeling (DCM). The results showed that the cerebellum has reciprocal connections with both left inferior frontal gyrus and left lateral temporal cortex, whereas the putamen has unidirectional connections into these two brain regions. Furthermore, the connections between cerebellum and these phonological processing areas were stronger than the connections between putamen and these areas. This pattern of results suggests that the putamen and cerebellum may have distinct roles in language processing. Based on research in the motor planning and control literature, we argue that the putamen engages in cortical initiation while the cerebellum amplifies and refines this signal to facilitate correct decision making.

Definitions of state variables and state space for brain-computer interface : Part 1. Multiple hierarchical levels of brain function

Neocortical state variables are defined and evaluated at three levels: microscopic using multiple spike activity (MSA), mesoscopic using local field potentials (LFP) and electrocorticograms (ECoG), and macroscopic using electroencephalograms (EEG) and brain imaging. Transactions between levels occur in all areas of cortex, upwardly by integration (abstraction, generalization) and downwardly by differentiation (speciation). The levels are joined by circular causality: microscopic activity upwardly creates mesoscopic order parameters, which downwardly constrain the microscopic activity that creates them. Integration dominates in sensory cortices. Microscopic activity evoked by receptor input in sensation induces emergence of mesoscopic activity in perception, followed by integration of perceptual activity into macroscopic activity in concept formation. The reverse process dominates in motor cortices, where the macroscopic activity embodying the concepts supports predictions of future states as goals. These macroscopic states are conceived to order mesoscopic activity in patterns that constitute plans for actions to achieve the goals. These planning patterns are conceived to provide frames in which the microscopic activity evolves in trajectories that adapted to the immediate environmental conditions detected by new stimuli. This circular sequence forms the action-perception cycle. Its upward limb is understood through correlation of sensory cortical activity with behavior. Now brain-machine interfaces (BMI) offer a means to understand the downward sequence through correlation of behavior with motor cortical activity, beginning with macroscopic goal states and concluding with recording of microscopic MSA trajectories that operate neuroprostheses. Part 1 develops a hypothesis that describes qualitatively the neurodynamics that supports the action-perception cycle and derivative reflex arc. Part 2 describes episodic, "cinematographic" spatial pattern formation and predicts some properties of the macroscopic and mesoscopic frames by which the embedded trajectories of the microscopic activity of cortical sensorimotor neurons might be organized and controlled.

The short-latency dopamine signal: a role in discovering novel actions?

An influential concept in contemporary computational neuroscience is the reward prediction error hypothesis of phasic dopaminergic function. It maintains that midbrain dopaminergic neurons signal the occurrence of unpredicted reward, which is used in appetitive learning to reinforce existing actions that most often lead to reward. However, the availability of limited afferent sensory processing and the precise timing of dopaminergic signals suggest that they might instead have a central role in identifying which aspects of context and behavioural output are crucial in causing unpredicted events.

Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making

The basal ganglia (BG) coordinate decision making processes by facilitating adaptive frontal motor commands while suppressing others. In previous work, neural network simulations accounted for response selection deficits associated with BG dopamine depletion in Parkinson's disease. Novel predictions from this model have been subsequently confirmed in Parkinson patients and in healthy participants under pharmacological challenge. Nevertheless, one clear limitation of that model is in its omission of the subthalamic nucleus (STN), a key BG structure that participates in both motor and cognitive processes. The present model incorporates the STN and shows that by modulating when a response is executed, the STN reduces premature responding and therefore has substantial effects on which response is ultimately selected, particularly when there are multiple competing responses. Increased cortical response conflict leads to dynamic adjustments in response thresholds via cortico-subthalamic-pallidal pathways. The model accurately captures the dynamics of activity in various BG areas during response selection. Simulated dopamine depletion results in emergent oscillatory activity in BG structures, which has been linked with Parkinson's tremor. Finally, the model accounts for the beneficial effects of STN lesions on these oscillations, but suggests that this benefit may come at the expense of impaired decision making.

Origin, structure, and role of background EEG activity. Part 4: Neural frame simulation

OBJECTIVE

To develop a method for simulating background EEG based on the premise that the self-organized activity from synaptic interaction among populations of neurons creates sustained fluctuations that can be modeled with the filtered output of a random number generator.

METHODS

The logarithm of the amplitude of activity was weighted in accordance with 1/f, the log frequency in both temporal (PSD(T)) and spatial (PSD(X)) power spectral densities. The activity was spatially smoothed by volume conduction. Further deviation from full randomness was by sustained spatial coherence averaging 25% of total power. The departure from the background state to an active state, as seen in the awake EEG, was simulated by adding segments that were 90% correlated while attenuating by 50% the uncorrelated background activity in those segments. Spatial amplitude modulation was imposed on the correlated noise to create signals that simulated AM patterns.

RESULTS

The statistical properties of the EEG that were replicated (Freeman, 2004a,b, 2005) included the PSD(T), PSD(X), point spread function (PSF), partitioning of the variance with PCA, and the percentages of correct classification of AM patterns.

CONCLUSIONS

The origin of background EEG was traced to self-sustaining mutual excitation among pyramidal cells creating stable noise that was filtered by self-organized criticality to give 1/f(2) PSD, by inhibitory feedback to give oscillations in the classic clinical bands, and by volume conduction to give smoothing. The essential change that identified a frame in EEG was transient synchrony by phase transition among cortical populations in beta and gamma bands of the PSD(T).

SIGNIFICANCE

This simulation can provide test data with which to optimize techniques for noninvasively extracting information from the EEG for diagnosis and treatment evaluation of neuropsychiatric disorders and for operation by paraplegics of prosthetic devices.

A field-theoretic approach to understanding scale-free neocortical dynamics

A mesoscopic field-theoretic approach is compared with neural network and brain imaging approaches to understanding brain dynamics. Analysis of high spatiotemporal resolution rabbit electroencephalogram (EEG) reveals neural fields in the form of spatial patterns in amplitude (AM) and phase (PM) modulation of gamma and beta carrier waves that serve to classify EEGs from trials with differing conditioned stimuli (CS+/-). Paleocortex exemplified by olfactory EEG has one AM-PM pattern at a time that forms by an input-dependent phase transition. Neocortex shows multiple overlapping AM-PM patterns before and during presentation of CSs. Modeling suggests that neocortex is stabilized in a scale-free state of self-organized criticality, enabling cooperative domains to form virtually instantaneously by phase transitions ranging in size from a few hypercolumns to an entire hemisphere. Self-organized local domains precede formation of global domains that supervene and contribute global modulations to local domains. This mechanism is proposed to explain Gestalt formation in perception.