Dopamine Increases the Gain of the Input-Output Response of Rat Prefrontal Pyramidal Neurons

Motor Cortex Latent Dynamics Encode Spatial and Temporal Arm Movement Parameters Independently

The fluid movement of an arm requires multiple spatiotemporal parameters to be set independently. Recent studies have argued that arm movements are generated by the collective dynamics of neurons in motor cortex. An untested prediction of this hypothesis is that independent parameters of movement must map to independent components of the neural dynamics. Using a task where three male monkeys made a sequence of reaching movements to randomly placed targets, we show that the spatial and temporal parameters of arm movements are independently encoded in the low-dimensional trajectories of population activity in motor cortex: each movement's direction corresponds to a fixed neural trajectory through neural state space and its speed to how quickly that trajectory is traversed. Recurrent neural network models show that this coding allows independent control over the spatial and temporal parameters of movement by separate network parameters. Our results support a key prediction of the dynamical systems view of motor cortex, and also argue that not all parameters of movement are defined by different trajectories of population activity.

Motor cortex latent dynamics encode spatial and temporal arm movement parameters independently

The fluid movement of an arm requires multiple spatiotemporal parameters to be set independently. Recent studies have argued that arm movements are generated by the collective dynamics of neurons in motor cortex. An untested prediction of this hypothesis is that independent parameters of movement must map to independent components of the neural dynamics. Using a task where monkeys made a sequence of reaching movements to randomly placed targets, we show that the spatial and temporal parameters of arm movements are independently encoded in the low-dimensional trajectories of population activity in motor cortex: Each movement's direction corresponds to a fixed neural trajectory through neural state space and its speed to how quickly that trajectory is traversed. Recurrent neural network models show this coding allows independent control over the spatial and temporal parameters of movement by separate network parameters. Our results support a key prediction of the dynamical systems view of motor cortex, but also argue that not all parameters of movement are defined by different trajectories of population activity.

Dopamine encoding of novelty facilitates efficient uncertainty-driven exploration

When facing an unfamiliar environment, animals need to explore to gain new knowledge about which actions provide reward, but also put the newly acquired knowledge to use as quickly as possible. Optimal reinforcement learning strategies should therefore assess the uncertainties of these action-reward associations and utilise them to inform decision making. We propose a novel model whereby direct and indirect striatal pathways act together to estimate both the mean and variance of reward distributions, and mesolimbic dopaminergic neurons provide transient novelty signals, facilitating effective uncertainty-driven exploration. We utilised electrophysiological recording data to verify our model of the basal ganglia, and we fitted exploration strategies derived from the neural model to data from behavioural experiments. We also compared the performance of directed exploration strategies inspired by our basal ganglia model with other exploration algorithms including classic variants of upper confidence bound (UCB) strategy in simulation. The exploration strategies inspired by the basal ganglia model can achieve overall superior performance in simulation, and we found qualitatively similar results in fitting model to behavioural data compared with the fitting of more idealised normative models with less implementation level detail. Overall, our results suggest that transient dopamine levels in the basal ganglia that encode novelty could contribute to an uncertainty representation which efficiently drives exploration in reinforcement learning.

L-DOPA enhances neural direction signals in younger and older adults

Previous studies indicate a role of dopamine in spatial navigation. Although neural representations of direction are an important aspect of spatial cognition, it is not well understood whether dopamine directly affects these representations, or only impacts other aspects of spatial brain function. Moreover, both dopamine and spatial cognition decline sharply during age, raising the question which effect dopamine has on directional signals in the brain of older adults. To investigate these questions, we used a double-blind cross-over L-DOPA/Placebo intervention design in which 43 younger and 37 older adults navigated in a virtual spatial environment while undergoing functional magnetic resonance imaging (fMRI). We studied the effect of L-DOPA, a dopamine precursor, on fMRI activation patterns that encode spatial walking directions that have previously been shown to lose specificity with age. This was done in predefined regions of interest, including the early visual cortex, retrosplenial cortex, and hippocampus. Classification of brain activation patterns associated with different walking directions was improved across all regions following L-DOPA administration, suggesting that dopamine broadly enhances neural representations of direction. No evidence for differences between regions was found. In the hippocampus these results were found in both age groups, while in the retrosplenial cortex they were only observed in younger adults. Taken together, our study provides evidence for a link between dopamine and the specificity of neural responses during spatial navigation. SIGNIFICANCE STATEMENT: The sense of direction is an important aspect of spatial navigation, and neural representations of direction can be found throughout a large network of space-related brain regions. But what influences how well these representations track someone's true direction? Using a double-blind cross-over L-DOPA/Placebo intervention design, we find causal evidence that the neurotransmitter dopamine impacts the fidelity of direction selective neural representations in the human hippocampus and retrosplenial cortex. Interestingly, the effect of L-DOPA was either equally present or even smaller in older adults, despite the well-known age related decline of dopamine. These results provide novel insights into how dopamine shapes the neural representations that underlie spatial navigation.

Dynamic branching in a neural network model for probabilistic prediction of sequences

An important function of the brain is to predict which stimulus is likely to occur based on the perceived cues. The present research studied the branching behavior of a computational network model of populations of excitatory and inhibitory neurons, both analytically and through simulations. Results show how synaptic efficacy, retroactive inhibition and short-term synaptic depression determine the dynamics of selection between different branches predicting sequences of stimuli of different probabilities. Further results show that changes in the probability of the different predictions depend on variations of neuronal gain. Such variations allow the network to optimize the probability of its predictions to changing probabilities of the sequences without changing synaptic efficacy.

Invariant neural subspaces maintained by feedback modulation

Sensory systems reliably process incoming stimuli in spite of changes in context. Most recent models accredit this context invariance to an extraction of increasingly complex sensory features in hierarchical feedforward networks. Here, we study how context-invariant representations can be established by feedback rather than feedforward processing. We show that feedforward neural networks modulated by feedback can dynamically generate invariant sensory representations. The required feedback can be implemented as a slow and spatially diffuse gain modulation. The invariance is not present on the level of individual neurons, but emerges only on the population level. Mechanistically, the feedback modulation dynamically reorients the manifold of neural activity and thereby maintains an invariant neural subspace in spite of contextual variations. Our results highlight the importance of population-level analyses for understanding the role of feedback in flexible sensory processing.

CRF-receptor1 modulation of the dopamine projection to prelimbic cortex facilitates cognitive flexibility after acute and chronic stress

Stress reduces cognitive flexibility and dopamine D1 receptor-related activity in the prelimbic cortex (PL), effects hypothesized to depend on reduced corticotropic releasing factor receptor type 1 (CRFr1) regulation of dopamine neurons in the ventral tegmental area (VTA). We assessed this hypothesis in rats by examining the effect of chronic unpredictable restraint stress (CUS), mild acute stress, or their combination on cognitive flexibility, CRFr1 expression in the VTA and D1-related activity in PL. In Experiment 1, rats received either CUS or equivalent handling for 14 days before being trained to press two levers to earn distinct food outcomes. Initial learning was assessed using an outcome devaluation test after which cognitive flexibility was assessed by reversing the outcomes earned by the actions. Prior to each reversal training session, half the CUS and controls receiving acute stress with action-outcome updating assessed using a second devaluation test and CRFr1 expression in the VTA assessed using in-situ hybridisation. Although CUS did not itself affect action-outcome learning, its combination with acute stress blocked reversal learning and decreased VTA CRFr1 expression after acute shock. The relationship between these latter two effects was assessed in Experiment 2 by pharmacologically disconnecting the VTA and PL, unilaterally blocking neurons expressing CRFr1 in the VTA and D1 receptors in the contralateral PL during reversal learning after acute stress. Acute stress again blocked reversal learning but only in the group with VTA-PL disconnection, demonstrating that VTA CRFr1-induced facilitation of dopaminergic activity in the PL is necessary for maintaining cognitive flexibility after acute stress. [250].

•

Acute stress increased CRF receptor1 expression in the VTA.

•

Chronic stress attenuated the effect of acute stress on CRFr1 expression.

•

Chronic stress plus acute stress produced a loss of cognitive flexibility.

•

Blocking VTA CFRr1 and dopamine D1r in PL reduced cognitive flexibility following stress.

Nicotine Exposure during Adolescence Leads to Changes of Synaptic Plasticity and Intrinsic Excitability of Mice Insular Pyramidal Cells at Later Life

To find satisfactory treatment for nicotine addiction, synaptic and cellular mechanisms should be investigated comprehensively. Synaptic transmission, plasticity and intrinsic excitability in various brain regions are known to be altered by acute nicotine exposure. However, it has not been addressed whether and how nicotine exposure during adolescence alters these synaptic events and intrinsic excitability in the insular cortex in adulthood. To address this question, we performed whole-cell patch-clamp recordings to examine the effects of adolescent nicotine exposure on synaptic transmission, plasticity and intrinsic excitability in layer V pyramidal neurons (PNs) of the mice insular cortex five weeks after the treatment. We found that excitatory synaptic transmission and potentiation were enhanced in these neurons. Following adolescent nicotine exposure, insular layer V PNs displayed enhanced intrinsic excitability, which was reflected in changes in relationship between current strength and spike number, inter-spike interval, spike current threshold and refractory period. In addition, spike-timing precision evaluated by standard deviation of spike timing was decreased following nicotine exposure. Our data indicate that adolescent nicotine exposure enhances synaptic transmission, plasticity and intrinsic excitability in layer V PNs of the mice insular cortex at later life, which might contribute to severe nicotine dependence in adulthood.

Event-based control of autonomic and emotional states by the anterior cingulate cortex

Despite being an intensive area of research, the function of the anterior cingulate cortex (ACC) remains somewhat of a mystery. Human imaging studies implicate the ACC in various cognitive functions, yet surgical ACC lesions used to treat emotional disorders have minimal lasting effects on cognition. An alternative view is that ACC regulates autonomic states, consistent with its interconnectivity with autonomic control regions and that stimulation evokes changes in autonomic/emotional states. At the cellular level, ACC neurons are highly multi-modal and promiscuous, and can represent a staggering array of task events. These neurons nevertheless combine to produce highly event-specific ensemble patterns that likely alter activity in downstream regions controlling emotional and autonomic tone. Since neuromodulators regulate the strength of the ensemble activity patterns, they would regulate the impact these patterns have on downstream targets. Through these mechanisms, the ACC may determine how strongly to react to the very events its ensembles represent. Pathologies arise when specific event-related representations gain excessive control over autonomic/emotional states.

Gated recurrence enables simple and accurate sequence prediction in stochastic, changing, and structured environments

From decision making to perception to language, predicting what is coming next is crucial. It is also challenging in stochastic, changing, and structured environments; yet the brain makes accurate predictions in many situations. What computational architecture could enable this feat? Bayesian inference makes optimal predictions but is prohibitively difficult to compute. Here, we show that a specific recurrent neural network architecture enables simple and accurate solutions in several environments. This architecture relies on three mechanisms: gating, lateral connections, and recurrent weight training. Like the optimal solution and the human brain, such networks develop internal representations of their changing environment (including estimates of the environment’s latent variables and the precision of these estimates), leverage multiple levels of latent structure, and adapt their effective learning rate to changes without changing their connection weights. Being ubiquitous in the brain, gated recurrence could therefore serve as a generic building block to predict in real-life environments.

Reward motivation increases univariate activity but has limited effect on coding of task-relevant information across the frontoparietal cortex

Selection and integration of information based on current goals is fundamental for goal-directed behavior. Reward motivation has been shown to improve behavioral performance, yet the neural mechanisms that link motivation and control processes, and in particular its effect on context-dependent information processing, remain unclear. We used functional magnetic resonance imaging (fMRI) in 24 human volunteers (13 females) to test whether reward motivation enhances the coding of task-relevant information across the frontoparietal cortex, as would be predicted based on previous experimental evidence and theoretical accounts. In a cued target detection task, participants detected whether an object from a cued visual category was present in a subsequent display. The combination of the cue and the object visual category determined the behavioral status of the objects. To manipulate reward motivation, half of all trials offered the possibility of a monetary reward. We observed an increase with reward in overall univariate activity across the frontoparietal control network when the cue and subsequent object were presented. Multivariate pattern analysis (MVPA) showed that behavioral status information for the objects was conveyed across the network. However, in contrast to our prediction, reward did not increase the discrimination between behavioral status conditions in the stimulus epoch of a trial when object information was processed depending on a current context. In the high-level general-object visual region, the lateral occipital complex, the representation of behavioral status was driven by visual differences and was not modulated by reward. Our study provides useful evidence for the limited effects of reward motivation on task-related neural representations and highlights the necessity to unravel the diverse forms and extent of these effects.

•

Reward motivation leads to increased activity in the frontoparietal control network.

•

Task-relevant information is coded across the control network.

•

Reward does not increase coding of behavioral relevance of task information.

•

Visual category coding in the general-object region does not increase with reward.

Transcriptomics-informed large-scale cortical model captures topography of pharmacological neuroimaging effects of LSD

Psychoactive drugs can transiently perturb brain physiology while preserving brain structure. The role of physiological state in shaping neural function can therefore be investigated through neuroimaging of pharmacologically induced effects. Previously, using pharmacological neuroimaging, we found that neural and experiential effects of lysergic acid diethylamide (LSD) are attributable to agonism of the serotonin-2A receptor (Preller et al., 2018). Here, we integrate brain-wide transcriptomics with biophysically based circuit modeling to simulate acute neuromodulatory effects of LSD on human cortical large-scale spatiotemporal dynamics. Our model captures the inter-areal topography of LSD-induced changes in cortical blood oxygen level-dependent (BOLD) functional connectivity. These findings suggest that serotonin-2A-mediated modulation of pyramidal-neuronal gain is a circuit mechanism through which LSD alters cortical functional topography. Individual-subject model fitting captures patterns of individual neural differences in pharmacological response related to altered states of consciousness. This work establishes a framework for linking molecular-level manipulations to systems-level functional alterations, with implications for precision medicine.

Neurochemical mechanisms and neurocircuitry underlying the contribution of stress to cocaine seeking

In individuals with substance use disorders, stress is a critical determinant of relapse susceptibility. In some cases, stressors directly trigger cocaine use. In others, stressors interact with other stimuli to promote drug seeking, thereby setting the stage for relapse. Here, we review the mechanisms and neurocircuitry that mediate stress-triggered and stress-potentiated cocaine seeking. Stressors trigger cocaine seeking by activating noradrenergic projections originating in the lateral tegmentum that innervate the bed nucleus of the stria terminalis to produce beta adrenergic receptor-dependent regulation of neurons that release corticotropin releasing factor (CRF) into the ventral tegmental area (VTA). CRF promotes the activation of VTA dopamine neurons that innervate the prelimbic prefrontal cortex resulting in D1 receptor-dependent excitation of a pathway to the nucleus accumbens core that mediates cocaine seeking. The stage-setting effects of stress require glucocorticoids, which exert rapid non-canonical effects at several sites within the mesocorticolimbic system. In the nucleus accumbens, corticosterone attenuates dopamine clearance via the organic cation transporter 3 to promote dopamine signaling. In the prelimbic cortex, corticosterone mobilizes the endocannabinoid, 2-arachidonoylglycerol (2-AG), which produces CB1 receptor-dependent reductions in inhibitory transmission, thereby increasing excitability of neurons which comprise output pathways responsible for cocaine seeking. Factors that influence the role of stress in cocaine seeking, including prior history of drug use, biological sex, chronic stress/co-morbid stress-related disorders, adolescence, social variables, and genetics are discussed. Better understanding when and how stress contributes to drug seeking should guide the development of more effective interventions, particularly for those whose drug use is stress related.

The anterior cingulate cortex and event-based modulation of autonomic states

In spite of being an intensive area of research focus, the anterior cingulate cortex (ACC) remains somewhat of an enigma. Many theories have focused on its role in various aspects of cognition yet surgically precise lesions of the ACC, used to treat severe emotional disorders in human patients, typically have no lasting effects on cognition. An alternative view is that the ACC has a prominent role in regulating autonomic states. This view is consistent with anatomical data showing that a main target of the ACC are regions involved in autonomic control and with the observation that stimulation of the ACC evokes changes in autonomic states in both animals and humans. From an electrophysiological perspective, ACC neurons appear able to represent virtually any event or internal state, even though there is not always a strong link between these representations and behavior. Ensembles of neurons form robust contextual representations that strongly influence how specific events are encoded. The activity patterns associated with these contextually-based event representations presumably impact activity in downstream regions that control autonomic state. As a result, the ACC may regulate the autonomic and perhaps emotional reactions to events it is representing. This event-based control of autonomic tone by the ACC would likely arise during all types of cognitive and affective processes, without necessarily being critical for any of them.

Dopamine role in learning and action inference

This paper describes a framework for modelling dopamine function in the mammalian brain. It proposes that both learning and action planning involve processes minimizing prediction errors encoded by dopaminergic neurons. In this framework, dopaminergic neurons projecting to different parts of the striatum encode errors in predictions made by the corresponding systems within the basal ganglia. The dopaminergic neurons encode differences between rewards and expectations in the goal-directed system, and differences between the chosen and habitual actions in the habit system. These prediction errors trigger learning about rewards and habit formation, respectively. Additionally, dopaminergic neurons in the goal-directed system play a key role in action planning: They compute the difference between a desired reward and the reward expected from the current motor plan, and they facilitate action planning until this difference diminishes. Presented models account for dopaminergic responses during movements, effects of dopamine depletion on behaviour, and make several experimental predictions.

In the brain, chemicals such as dopamine allow nerve cells to ‘talk’ to each other and to relay information from and to the environment. Dopamine, in particular, is released when pleasant surprises are experienced: this helps the organism to learn about the consequences of certain actions. If a new flavour of ice-cream tastes better than expected, for example, the release of dopamine tells the brain that this flavour is worth choosing again.

However, dopamine has an additional role in controlling movement. When the cells that produce dopamine die, for instance in Parkinson’s disease, individuals may find it difficult to initiate deliberate movements. Here, Rafal Bogacz aimed to develop a comprehensive framework that could reconcile the two seemingly unrelated roles played by dopamine.

The new theory proposes that dopamine is released when an outcome differs from expectations, which helps the organism to adjust and minimise these differences. In the ice-cream example, the difference is between how good the treat is expected to taste, and how tasty it really is. By learning to select the same flavour repeatedly, the brain aligns expectation and the result of the choice. This ability would also apply when movements are planned. In this case, the brain compares the desired reward with the predicted results of the planned actions. For example, while planning to get a spoonful of ice-cream, the brain compares the pleasure expected from the movement that is currently planned, and the pleasure of eating a full spoon of the treat. If the two differ, for example because no movement has been planned yet, the brain releases dopamine to form a better version of the action plan. The theory was then tested using a computer simulation of nerve cells that release dopamine; this showed that the behaviour of the virtual cells closely matched that of their real-life counterparts.

This work offers a comprehensive description of the fundamental role of dopamine in the brain. The model now needs to be verified through experiments on living nerve cells; ultimately, it could help doctors and researchers to develop better treatments for conditions such as Parkinson’s disease or ADHD, which are linked to a lack of dopamine.

Modeling the effects of motivation on choice and learning in the basal ganglia

Decision making relies on adequately evaluating the consequences of actions on the basis of past experience and the current physiological state. A key role in this process is played by the basal ganglia, where neural activity and plasticity are modulated by dopaminergic input from the midbrain. Internal physiological factors, such as hunger, scale signals encoded by dopaminergic neurons and thus they alter the motivation for taking actions and learning. However, to our knowledge, no formal mathematical formulation exists for how a physiological state affects learning and action selection in the basal ganglia. We developed a framework for modelling the effect of motivation on choice and learning. The framework defines the motivation to obtain a particular resource as the difference between the desired and the current level of this resource, and proposes how the utility of reinforcements depends on the motivation. To account for dopaminergic activity previously recorded in different physiological states, the paper argues that the prediction error encoded in the dopaminergic activity needs to be redefined as the difference between utility and expected utility, which depends on both the objective reinforcement and the motivation. We also demonstrate a possible mechanism by which the evaluation and learning of utility of actions can be implemented in the basal ganglia network. The presented theory brings together models of learning in the basal ganglia with the incentive salience theory in a single simple framework, and it provides a mechanistic insight into how decision processes and learning in the basal ganglia are modulated by the motivation. Moreover, this theory is also consistent with data on neural underpinnings of overeating and obesity, and makes further experimental predictions.

Transient neocortical gamma oscillations induced by neuronal response modulation

In this paper a mean field model of spatio-temporal electroencephalographic activity in the neocortex is used to computationally study the emergence of neocortical gamma oscillations as a result of neuronal response modulation. It is shown using a numerical bifurcation analysis that gamma oscillations emerge robustly in the solutions of the model and transition to beta oscillations through coordinated modulation of the responsiveness of inhibitory and excitatory neuronal populations. The spatio-temporal pattern of the propagation of these oscillations across the neocortex is illustrated by solving the equations of the model using a finite element software package. Thereby, it is shown that the gamma oscillations remain localized to the regions of neuronal modulation. Moreover, it is discussed that the inherent spatial averaging effect of commonly used electrocortical measurement techniques can significantly alter the amplitude and pattern of fast oscillations in neocortical recordings, and hence can potentially affect physiological interpretations of these recordings.

Outcome contingency selectively affects the neural coding of outcomes but not of tasks

Value-based decision-making is ubiquitous in every-day life, and critically depends on the contingency between choices and their outcomes. Only if outcomes are contingent on our choices can we make meaningful value-based decisions. Here, we investigate the effect of outcome contingency on the neural coding of rewards and tasks. Participants performed a reversal-learning paradigm in which reward outcomes were contingent on trial-by-trial choices, and performed a 'free choice' paradigm in which rewards were random and not contingent on choices. We hypothesized that contingent outcomes enhance the neural coding of rewards and tasks, which was tested using multivariate pattern analysis of fMRI data. Reward outcomes were encoded in a large network including the striatum, dmPFC and parietal cortex, and these representations were indeed amplified for contingent rewards. Tasks were encoded in the dmPFC at the time of decision-making, and in parietal cortex in a subsequent maintenance phase. We found no evidence for contingency-dependent modulations of task signals, demonstrating highly similar coding across contingency conditions. Our findings suggest selective effects of contingency on reward coding only, and further highlight the role of dmPFC and parietal cortex in value-based decision-making, as these were the only regions strongly involved in both reward and task coding.

Bipolar oscillations between positive and negative mood states in a computational model of Basal Ganglia

Bipolar disorder is characterized by mood swings-oscillations between manic and depressive states. The swings (oscillations) mark the length of an episode in a patient's mood cycle (period), and can vary from hours to years. The proposed modeling study uses decision making framework to investigate the role of basal ganglia network in generating bipolar oscillations. In this model, the basal ganglia system performs a two-arm bandit task in which one of the arms (action responses) leads to a positive outcome, while the other leads to a negative outcome. We explore the dynamics of key reward and risk related parameters in the system while the model agent receives various outcomes. Particularly, we study the system using a model that represents the fast dynamics of decision making, and a module to capture the slow dynamics that describe the variation of some meta-parameters of fast dynamics over long time scales. The model is cast at three levels of abstraction: (1) a two-dimensional dynamical system model, that is a simple two variable model capable of showing bistability for rewarding and punitive outcomes; (2) a phenomenological basal ganglia model, to extend the implications from the reduced model to a cortico-basal ganglia setup; (3) a detailed network model of basal ganglia, that incorporates detailed cellular level models for a more realistic understanding. In healthy conditions, the model chooses positive action and avoids negative one, whereas under bipolar conditions, the model exhibits slow oscillations in its choice of positive or negative outcomes, reminiscent of bipolar oscillations. Phase-plane analyses on the simple reduced dynamical system with two variables reveal the essential parameters that generate pathological 'bipolar-like' oscillations. Phenomenological and network models of the basal ganglia extend that logic, and interpret bipolar oscillations in terms of the activity of dopaminergic and serotonergic projections on the cortico-basal ganglia network dynamics. The network's dysfunction, specifically in terms of reward and risk sensitivity, is shown to be responsible for the pathological bipolar oscillations. The study proposes a computational model that explores the effects of impaired serotonergic neuromodulation on the dynamics of the cortico basal ganglia network, and relates this impairment to abstract mood states (manic and depressive episodes) and oscillations of bipolar disorder.

Oscillatory recurrent gated neural integrator circuits (ORGaNICs), a unifying theoretical framework for neural dynamics

Working memory is an example of a cognitive and neural process that is not static but evolves dynamically with changing sensory inputs; another example is motor preparation and execution. We introduce a theoretical framework for neural dynamics, based on oscillatory recurrent gated neural integrator circuits (ORGaNICs), and apply it to simulate key phenomena of working memory and motor control. The model circuits simulate neural activity with complex dynamics, including sequential activity and traveling waves of activity, that manipulate (as well as maintain) information during working memory. The same circuits convert spatial patterns of premotor activity to temporal profiles of motor control activity and manipulate (e.g., time warp) the dynamics. Derivative-like recurrent connectivity, in particular, serves to manipulate and update internal models, an essential feature of working memory and motor execution. In addition, these circuits incorporate recurrent normalization, to ensure stability over time and robustness with respect to perturbations of synaptic weights.

Believing in dopamine

Midbrain dopamine signals are widely thought to report reward prediction errors that drive learning in the basal ganglia. However, dopamine has also been implicated in various probabilistic computations, such as encoding uncertainty and controlling exploration. Here, we show how these different facets of dopamine signalling can be brought together under a common reinforcement learning framework. The key idea is that multiple sources of uncertainty impinge on reinforcement learning computations: uncertainty about the state of the environment, the parameters of the value function and the optimal action policy. Each of these sources plays a distinct role in the prefrontal cortex-basal ganglia circuit for reinforcement learning and is ultimately reflected in dopamine activity. The view that dopamine plays a central role in the encoding and updating of beliefs brings the classical prediction error theory into alignment with more recent theories of Bayesian reinforcement learning.

Learning the payoffs and costs of actions

A set of sub-cortical nuclei called basal ganglia is critical for learning the values of actions. The basal ganglia include two pathways, which have been associated with approach and avoid behavior respectively and are differentially modulated by dopamine projections from the midbrain. Inspired by the influential opponent actor learning model, we demonstrate that, under certain circumstances, these pathways may represent learned estimates of the positive and negative consequences (payoffs and costs) of individual actions. In the model, the level of dopamine activity encodes the motivational state and controls to what extent payoffs and costs enter the overall evaluation of actions. We show that a set of previously proposed plasticity rules is suitable to extract payoffs and costs from a prediction error signal if they occur at different moments in time. For those plasticity rules, successful learning requires differential effects of positive and negative outcome prediction errors on the two pathways and a weak decay of synaptic weights over trials. We also confirm through simulations that the model reproduces drug-induced changes of willingness to work, as observed in classical experiments with the D2-antagonist haloperidol.

The basal ganglia are structures underneath the surface of the vertebrate brain, associated with error-driven learning. Much is known about the anatomical and biological features of the basal ganglia; scientists now try to understand the algorithms implemented by these structures. Numerous models aspire to capture the learning functionality, but many of them only cover some specific aspect of the algorithm. Instead of further adding to that pool of partial models, we unify two existing ones—one which captures what the basal ganglia learn, and one that describes the learning mechanism itself. The first model suggests that the basal ganglia weigh positive against negative consequences of actions according to the motivational state. It hints how payoff and cost might be represented, but does not explain how those representations arise. The other model consists of biologically plausible plasticity rules, which describe how learning takes place, but not how the brain makes use of what is learned. We show that the two theories are compatible. Together, they form a model of learning and decision making that integrates the motivational state as well as the learned payoffs and costs of opportunities.

Motor primitives in space and time via targeted gain modulation in cortical networks

Motor cortex (M1) exhibits a rich repertoire of neuronal activities to support the generation of complex movements. Although recent neuronal-network models capture many qualitative aspects of M1 dynamics, they can generate only a few distinct movements. Additionally, it is unclear how M1 efficiently controls movements over a wide range of shapes and speeds. We demonstrate that modulation of neuronal input-output gains in recurrent neuronal-network models with a fixed architecture can dramatically reorganize neuronal activity and thus downstream muscle outputs. Consistent with the observation of diffuse neuromodulatory projections to M1, a relatively small number of modulatory control units provide sufficient flexibility to adjust high-dimensional network activity using a simple reward-based learning rule. Furthermore, it is possible to assemble novel movements from previously learned primitives, and one can separately change movement speed while preserving movement shape. Our results provide a new perspective on the role of modulatory systems in controlling recurrent cortical activity.

The Changes of Intrinsic Excitability of Pyramidal Neurons in Anterior Cingulate Cortex in Neuropathic Pain

To find satisfactory treatment strategies for neuropathic pain syndromes, the cellular mechanisms should be illuminated. Central sensitization is a generator of pain hypersensitivity, and is mainly reflected in neuronal hyperexcitability in pain pathway. Neuronal excitability depends on two components, the synaptic inputs and the intrinsic excitability. Previous studies have focused on the synaptic plasticity in different forms of pain. But little is known about the changes of neuronal intrinsic excitability in neuropathic pain. To address this question, whole-cell patch clamp recordings were performed to study the synaptic transmission and neuronal intrinsic excitability 1 week after spared nerve injury (SNI) or sham operation in male C57BL/6J mice. We found increased spontaneous excitatory postsynaptic currents (sEPSC) frequency in layer II/III pyramidal neurons of anterior cingulate cortex (ACC) from mice with neuropathic pain. Elevated intrinsic excitability of these neurons after nerve injury was also picked up, which was reflected in gain of input-output curve, inter-spike interval (ISI), spike threshold and Refractory period (RP). Besides firing rate related to neuronal intrinsic excitability, spike timing also plays an important role in neural information processing. The precision of spike timing measured by standard deviation of spike timing (SDST) was decreased in neuropathic pain state. The electrophysiological studies revealed the elevated intrinsic excitation in layer II/III pyramidal neurons of ACC in mice with neuropathic pain, which might contribute to central excitation.

Immunolocalization of muscarinic M1 receptor in the rat medial prefrontal cortex

The medial prefrontal cortex (mPFC) has been considered to participate in many higher cognitive functions, such as memory formation and spatial navigation. These cognitive functions are modulated by cholinergic afferents via muscarinic acetylcholine receptors. Previous pharmacological studies have strongly suggested that the M1 receptor (M1R) is the most important subtype among muscarinic receptors to perform these cognitive functions. Actually, M1R is abundant in mPFC. However, the proportion of somata containing M1R among cortical cellular types, and the precise intracellular localization of M1R remain unclear. In this study, to clarify the precise immunolocalization of M1R in rat mPFC, we examined three major cellular types, pyramidal neurons, inhibitory neurons, and astrocytes. M1R immunopositivity signals were found in the majority of the somata of both pyramidal neurons and inhibitory neurons. In pyramidal neurons, strong M1R immunopositivity signals were usually found throughout their somata and dendrites including spines. On the other hand, the signal strength of M1R immunopositivity in the somata of inhibitory neurons significantly varied. Some neurons showed strong signals. Whereas about 40% of GAD67‐immunopositive neurons and 30% of parvalbumin‐immunopositive neurons (PV neurons) showed only weak signals. In PV neurons, M1R immunopositivity signals were preferentially distributed in somata. Furthermore, we found that many astrocytes showed substantial M1R immunopositivity signals. These signals were also mainly distributed in their somata. Thus, the distribution pattern of M1R markedly differs between cellular types. This difference might underlie the cholinergic modulation of higher cognitive functions subserved by mPFC.

Interactions of Motivation and Cognitive Control

There is general agreement that both motivation and cognitive control play critical roles in shaping goal-directed behavior, but only recently has scientific interest focused around the question of motivation-control interactions. Here we briefly survey this literature, organizing contemporary findings around three issues: 1) whether motivation preferentially impacts cognitive control processes, 2) the neural mechanisms that underlie motivation-cognition interactions, and 3) why motivation might be relevant for overcoming the costs of control. Dopamine (DA) is discussed as a key neuromodulator in these motivation-cognition interactions. We conclude by highlighting open issues, specifically Pavlovian versus instrumental control distinctions and effects of motivational valence and conflict, which could benefit from future research attention.

Dopamine terminals from the ventral tegmental area gate intrinsic inhibition in the prefrontal cortex

Spike frequency adaptation (SFA or accommodation) and calcium‐activated potassium channels that underlie after‐hyperpolarization potentials (AHP) regulate repetitive firing of neurons. Precisely how neuromodulators such as dopamine from the ventral tegmental area (VTA) regulate SFA and AHP (together referred to as intrinsic inhibition) in the prefrontal cortex (PFC) remains unclear. Using whole cell electrophysiology, we measured intrinsic inhibition in prelimbic (PL) layer 5 pyramidal cells of male adult rats. Results demonstrate that bath application of dopamine reduced intrinsic inhibition (EC₅₀: 25.0 μmol/L). This dopamine action was facilitated by coapplication of cocaine (1 μmol/L), a blocker of dopamine reuptake. To evaluate VTA dopamine terminals in PFC slices, we transfected VTA dopamine cells of TH::Cre rats in vivo with Cre‐dependent AAVs to express channelrhodopsin‐2 (ChR2) or designer receptors exclusively activated by designer drugs (DREADDS). In PFC slices from these animals, stimulation of VTA terminals with either blue light to activate ChR2 or bath application of clozapine‐N‐oxide (CNO) to activate Gq‐DREADDs produced a similar reduction in intrinsic inhibition in PL neurons. Electrophysiological recordings from cells expressing retrograde fluorescent tracers showed that this plasticity occurs in PL neurons projecting to the accumbens core. Collectively, these data highlight an ability of VTA terminals to gate intrinsic inhibition in the PFC, and under appropriate circumstances, enhance PL neuronal firing. These cellular actions of dopamine may be important for dopamine‐dependent behaviors involving cocaine and cue‐reward associations within cortical–striatal circuits.

Laminar fMRI and computational theories of brain function

Recently developed methods for functional MRI at the resolution of cortical layers (laminar fMRI) offer a novel window into neurophysiological mechanisms of cortical activity. Beyond physiology, laminar fMRI also offers an unprecedented opportunity to test influential theories of brain function. Specifically, hierarchical Bayesian theories of brain function, such as predictive coding, assign specific computational roles to different cortical layers. Combined with computational models, laminar fMRI offers a unique opportunity to test these proposals noninvasively in humans. This review provides a brief overview of predictive coding and related hierarchical Bayesian theories, summarises their predictions with regard to layered cortical computations, examines how these predictions could be tested by laminar fMRI, and considers methodological challenges. We conclude by discussing the potential of laminar fMRI for clinically useful computational assays of layer-specific information processing.

Conditioned task-set competition: Neural mechanisms of emotional interference in depression

Depression has been associated with increased response times at the incongruent-, neutral-, and negative-word trials of the classical and emotional Stroop tasks (Epp et al., Clinical Psychology Review, 32, 316-328, 2012). Response-time slowdown effects at incongruent- and negative-word trials of the Stroop tasks were reported to correlate with depressive severity, indicating strong relevance of the effects to the symptomatology. This study proposes a novel integrative computational model of neural mechanisms of both the classical and emotional Stroop effects, drawing on the previous prominent theoretical explanations of performance at the classical Stroop task (Cohen, Dunbar, & McClelland, Psychological Review, 97, 332-361, 1990; Herd, Banich, & O'Reilly, Journal of Cognitive Neuroscience, 18, 22-32, 2006), and in addition suggesting that negative emotional words represent conditioned stimuli for future negative outcomes. The model is shown to explain the classical Stroop effect and the slow (between-trial) emotional Stroop effect with biologically plausible mechanisms, providing an advantage over the previous theoretical accounts (Matthews & Harley, Cognition & Emotion, 10, 561-600, 1996; Wyble, Sharma, & Bowman, Cognition & Emotion, 22, 1019-1051, 2008). Simulation results suggested a candidate mechanism responsible for the pattern of depressive performance at the classical and the emotional Stroop tasks. Hyperactivity of the amygdala, together with increased inhibitory influence of the amygdala over dopaminergic neurotransmission, could be at the origin of the performance deficits.

Unified neural field theory of brain dynamics underlying oscillations in Parkinson's disease and generalized epilepsies

The mechanisms underlying pathologically synchronized neural oscillations in Parkinson's disease (PD) and generalized epilepsies are explored in parallel via a physiologically-based neural field model of the corticothalamic-basal ganglia (CTBG) system. The basal ganglia (BG) are approximated as a single effective population and their roles in the modulation of oscillatory dynamics of the corticothalamic (CT) system and vice versa are analyzed. In addition to normal EEG rhythms, enhanced activity around 4 Hz and 20 Hz exists in the model, consistent with the characteristic frequencies observed in PD. These rhythms result from resonances in loops formed between the BG and CT populations, analogous to those that underlie epileptic oscillations in a previous CT model, and which are still present in the combined CTBG system. Dopamine depletion is argued to weaken the dampening of these loop resonances in PD, and network connections then explain the significant coherence observed between BG, thalamic, and cortical population activity around 4-8 Hz and 20 Hz. Parallels between the afferent and efferent connection sites of the thalamic reticular nucleus (TRN) and BG predict low dopamine to correspond to a reduced likelihood of tonic-clonic (grand mal) seizures, which agrees with experimental findings. Furthermore, the model predicts an increased likelihood of absence (petit mal) seizure resulting from pathologically low dopamine levels in accordance with experimental observations. Suppression of absence seizure activity is demonstrated when afferent and efferent BG connections to the CT system are strengthened, which is consistent with other CTBG modeling studies. The BG are demonstrated to have a suppressive effect on activity of the CTBG system near tonic-clonic seizure states, which provides insight into the reported efficacy of current treatments in BG circuits. Sleep states of the TRN are also found to suppress pathological PD activity in accordance with observations. Overall, the findings demonstrate strong parallels between coherent oscillations in generalized epilepsies and PD, and provide insights into possible comorbidities.

Monoaminergic control of brain states and sensory processing: Existing knowledge and recent insights obtained with optogenetics

Monoamines are key neuromodulators involved in a variety of physiological and pathological brain functions. Classical studies using physiological and pharmacological tools have revealed several essential aspects of monoaminergic involvement in regulating the sleep-wake cycle and influencing sensory responses but many features have remained elusive due to technical limitations. The application of optogenetic tools led to the ability of monitoring and controlling neuronal populations with unprecedented temporal precision and neurochemical specificity. Here, we focus on recent advances in revealing the roles of some monoamines in brain state control and sensory information processing. We summarize the central position of monoamines in integrating sensory processing across sleep-wake states with an emphasis on research conducted using optogenetic techniques. Finally, we discuss the limitations and perspectives of new integrated experimental approaches in understanding the modulatory mechanisms of monoaminergic systems in the mammalian brain.

A Thalamocortical Neural Mass Model of the EEG during NREM Sleep and Its Response to Auditory Stimulation

Few models exist that accurately reproduce the complex rhythms of the thalamocortical system that are apparent in measured scalp EEG and at the same time, are suitable for large-scale simulations of brain activity. Here, we present a neural mass model of the thalamocortical system during natural non-REM sleep, which is able to generate fast sleep spindles (12–15 Hz), slow oscillations (<1 Hz) and K-complexes, as well as their distinct temporal relations, and response to auditory stimuli. We show that with the inclusion of detailed calcium currents, the thalamic neural mass model is able to generate different firing modes, and validate the model with EEG-data from a recent sleep study in humans, where closed-loop auditory stimulation was applied. The model output relates directly to the EEG, which makes it a useful basis to develop new stimulation protocols.

Sleep plays a pivotal role for the consolidation of memory. While REM sleep had originally been the focus of research due to its similarity with wakefulness, more recent studies suggest that different sleep stages are responsible for the consolidation of different types of memory. To better understand the changes in neuronal dynamics between the different sleep stages, neural mass models are a valuable tool, as they relate directly to the large-scale dynamics measured by an EEG. Here, we present a model of the sleeping thalamocortical system. We first show that the isolated thalamic submodule is able to generate different oscillatory behavior found in vivo. Furthermore, the full thalamocortical model reproduces the EEG of sleep stages N2 and N3 and preserves the temporal relationship between cortical K-complexes/slow oscillations and thalamic sleep spindles. A comparison with event related potential data from a recent sleep study in humans demonstrates its possible application in predicting the outcome of external stimulation on EEG rhythms. Our study shows, that a neural mass model incorporating few key mechanisms is sufficient to reproduce the complex brain dynamics observed during sleep.

Reward Motivation Enhances Task Coding in Frontoparietal Cortex

Reward motivation often enhances task performance, but the neural mechanisms underlying such cognitive enhancement remain unclear. Here, we used a multivariate pattern analysis (MVPA) approach to test the hypothesis that motivation-related enhancement of cognitive control results from improved encoding and representation of task set information. Participants underwent two fMRI sessions of cued task switching, the first under baseline conditions, and the second with randomly intermixed reward incentive and no-incentive trials. Information about the upcoming task could be successfully decoded from cue-related activation patterns in a set of frontoparietal regions typically associated with task control. More critically, MVPA classifiers trained on the baseline session had significantly higher decoding accuracy on incentive than non-incentive trials, with decoding improvement mediating reward-related enhancement of behavioral performance. These results strongly support the hypothesis that reward motivation enhances cognitive control, by improving the discriminability of task-relevant information coded and maintained in frontoparietal brain regions.

Enhanced Sensitivity to Rapid Input Fluctuations by Nonlinear Threshold Dynamics in Neocortical Pyramidal Neurons

The way in which single neurons transform input into output spike trains has fundamental consequences for network coding. Theories and modeling studies based on standard Integrate-and-Fire models implicitly assume that, in response to increasingly strong inputs, neurons modify their coding strategy by progressively reducing their selective sensitivity to rapid input fluctuations. Combining mathematical modeling with in vitro experiments, we demonstrate that, in L5 pyramidal neurons, the firing threshold dynamics adaptively adjust the effective timescale of somatic integration in order to preserve sensitivity to rapid signals over a broad range of input statistics. For that, a new Generalized Integrate-and-Fire model featuring nonlinear firing threshold dynamics and conductance-based adaptation is introduced that outperforms state-of-the-art neuron models in predicting the spiking activity of neurons responding to a variety of in vivo-like fluctuating currents. Our model allows for efficient parameter extraction and can be analytically mapped to a Generalized Linear Model in which both the input filter—describing somatic integration—and the spike-history filter—accounting for spike-frequency adaptation—dynamically adapt to the input statistics, as experimentally observed. Overall, our results provide new insights on the computational role of different biophysical processes known to underlie adaptive coding in single neurons and support previous theoretical findings indicating that the nonlinear dynamics of the firing threshold due to Na⁺-channel inactivation regulate the sensitivity to rapid input fluctuations.

Over the last decades, a variety of simplified spiking models have been shown to achieve a surprisingly high performance in predicting the neuronal responses to in vitro somatic current injections. Because of the complex adaptive behavior featured by cortical neurons, this success is however restricted to limited stimulus ranges: model parameters optimized for a specific input regime are often inappropriate to describe the response to input currents with different statistical properties. In the present study, a new spiking neuron model is introduced that captures single-neuron computation over a wide range of input statistics and explains different aspects of the neuronal dynamics within a single framework. Our results indicate that complex forms of single neuron adaptation are mediated by the nonlinear dynamics of the firing threshold and that the input-output transformation performed by cortical pyramidal neurons can be intuitively understood in terms of an enhanced Generalized Linear Model in which both the input filter and the spike-history filter adapt to the input statistics.

Identifying the Basal Ganglia network model markers for medication-induced impulsivity in Parkinson's disease patients

Impulsivity, i.e. irresistibility in the execution of actions, may be prominent in Parkinson's disease (PD) patients who are treated with dopamine precursors or dopamine receptor agonists. In this study, we combine clinical investigations with computational modeling to explore whether impulsivity in PD patients on medication may arise as a result of abnormalities in risk, reward and punishment learning. In order to empirically assess learning outcomes involving risk, reward and punishment, four subject groups were examined: healthy controls, ON medication PD patients with impulse control disorder (PD-ON ICD) or without ICD (PD-ON non-ICD), and OFF medication PD patients (PD-OFF). A neural network model of the Basal Ganglia (BG) that has the capacity to predict the dysfunction of both the dopaminergic (DA) and the serotonergic (5HT) neuromodulator systems was developed and used to facilitate the interpretation of experimental results. In the model, the BG action selection dynamics were mimicked using a utility function based decision making framework, with DA controlling reward prediction and 5HT controlling punishment and risk predictions. The striatal model included three pools of Medium Spiny Neurons (MSNs), with D1 receptor (R) alone, D2R alone and co-expressing D1R-D2R. Empirical studies showed that reward optimality was increased in PD-ON ICD patients while punishment optimality was increased in PD-OFF patients. Empirical studies also revealed that PD-ON ICD subjects had lower reaction times (RT) compared to that of the PD-ON non-ICD patients. Computational modeling suggested that PD-OFF patients have higher punishment sensitivity, while healthy controls showed comparatively higher risk sensitivity. A significant decrease in sensitivity to punishment and risk was crucial for explaining behavioral changes observed in PD-ON ICD patients. Our results highlight the power of computational modelling for identifying neuronal circuitry implicated in learning, and its impairment in PD. The results presented here not only show that computational modelling can be used as a valuable tool for understanding and interpreting clinical data, but they also show that computational modeling has the potential to become an invaluable tool to predict the onset of behavioral changes during disease progression.

Chronic cocaine disrupts mesocortical learning mechanisms

The addictive power of drugs of abuse such as cocaine comes from their ability to hijack natural reward and plasticity mechanisms mediated by dopamine signaling in the brain. Reward learning involves burst firing of midbrain dopamine neurons in response to rewards and cues predictive of reward. The resulting release of dopamine in terminal regions is thought to act as a teaching signaling to areas such as the prefrontal cortex and striatum. In this review, we posit that a pool of extrasynaptic dopaminergic D1-like receptors activated in response to dopamine neuron burst firing serve to enable synaptic plasticity in the prefrontal cortex in response to rewards and their cues. We propose that disruptions in these mechanisms following chronic cocaine use contribute to addiction pathology, in part due to the unique architecture of the mesocortical pathway. By blocking dopamine reuptake in the cortex, cocaine elevates dopamine signaling at these extrasynaptic receptors, prolonging D1-receptor activation and the subsequent activation of intracellular signaling cascades, and thus inducing long-lasting maladaptive plasticity. These cellular adaptations may account for many of the changes in cortical function observed in drug addicts, including an enduring vulnerability to relapse. Therefore, understanding and targeting these neuroadaptations may provide cognitive benefits and help prevent relapse in human drug addicts.

Mechanisms of motivation-cognition interaction: challenges and opportunities

Recent years have seen a rejuvenation of interest in studies of motivation-cognition interactions arising from many different areas of psychology and neuroscience. The present issue of Cognitive, Affective, & Behavioral Neuroscience provides a sampling of some of the latest research from a number of these different areas. In this introductory article, we provide an overview of the current state of the field, in terms of key research developments and candidate neural mechanisms receiving focused investigation as potential sources of motivation-cognition interaction. However, our primary goal is conceptual: to highlight the distinct perspectives taken by different research areas, in terms of how motivation is defined, the relevant dimensions and dissociations that are emphasized, and the theoretical questions being targeted. Together, these distinctions present both challenges and opportunities for efforts aiming toward a more unified and cross-disciplinary approach. We identify a set of pressing research questions calling for this sort of cross-disciplinary approach, with the explicit goal of encouraging integrative and collaborative investigations directed toward them.

Differential effects of static and dynamic inputs on neuronal excitability

The intrinsic excitability of neurons is known to be dynamically regulated by activity-dependent plasticity and homeostatic mechanisms. Such processes are commonly analyzed in the context of input-output functions that describe how neurons fire in response to constant levels of current. However, it is not well understood how changes of excitability as observed under static inputs translate to the function of the same neurons in their natural synaptic environment. Here we performed a computational study and hybrid experiments on rat bed nucleus of stria terminalis neurons to compare the two scenarios. The inward rectifying Kir current (IKir) and the hyperpolarization-activated cation current (Ih) were found to be considerably more effective in regulating the firing under synaptic inputs than under static stimuli. This prediction was experimentally confirmed by dynamic-clamp insertion of a synthetic inwardly rectifying Kir current into the biological neurons. At the same time, ionic currents that activate with depolarization were more effective regulating the firing under static inputs. When two intrinsic currents are concurrently altered such as those under homeostatic regulation, the effects in firing responses under static vs. dynamic inputs can be even more contrasting. Our results show that plastic or homeostatic changes of intrinsic membrane currents can shape the current step responses of neurons and their firing under synaptic inputs in a differential manner.

Characterization of K-complexes and slow wave activity in a neural mass model

NREM sleep is characterized by two hallmarks, namely K-complexes (KCs) during sleep stage N2 and cortical slow oscillations (SOs) during sleep stage N3. While the underlying dynamics on the neuronal level is well known and can be easily measured, the resulting behavior on the macroscopic population level remains unclear. On the basis of an extended neural mass model of the cortex, we suggest a new interpretation of the mechanisms responsible for the generation of KCs and SOs. As the cortex transitions from wake to deep sleep, in our model it approaches an oscillatory regime via a Hopf bifurcation. Importantly, there is a canard phenomenon arising from a homoclinic bifurcation, whose orbit determines the shape of large amplitude SOs. A KC corresponds to a single excursion along the homoclinic orbit, while SOs are noise-driven oscillations around a stable focus. The model generates both time series and spectra that strikingly resemble real electroencephalogram data and points out possible differences between the different stages of natural sleep.

Motivation and cognitive control: from behavior to neural mechanism

Research on cognitive control and executive function has long recognized the relevance of motivational factors. Recently, however, the topic has come increasingly to center stage, with a surge of new studies examining the interface of motivation and cognitive control. In the present article we survey research situated at this interface, considering work from cognitive and social psychology and behavioral economics, but with a particular focus on neuroscience research. We organize existing findings into three core areas, considering them in the light of currently vying theoretical perspectives. Based on the accumulated evidence, we advocate for a view of control function that treats it as a domain of reward-based decision making. More broadly, we argue that neuroscientific evidence plays a critical role in understanding the mechanisms by which motivation and cognitive control interact. Opportunities for further cross-fertilization between behavioral and neuroscientific research are highlighted.

Fast oscillatory activity in the anterior cingulate cortex: dopaminergic modulation and effect of perineuronal net loss

Dopamine release in the prefrontal cortex plays a critical role in cognitive function such as working memory, attention and planning. Dopamine exerts complex modulation on excitability of pyramidal neurons and interneurons, and regulates excitatory and inhibitory synaptic transmission. Because of the complexity of this modulation, it is difficult to fully comprehend the effect of dopamine on neuronal network activity. In this study, we investigated the effect of dopamine on local high-frequency oscillatory neuronal activity (in β band) in slices of the mouse anterior cingulate cortex (ACC). We found that dopamine enhanced the power of these oscillations induced by kainate and carbachol, but did not affect their peak frequency. Activation of D2R and in a lesser degree D1R increased the oscillation power, while activation of D4R had no effect. These high-frequency oscillations in the ACC relied on both phasic inhibitory and excitatory transmission and functional gap junctions. Thus, dopamine released in the ACC promotes high-frequency synchronized local cortical activity which is known to favor information transfer, fast selection and binding of distributed neuronal responses. Finally, the power of these oscillations was significantly enhanced after degradation of the perineuronal nets (PNNs) enwrapping most parvalbumin interneurons. This study provides new insights for a better understanding of the abnormal prefrontal gamma activity in schizophrenia (SZ) patients who display prefrontal anomalies of both the dopaminergic system and the PNNs.

Touchscreen-paradigm for mice reveals cross-species evidence for an antagonistic relationship of cognitive flexibility and stability

The abilities to either flexibly adjust behavior according to changing demands (cognitive flexibility) or to maintain it in the face of potential distractors (cognitive stability) are critical for adaptive behavior in many situations. Recently, a novel human paradigm has found individual differences of cognitive flexibility and stability to be related to common prefrontal networks. The aims of the present study were, first, to translate this paradigm from humans to mice and, second, to test conceptual predictions of a computational model of prefrontal working memory mechanisms, the Dual State Theory, which assumes an antagonistic relation between cognitive flexibility and stability. Mice were trained in a touchscreen-paradigm to discriminate visual cues. The task involved “ongoing” and cued “switch” trials. In addition distractor cues were interspersed to test the ability to resist distraction, and an ambiguous condition assessed the spontaneous switching between two possible responses without explicit cues. While response times did not differ substantially between conditions, error rates (ER) increased from the “ongoing” baseline condition to the most complex condition, where subjects were required to switch between two responses in the presence of a distracting cue. Importantly, subjects switching more often spontaneously were found to be more distractible by task irrelevant cues, but also more flexible in situations, where switching was required. These results support a dichotomy of cognitive flexibility and stability as predicted by the Dual State Theory. Furthermore, they replicate critical aspects of the human paradigm, which indicates the translational potential of the testing procedure and supports the use of touchscreen procedures in preclinical animal research.

Behavioral abnormalities and circuit defects in the basal ganglia of a mouse model of 16p11.2 deletion syndrome

A deletion on human chromosome 16p11.2 is associated with autism spectrum disorders. We deleted the syntenic region on mouse chromosome 7F3. MRI and high-throughput single-cell transcriptomics revealed anatomical and cellular abnormalities, particularly in cortex and striatum of juvenile mutant mice (16p11(+/-)). We found elevated numbers of striatal medium spiny neurons (MSNs) expressing the dopamine D2 receptor (Drd2(+)) and fewer dopamine-sensitive (Drd1(+)) neurons in deep layers of cortex. Electrophysiological recordings of Drd2(+) MSN revealed synaptic defects, suggesting abnormal basal ganglia circuitry function in 16p11(+/-) mice. This is further supported by behavioral experiments showing hyperactivity, circling, and deficits in movement control. Strikingly, 16p11(+/-) mice showed a complete lack of habituation reminiscent of what is observed in some autistic individuals. Our findings unveil a fundamental role of genes affected by the 16p11.2 deletion in establishing the basal ganglia circuitry and provide insights in the pathophysiology of autism.

An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning

Although empirical and neural studies show that serotonin (5HT) plays many functional roles in the brain, prior computational models mostly focus on its role in behavioral inhibition. In this study, we present a model of risk based decision making in a modified Reinforcement Learning (RL)-framework. The model depicts the roles of dopamine (DA) and serotonin (5HT) in Basal Ganglia (BG). In this model, the DA signal is represented by the temporal difference error (δ), while the 5HT signal is represented by a parameter (α) that controls risk prediction error. This formulation that accommodates both 5HT and DA reconciles some of the diverse roles of 5HT particularly in connection with the BG system. We apply the model to different experimental paradigms used to study the role of 5HT: (1) Risk-sensitive decision making, where 5HT controls risk assessment, (2) Temporal reward prediction, where 5HT controls time-scale of reward prediction, and (3) Reward/Punishment sensitivity, in which the punishment prediction error depends on 5HT levels. Thus the proposed integrated RL model reconciles several existing theories of 5HT and DA in the BG.

Action, time and the basal ganglia

The ability to control the speed of movement is compromised in neurological disorders involving the basal ganglia, a set of subcortical cerebral nuclei that receive prominent dopaminergic projections from the midbrain. For example, bradykinesia, slowness of movement, is a major symptom of Parkinson's disease, whereas rapid tics are observed in patients with Tourette syndrome. Recent experimental work has also implicated dopamine (DA) and the basal ganglia in action timing. Here, I advance the hypothesis that the basal ganglia control the rate of change in kinaesthetic perceptual variables. In particular, the sensorimotor cortico-basal ganglia network implements a feedback circuit for the control of movement velocity. By modulating activity in this network, DA can change the gain of velocity reference signals. The lack of DA thus reduces the output of the velocity control system which specifies the rate of change in body configurations, slowing the transition from one body configuration to another.

Dopaminergic control of long-term depression/long-term potentiation threshold in prefrontal cortex

Long-term memory in the prefrontal cortex is a necessary component of adaptive executive control and is strongly modulated by dopamine. However, the functional significance of this dopaminergic modulation remains elusive. In vitro experimental results on dopamine-dependent shaping of prefrontal long-term plasticity often appear inconsistent and, altogether, draw a complicated picture. It is also generally difficult to relate these findings to in vivo observations given strong differences between the two experimental conditions. This study presents a unified view of the functional role of dopamine in the prefrontal cortex by framing it within the Bienenstock-Cooper-Munro theory of cortical plasticity. We investigate dopaminergic modulation of long-term plasticity through a multicompartment Hodgkin-Huxley model of a prefrontal pyramidal neuron. Long-term synaptic plasticity in the model is governed by a calcium- and dopamine-dependent learning rule, in which dopamine exerts its action via D1 and D2 dopamine receptors in a concentration-dependent manner. Our results support a novel function of dopamine in the prefrontal cortex, namely that it controls the synaptic modification threshold between long-term depression and potentiation in pyramidal neurons. The proposed theoretical framework explains a wide range of experimental results and provides a link between in vitro and in vivo studies of dopaminergic plasticity modulation. It also suggests that dopamine may constitute a new player in metaplastic and homeostatic processes in the prefrontal cortex.

Dopamine regulates two classes of primate prefrontal neurons that represent sensory signals

The lateral prefrontal cortex (PFC), a hub of higher-level cognitive processing, is strongly modulated by midbrain dopamine (DA) neurons. The cellular mechanisms have been comprehensively studied in the context of short-term memory, but little is known about how DA regulates sensory inputs to PFC that precede and give rise to such memory activity. By preparing recipient cortical circuits for incoming signals, DA could be a powerful determinant of downstream cognitive processing. Here, we tested the hypothesis that prefrontal DA regulates the representation of sensory signals that are required for perceptual decisions. In rhesus monkeys trained to report the presence or absence of visual stimuli at varying levels of contrast, we simultaneously recorded extracellular single-unit activity and applied DA to the immediate vicinity of the neurons by micro-iontophoresis. We found that DA modulation of prefrontal neurons is not uniform but tailored to specialized neuronal classes. In one population of neurons, DA suppressed activity with high temporal precision but preserved signal/noise ratio. Neurons in this group had short visual response latencies and comprised all recorded narrow-spiking, putative interneurons. In a distinct population, DA increased excitability and enhanced signal/noise ratio by reducing response variability. These neurons had longer visual response latencies and were composed exclusively of broad-spiking, putative pyramidal neurons. By gating sensory inputs to PFC and subsequently strengthening the representation of sensory signals, DA might play an important role in shaping how the PFC initiates appropriate behavior in response to changes in the sensory environment.

Induction and modulation of persistent activity in a layer V PFC microcircuit model

Working memory refers to the temporary storage of information and is strongly associated with the prefrontal cortex (PFC). Persistent activity of cortical neurons, namely the activity that persists beyond the stimulus presentation, is considered the cellular correlate of working memory. Although past studies suggested that this type of activity is characteristic of large scale networks, recent experimental evidence imply that small, tightly interconnected clusters of neurons in the cortex may support similar functionalities. However, very little is known about the biophysical mechanisms giving rise to persistent activity in small-sized microcircuits in the PFC. Here, we present a detailed biophysically—yet morphologically simplified—microcircuit model of layer V PFC neurons that incorporates connectivity constraints and is validated against a multitude of experimental data. We show that (a) a small-sized network can exhibit persistent activity under realistic stimulus conditions. (b) Its emergence depends strongly on the interplay of dADP, NMDA, and GABA_B currents. (c) Although increases in stimulus duration increase the probability of persistent activity induction, variability in the stimulus firing frequency does not consistently influence it. (d) Modulation of ionic conductances (I_h, I_D, I_sAHP, I_caL, I_caN, I_caR) differentially controls persistent activity properties in a location dependent manner. These findings suggest that modulation of the microcircuit's firing characteristics is achieved primarily through changes in its intrinsic mechanism makeup, supporting the hypothesis of multiple bi-stable units in the PFC. Overall, the model generates a number of experimentally testable predictions that may lead to a better understanding of the biophysical mechanisms of persistent activity induction and modulation in the PFC.