Why skill matters

Increasing Motivation Increases Intelligibility Benefits of Perceptual Training in Dysarthria

PURPOSE

Perceptual training offers a promising, listener-targeted option for improving intelligibility of dysarthric speech. Cognitive resources are required for learning, and theoretical models of listening effort and engagement account for a role of listener motivation in allocation of such resources. Here, we manipulate training instructions to enhance motivation to test the hypothesis that increased motivation increases the intelligibility benefits of perceptual training.

METHOD

Across two data collection sites, which differed with respect to many elements of study design including age of speaker with dysarthria, dysarthria type and severity, type of testing and training stimuli, and participant compensation, 84 neurotypical adults were randomly assigned to one of two training instruction conditions: enhanced instructions or standard instructions. Intelligibility, quantified as percent words correct, was measured before and after training.

RESULTS

Listeners who received the enhanced instructions achieved greater intelligibility improvements from training relative to listeners who received the standard instructions. This result was robust across data collection sites and the many differences in methodology.

CONCLUSIONS

This study provides evidence for the role of motivation in improved understanding of dysarthric speech-increasing motivation increases allocation of cognitive resources to the learning process, resulting in improved mapping of the degraded speech signal. This provides empirical support for theoretical models of listening effort and engagement. Clinically, the results show that a simple addition to the training instructions can elevate learning outcomes.

Implication of regional selectivity of dopamine deficits in impaired suppressing of involuntary movements in Parkinson's disease

To improve the initiation and speed of intended action, one of the crucial mechanisms is suppressing unwanted movements that interfere with goal-directed behavior, which is observed relatively aberrant in Parkinson's disease patients. Recent research has highlighted that dopamine deficits in Parkinson's disease predominantly occur in the caudal lateral part of the substantia nigra pars compacta (SNc) in human patients. We previously found two parallel circuits within the basal ganglia, primarily divided into circuits mediated by the rostral medial part and caudal lateral part of the SNc dopamine neurons. We have further discovered that the indirect pathway in caudal basal ganglia circuits, facilitated by the caudal lateral part of the SNc dopamine neurons, plays a critical role in suppressing unnecessary involuntary movements when animals perform voluntary goal-directed actions. We thus explored recent research in humans and non-human primates focusing on the distinct functions and networks of the caudal lateral part of the SNc dopamine neurons to elucidate the mechanisms involved in the impairment of suppressing involuntary movements in Parkinson's disease patients.

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

Updating functional brain units: Insights far beyond Luria

This paper reviews Luria's model of the three functional units of the brain. To meet this objective, several issues were reviewed: the theory of functional systems and the contributions of phylogenesis and embryogenesis to the brain's functional organization. This review revealed several facts. In the first place, the relationship/integration of basic homeostatic needs with complex forms of behavior. Secondly, the multi-scale hierarchical and distributed organization of the brain and interactions between cells and systems. Thirdly, the phylogenetic role of exaptation, especially in basal ganglia and cerebellum expansion. Finally, the tripartite embryogenetic organization of the brain: rhinic, limbic/paralimbic, and supralimbic zones. Obviously, these principles of brain organization are in contradiction with attempts to establish separate functional brain units. The proposed new model is made up of two large integrated complexes: a primordial-limbic complex (Luria's Unit I) and a telencephalic-cortical complex (Luria's Units II and III). As a result, five functional units were delineated: Unit I. Primordial or preferential (brainstem), for life-support, behavioral modulation, and waking regulation; Unit II. Limbic and paralimbic systems, for emotions and hedonic evaluation (danger and relevance detection and contribution to reward/motivational processing) and the creation of cognitive maps (contextual memory, navigation, and generativity [imagination]); Unit III. Telencephalic-cortical, for sensorimotor and cognitive processing (gnosis, praxis, language, calculation, etc.), semantic and episodic (contextual) memory processing, and multimodal conscious agency; Unit IV. Basal ganglia systems, for behavior selection and reinforcement (reward-oriented behavior); Unit V. Cerebellar systems, for the prediction/anticipation (orthometric supervision) of the outcome of an action. The proposed brain units are nothing more than abstractions within the brain's simultaneous and distributed physiological processes. As function transcends anatomy, the model necessarily involves transition and overlap between structures. Beyond the classic approaches, this review includes information on recent systemic perspectives on functional brain organization. The limitations of this review are discussed.

One Size Does Not Fit All: Idiographic Computational Models Reveal Individual Differences in Learning and Meta-Learning Strategies

Complex skill learning depends on the joint contribution of multiple interacting systems: working memory (WM), declarative long-term memory (LTM) and reinforcement learning (RL). The present study aims to understand individual differences in the relative contributions of these systems during learning. We built four idiographic, ACT-R models of performance on the stimulus-response learning, Reinforcement Learning Working Memory task. The task consisted of short 3-image, and long 6-image, feedback-based learning blocks. A no-feedback test phase was administered after learning, with an interfering task inserted between learning and test. Our four models included two single-mechanism RL and LTM models, and two integrated RL-LTM models: (a) RL-based meta-learning, which selects RL or LTM to learn based on recent success, and (b) a parameterized RL-LTM selection model at fixed proportions independent of learning success. Each model was the best fit for some proportion of our learners (LTM: 68.7%, RL: 4.8%, Meta-RL: 13.25%, bias-RL:13.25% of participants), suggesting fundamental differences in the way individuals deploy basic learning mechanisms, even for a simple stimulus-response task. Finally, long-term declarative memory seems to be the preferred learning strategy for this task regardless of block length (3- vs 6-image blocks), as determined by the large number of subjects whose learning characteristics were best captured by the LTM only model, and a preference for LTM over RL in both of our integrated-models, owing to the strength of our idiographic approach.

Layer 5 Intratelencephalic Neurons in the Motor Cortex Stably Encode Skilled Movement

The primary motor cortex (M1) and the dorsal striatum play a critical role in motor learning and the retention of learned behaviors. Motor representations of corticostriatal ensembles emerge during motor learning. In the coordinated reorganization of M1 and the dorsal striatum for motor learning, layer 5a (L5a) which connects M1 to the ipsilateral and contralateral dorsal striatum, should be a key layer. Although M1 L5a neurons represent movement-related activity in the late stage of learning, it is unclear whether the activity is retained as a memory engram. Here, using Tlx3-Cre male transgenic mice, we conducted two-photon calcium imaging of striatum-projecting L5a intratelencephalic (IT) neurons in forelimb M1 during late sessions of a self-initiated lever-pull task and in sessions after 6 d of nontraining following the late sessions. We found that trained male animals exhibited stable motor performance before and after the nontraining days. At the same time, we found that M1 L5a IT neurons strongly represented the well-learned forelimb movement but not uninstructed orofacial movements. A subset of M1 L5a IT neurons consistently coded the well-learned forelimb movement before and after the nontraining days. Inactivation of M1 IT neurons after learning impaired task performance when the lever was made heavier or when the target range of the pull distance was narrowed. These results suggest that a subset of M1 L5a IT neurons continuously represent skilled movement after learning and serve to fine-tune the kinematics of well-learned movement.SIGNIFICANCE STATEMENT Motor memory persists even when it is not used for a while. IT neurons in L5a of the M1 gradually come to represent skilled forelimb movements during motor learning. However, it remains to be determined whether these changes persist over a long period and how these neurons contribute to skilled movements. Here, we show that a subset of M1 L5a IT neurons retain information for skilled forelimb movements even after nontraining days. Furthermore, suppressing the activity of these neurons during skilled forelimb movements impaired behavioral stability and adaptability. Our results suggest the importance of M1 L5a IT neurons for tuning skilled forelimb movements over a long period.

Prefrontal cortex encodes value pop-out in visual search

Recent evidence demonstrates that long-term object value association can enhance visual search efficiency, a phenomenon known as value pop-out. However, the neural mechanism underlying this effect is not fully understood. Given the known role of the ventrolateral prefrontal cortex (vlPFC) in visual search and value memory, we recorded its single-unit activity (n = 526) in two macaque monkeys while they engaged in the value-driven search. Monkeys had to determine whether a high-value target was present within a variable number of low-value objects. Differential neural firing, as well as gamma-band power, indicated the presence of a target within ∼150ms of display onset. Notably, this differential activity was negatively correlated with search time and had reduced set-size dependence during efficient search. On the other hand, neural firing and its variability were higher in inefficient search. These findings implicate vlPFC in rapid detection of valuable targets which would be a crucial skill in competitive environments.

Neuronal mechanism of the encoding of socially familiar faces in the striatum tail

Although we can quickly locate a familiar person even in a crowd, the underlying neuronal mechanism remains unclear. Recently, we found that the striatum tail (STRt), which is part of the basal ganglia, is sensitive to long-term reward history. Here, we show that long-term value-coding neurons are involved in the detection of socially familiar faces. Many STRt neurons respond to facial images, especially to those of socially familiar persons. Additionally, we found that these face-responsive neurons also encode the stable values of many objects based on long-term reward experiences. Interestingly, the strength of neuronal modulation of social familiarity bias (familiar or unfamiliar) and object value bias (high-valued or low-valued) were positively correlated. These results suggest that both social familiarity and stable object-value information are mediated by a common neuronal mechanism. This mechanism may contribute to the rapid detection of familiar faces in real-world contexts.

Motivational salience drives habitual gazes during value memory retention and facilitates relearning of forgotten value

A habitual gaze is critical to efficiently identify and exploit valuable objects. However, it is unclear what salience components drive the habitual gaze choice. Here, we trained subjects to assign positive, neutral, and negative values to objects and found that motivational salience guided habitual gaze choices over 30 days of memory retention. The habitual preference for negatively valued objects emerged during memory retention. This habitual choice was not explained by a general model with salience components driven by physical features of objects and the rank of learned values. Instead, this is better explained by a model that contains an additional component driven by motivational salience. In a simulated value-forgotten condition, these motivational salience-based habitual choices facilitated re-learning. Our data indicate that after long-term retention, habitual gaze results from increased motivational salience, potentially facilitating the re-learning of forgotten values.

Laser stimulation of the skin for quantitative study of decision-making and motivation

Neuroeconomics studies how decision-making is guided by the value of rewards and punishments. But to date, little is known about how noxious experiences impact decisions. A challenge is the lack of an aversive stimulus that is dynamically adjustable in intensity and location, readily usable over many trials in a single experimental session, and compatible with multiple ways to measure neuronal activity. We show that skin laser stimulation used in human studies of aversion can be used for this purpose in several key animal models. We then use laser stimulation to study how neurons in the orbitofrontal cortex (OFC), an area whose many roles include guiding decisions among different rewards, encode the value of rewards and punishments. We show that some OFC neurons integrated the positive value of rewards with the negative value of aversive laser stimulation, suggesting that the OFC can play a role in more complex choices than previously appreciated.

Dissecting motor skill acquisition: Spatial coordinates take precedence

Practicing a previously unknown motor sequence often leads to the consolidation of motor chunks, which enable its accurate execution at increasing speeds. Recent imaging studies suggest the function of these structures to be more related to the encoding, storage, and retrieval of sequences rather than their sole execution. We found that optimal motor skill acquisition prioritizes the storage of the spatial features of the sequence in memory over its rapid execution early in training, as proposed by Hikosaka in 1999. This process, seemingly diminished in older adults, was partially restored by anodal transcranial direct current stimulation over the motor cortex, as shown by a sharp improvement in accuracy and an earlier yet gradual emergence of motor chunks. These results suggest that the emergence of motor chunks is preceded by the storage of the sequence in memory but is not its direct consequence; rather, these structures depend on, and result from, motor practice.

Interactions between ventrolateral prefrontal and anterior cingulate cortex during learning and behavioural change

Hypotheses and beliefs guide credit assignment - the process of determining which previous events or actions caused an outcome. Adaptive hypothesis formation and testing are crucial in uncertain and changing environments in which associations and meanings are volatile. Despite primates' abilities to form and test hypotheses, establishing what is causally responsible for the occurrence of particular outcomes remains a fundamental challenge for credit assignment and learning. Hypotheses about what surprises are due to stochasticity inherent in an environment as opposed to real, systematic changes are necessary for identifying the environment's predictive features, but are often hard to test. We review evidence that two highly interconnected frontal cortical regions, anterior cingulate cortex and ventrolateral prefrontal area 47/12o, provide a biological substrate for linking two crucial components of hypothesis-formation and testing: the control of information seeking and credit assignment. Neuroimaging, targeted disruptions, and neurophysiological studies link an anterior cingulate - 47/12o circuit to generation of exploratory behaviour, non-instrumental information seeking, and interpretation of subsequent feedback in the service of credit assignment. Our observations support the idea that information seeking and credit assignment are linked at the level of neural circuits and explain why this circuit is important for ensuring behaviour is flexible and adaptive.

How the value of the environment controls persistence in visual search

Classic foraging theory predicts that humans and animals aim to gain maximum reward per unit time. However, in standard instrumental conditioning tasks individuals adopt an apparently suboptimal strategy: they respond slowly when the expected value is low. This reward-related bias is often explained as reduced motivation in response to low rewards. Here we present evidence this behavior is associated with a complementary increased motivation to search the environment for alternatives. We trained monkeys to search for reward-related visual targets in environments with different values. We found that the reward-related bias scaled with environment value, was consistent with persistent searching after the target was already found, and was associated with increased exploratory gaze to objects in the environment. A novel computational model of foraging suggests that this search strategy could be adaptive in naturalistic settings where both environments and the objects within them provide partial information about hidden, uncertain rewards.

Coupling between motor cortex and striatum increases during sleep over long-term skill learning

The strength of cortical connectivity to the striatum influences the balance between behavioral variability and stability. Learning to consistently produce a skilled action requires plasticity in corticostriatal connectivity associated with repeated training of the action. However, it remains unknown whether such corticostriatal plasticity occurs during training itself or 'offline' during time away from training, such as sleep. Here, we monitor the corticostriatal network throughout long-term skill learning in rats and find that non-rapid-eye-movement (NREM) sleep is a relevant period for corticostriatal plasticity. We first show that the offline activation of striatal NMDA receptors is required for skill learning. We then show that corticostriatal functional connectivity increases offline, coupled to emerging consistent skilled movements, and coupled cross-area neural dynamics. We then identify NREM sleep spindles as uniquely poised to mediate corticostriatal plasticity, through interactions with slow oscillations. Our results provide evidence that sleep shapes cross-area coupling required for skill learning.

Common coding of expected value and value uncertainty memories in the prefrontal cortex and basal ganglia output

Recent evidence implicates both basal ganglia and ventrolateral prefrontal cortex (vlPFC) in encoding value memories. However, comparative roles of cortical and basal nodes in value memory are not well understood. Here, single-unit recordings in vlPFC and substantia nigra reticulata (SNr), within macaque monkeys, revealed a larger value signal in SNr that was nevertheless correlated with and had a comparable onset to the vlPFC value signal. The value signal was maintained for many objects (>90) many weeks after reward learning and was resistant to extinction in both regions and to repetition suppression in vlPFC. Both regions showed comparable granularity in encoding expected value and value uncertainty, which was paralleled by enhanced gaze bias during free viewing. The value signal dynamics in SNr could be predicted by combining responses of vlPFC neurons according to their value preferences consistent with a scheme in which cortical neurons reached SNr via direct and indirect pathways.

Decreased modulation of segregated SEEKING and selective attention systems in chronic insomnia

Sleep-related attentional bias and instinctual craving-sleep status may be associated with value-driven selective attention network and SEEKING system. We hypothesized that the two networks might be important components and underlie etiology of inability to initiate or/and maintain sleep in patients with chronic insomnia (PIs). Our aim is to investigate whether frequency-frequency couplings(temporal and spatial coupling, and differences of a set of imaging parameters) could elevate the sensibility to characterize the two insomnia-related networks in studying their relationships with sleep parameters and post-insomnia emotions. Forty-eight PIs and 48 status-matched good sleepers were requested to complete sleep and emotion-related questionnaires. Receiver operating characteristic curve was used to calculate the discriminatory power of a set of parameters. Granger causality and mediating causality analysis were used to address the causal relationships between the two networks and sleep/emotion-related parameters. Frequency-frequency couplings could characterize the two networks with high discriminatory power (AUC, 0.951; sensitivity, 87.5%; specificity, 95.8%), which suggested that the frequency-frequency couplings could be served as a useful biomarker to address the insomnia-related brain networks. Functional deficits of the SEEKING system played decreased mediator acting in post-insomnia negative emotions (decreased frequency-frequency coupling). Functional hyperarousal of the value-driven attention network played decreased mediator acting in sleep regulation (increased frequency-frequency coupling). Granger causality analysis showed decreased causal effect connectivity between and within the two networks. The between-network causal effect connectivity segregation played decreased mediator acting in sleep regulation (decreased connectivity). These findings suggest that the functional deficits and segregation of the two systems may underlie etiology of PIs.

Prior context influences motor brain areas in an auditory oddball task and prefrontal cortex multitasking modelling

In this study, the relationship of orienting of attention, motor control and the Stimulus- (SDN) and Goal-Driven Networks (GDN) was explored through an innovative method for fMRI analysis considering all voxels in four experimental conditions: standard target (Goal; G), novel (N), neutral (Z) and noisy target (NG). First, average reaction times (RTs) for each condition were calculated. In the second-level analysis, 'distracted' participants, as indicated by slower RTs, evoked brain activations and differences in both hemispheres' neural networks for selective attention, while the participants, as a whole, demonstrated mainly left cortical and subcortical activations. A context analysis was run in the behaviourally distracted participant group contrasting the trials immediately prior to the G trials, namely one of the Z, N or NG conditions, i.e. Z.G, N.G, NG.G. Results showed different prefrontal activations dependent on prior context in the auditory modality, recruiting between 1 to 10 prefrontal areas. The higher the motor response and influence of the previous novel stimulus, the more prefrontal areas were engaged, which extends the findings of hierarchical studies of prefrontal control of attention and better explains how auditory processing interferes with movement. Also, the current study addressed how subcortical loops and models of previous motor response affected the signal processing of the novel stimulus, when this was presented laterally or simultaneously with the target. This multitasking model could enhance our understanding on how an auditory stimulus is affecting motor responses in a way that is self-induced, by taking into account prior context, as demonstrated in the standard condition and as supported by Pulvinar activations complementing visual findings. Moreover, current BCI works address some multimodal stimulus-driven systems.

Neurotransmitters, Cell Types, and Circuit Mechanisms of Motor Skill Learning and Clinical Applications

Animals acquire motor skills to better survive and adapt to a changing environment. The ability to learn novel motor actions without disturbing learned ones is essential to maintaining a broad motor repertoire. During motor learning, the brain makes a series of adjustments to build novel sensory–motor relationships that are stored within specific circuits for long-term retention. The neural mechanism of learning novel motor actions and transforming them into long-term memory still remains unclear. Here we review the latest findings with regard to the contributions of various brain subregions, cell types, and neurotransmitters to motor learning. Aiming to seek therapeutic strategies to restore the motor memory in relative neurodegenerative disorders, we also briefly describe the common experimental tests and manipulations for motor memory in rodents.

Environment-based object values learned by local network in the striatum tail

Choosing good objects is a fundamental behavior for all animals, to which the basal ganglia (BG) contribute extensively. However, the object choice needs to be changed in different environments. The mechanism of object choice is based on the neuronal circuits originating from output neurons (MSNs) in the striatum. We found that the environment information is provided by fast-spiking interneurons (FSIs) connecting to the MSN circuit. More critically, the experimental reduction of the FSI-input to MSNs disabled the monkey to learn the environment-based object choice. This proved that the object choice controlled by the downstream BG circuit is modulated by the environmental context controlled by the internal circuits in the top of BG circuit. This is important for our flexible decision.

Basal ganglia contribute to object-value learning, which is critical for survival. The underlying neuronal mechanism is the association of each object with its rewarding outcome. However, object values may change in different environments and we then need to choose different objects accordingly. The mechanism of this environment-based value learning is unknown. To address this question, we created an environment-based value task in which the value of each object was reversed depending on the two scene-environments (X and Y). After experiencing this task repeatedly, the monkeys became able to switch the choice of object when the scene-environment changed unexpectedly. When we blocked the inhibitory input from fast-spiking interneurons (FSIs) to medium spiny projection neurons (MSNs) in the striatum tail by locally injecting IEM-1460, the monkeys became unable to learn scene-selective object values. We then studied the mechanism of the FSI-MSN connection. Before and during this learning, FSIs responded to the scenes selectively, but were insensitive to object values. In contrast, MSNs became able to discriminate the objects (i.e., stronger response to good objects), but this occurred clearly in one of the two scenes (X or Y). This was caused by the scene-selective inhibition by FSI. As a whole, MSNs were divided into two groups that were sensitive to object values in scene X or in scene Y. These data indicate that the local network of striatum tail controls the learning of object values that are selective to the scene-environment. This mechanism may support our flexible switching behavior in various environments.

How Outcome Uncertainty Mediates Attention, Learning, and Decision-Making

Animals and humans evolved sophisticated nervous systems that endowed them with the ability to form internal-models or beliefs and make predictions about the future to survive and flourish in a world in which future outcomes are often uncertain. Crucial to this capacity is the ability to adjust behavioral and learning policies in response to the level of uncertainty. Until recently, the neuronal mechanisms that could underlie such uncertainty-guided control have been largely unknown. In this review, I discuss newly discovered neuronal circuits in primates that represent uncertainty about future rewards and propose how they guide information-seeking, attention, decision-making, and learning to help us survive in an uncertain world. Lastly, I discuss the possible relevance of these findings to learning in artificial systems.

Spatiotemporal dissociation of fMRI activity in the caudate nucleus underlies human de novo motor skill learning

Numerous real-world motor skills require learning arbitrary relationships between actions and their consequences from scratch. However, little is understood about the neural signatures of de novo motor learning and associated individual variability. In a longitudinal fMRI experiment, where participants learned to control a cursor by moving fingers, we found a gradual transition of performance-related activity from the head to tail of the caudate nucleus. This finding reflects the flexible and stable reward representations in the head and tail, respectively. Additionally, intrinsic cortico-caudate connectivity predicted better learners with weaker head–prefrontal and stronger tail–sensorimotor interactions. The present study provides unprecedented insight into de novo motor learning, which may contribute to the understanding of motor-related disorders, and infant learning.

Motor skill learning involves a complex process of generating novel movement patterns guided by evaluative feedback, such as a reward. Previous literature has suggested anteroposteriorly separated circuits in the striatum to be implicated in early goal-directed and later automatic stages of motor skill learning, respectively. However, the involvement of these circuits has not been well elucidated in human de novomotor skill learning, which requires learning arbitrary action–outcome associations and value-based action selection. To investigate this issue, we conducted a human functional MRI (fMRI) experiment in which participants learned to control a computer cursor by manipulating their right fingers. We discovered a double dissociation of fMRI activity in the anterior and posterior caudate nucleus, which was associated with performance in the early and late learning stages. Moreover, cognitive and sensorimotor cortico-caudate interactions predicted individual learning performance. Our results suggest parallel cortico-caudate networks operating in different stages of human de novomotor skill learning.

Optogenetic manipulation of a value-coding pathway from the primate caudate tail facilitates saccadic gaze shift

In the primate basal ganglia, the caudate tail (CDt) encodes the historical values (good or bad) of visual objects (i.e., stable values), and electrical stimulation of CDt evokes saccadic eye movements. However, it is still unknown how output from CDt conveys stable value signals to govern behavior. Here, we apply a pathway-selective optogenetic manipulation to elucidate how such value information modulates saccades. We express channelrhodopsin-2 in CDt delivered by viral vector injections. Selective optical activation of CDt-derived terminals in the substantia nigra pars reticulata (SNr) inhibits SNr neurons. Notably, these SNr neurons show inhibitory responses to good objects. Furthermore, the optical stimulation causes prolonged excitation of visual-saccadic neurons in the superior colliculus (SC), and induces contralateral saccades. These SC neurons respond more strongly to good than to bad objects in the contralateral hemifield. The present results demonstrate that CDt facilitates saccades toward good objects by serial inhibitory pathways through SNr.

Learning in Visual Regions as Support for the Bias in Future Value-Driven Choice

Reinforcement learning can bias decision-making toward the option with the highest expected outcome. Cognitive learning theories associate this bias with the constant tracking of stimulus values and the evaluation of choice outcomes in the striatum and prefrontal cortex. Decisions however first require processing of sensory input, and to date, we know far less about the interplay between learning and perception. This functional magnetic resonance imaging study (N = 43) relates visual blood oxygen level-dependent (BOLD) responses to value beliefs during choice and signed prediction errors after outcomes. To understand these relationships, which co-occurred in the striatum, we sought relevance by evaluating the prediction of future value-based decisions in a separate transfer phase where learning was already established. We decoded choice outcomes with a 70% accuracy with a supervised machine learning algorithm that was given trial-by-trial BOLD from visual regions alongside more traditional motor, prefrontal, and striatal regions. Importantly, this decoding of future value-driven choice outcomes again highlighted an important role for visual activity. These results raise the intriguing possibility that the tracking of value in visual cortex is supportive for the striatal bias toward the more valued option in future choice.

Sensory representations in the striatum provide a temporal reference for learning and executing motor habits

Previous studies indicate that the dorsolateral striatum (DLS) integrates sensorimotor information from cortical and thalamic regions to learn and execute motor habits. However, the exact contribution of sensory representations to this process is still unknown. Here we explore the role of the forelimb somatosensory flow in the DLS during the learning and execution of motor habits. First, we compare rhythmic somesthetic representations in the DLS and primary somatosensory cortex in anesthetized rats, and find that sequential and temporal stimuli contents are more strongly represented in the DLS. Then, using a behavioral protocol in which rats developed a stereotyped motor sequence, functional disconnection experiments, and pharmacologic and optogenetic manipulations in apprentice and expert animals, we reveal that somatosensory thalamic- and cortical-striatal pathways are indispensable for the temporal component of execution. Our results indicate that the somatosensory flow in the DLS provides the temporal reference for the development and execution of motor habits.

The authors combine anatomical mapping, electrophysiological recordings, lesions, and pharmacological and optogenetic manipulations in rats to examine the role of forelimb somatosensory flow in the dorsolateral striatum in the learning and execution of motor habits.

Emergent modular neural control drives coordinated motor actions

A remarkable feature of motor control is the ability to coordinate movements across distinct body parts into a consistent, skilled action. To reach and grasp an object, 'gross' arm and 'fine' dexterous movements must be coordinated as a single action. How the nervous system achieves this coordination is currently unknown. One possibility is that, with training, gross and fine movements are co-optimized to produce a coordinated action; alternatively, gross and fine movements may be modularly refined to function together. To address this question, we recorded neural activity in the primary motor cortex and dorsolateral striatum during reach-to-grasp skill learning in rats. During learning, the refinement of fine and gross movements was behaviorally and neurally dissociable. Furthermore, inactivation of the primary motor cortex and dorsolateral striatum had distinct effects on skilled fine and gross movements. Our results indicate that skilled movement coordination is achieved through emergent modular neural control.

Emotional Theory of Rationality

In recent decades, the existence of a close relationship between emotional phenomena and rational processes has certainly been established, yet there is still no unified definition or effective model to describe them. To advance our understanding of the mechanisms governing the behavior of living beings, we must integrate multiple theories, experiments, and models from both fields. In this article we propose a new theoretical framework that allows integrating and understanding the emotion-cognition duality, from a functional point of view. Based on evolutionary principles, our reasoning adds to the definition and understanding of emotion, justifying its origin, explaining its mission and dynamics, and linking it to higher cognitive processes, mainly with attention, cognition, decision-making, and consciousness. According to our theory, emotions are the mechanism for brain function optimization, aside from the contingency and stimuli prioritization system. As a result of this approach, we have developed a dynamic systems-level model capable of providing plausible explanations for certain psychological and behavioral phenomena and establishing a new framework for the scientific definition of some fundamental psychological terms.

Influence of reward learning on visual attention and eye movements in a naturalistic environment: A virtual reality study

Rewards constitute crucial signals that motivate approach behavior and facilitate the perceptual processing of objects associated with favorable outcomes in past encounters. Reward-related influences on perception and attention have been reliably observed in studies where a reward is paired with a unidimensional low-level visual feature, such as the color or orientation of a line in visual search tasks. However, our environment is drastically different and composed of multidimensional and changing visual features, encountered in complex and dynamic scenes. Here, we designed an immersive virtual reality (VR) experiment using a 4-frame CAVE system to investigate the impact of rewards on attentional orienting and gaze patterns in a naturalistic and ecological environment. Forty-one healthy participants explored a virtual forest and responded to targets appearing on either the left or right side of their path. To test for reward-induced biases in spatial orienting, targets on one side were associated with high reward, whereas those on the opposite side were paired with a low reward. Eye-movements recording showed that left-side high rewards led to subsequent increase of eye gaze fixations towards this side of the path, but no such asymmetry was found after exposure to right-sided high rewards. A milder spatial bias was also observed after left-side high rewards during subsequent exploration of a virtual castle yard, but not during route turn choices along the forest path. Our results indicate that reward-related influences on attention and behavior may be better learned in left than right space, in line with a right hemisphere dominance, and could generalize to another environment to some extent, but not to spatial choices in another decision task, suggesting some domain- or context-specificity. This proof-of-concept study also outlines the advantages and the possible drawbacks of the use of the 3D CAVE immersive platform for VR in neuroscience.

Prefrontal Cortex Represents Long-Term Memory of Object Values for Months

As a central hub for cognitive control, prefrontal cortex (PFC) is thought to utilize memories. However, unlike working or short-term memory, the neuronal representation of long-term memory in PFC has not been systematically investigated. Using single-unit recordings in macaques, we show that PFC neurons rapidly update and maintain responses to objects based on short-term reward history. Interestingly, after repeated object-reward association, PFC neurons continue to show value-biased responses to objects even in the absence of reward. This value-biased response is retained for several months after training and is resistant to extinction and to interference from new object-reward learning for many complex objects (>90). Accordingly, the monkeys remember the values of the learned objects for several months in separate testing. These findings reveal that in addition to flexible short-term and low-capacity memories, primate PFC represents stable long-term and high-capacity memories, which could prioritize valuable objects far into the future.

Direct and indirect pathways for choosing objects and actions

A prominent target of the basal ganglia is the superior colliculus (SC) which controls gaze orientation (saccadic eye movement in primates) to an important object. This 'object choice' is crucial for choosing an action on the object. SC is innervated by the substantia nigra pars reticulata (SNr) which is controlled mainly by the caudate nucleus (CD). This CD-SNr-SC circuit is sensitive to the values of individual objects and facilitates saccades to good objects. The object values are processed differently in two parallel circuits: flexibly by the caudate head (CDh) and stably by the caudate tail (CDt). To choose good objects, we need to reject bad objects. In fact, these contrasting functions are accomplished by the circuit originating from CDt: The direct pathway focuses on good objects and facilitates saccades to them; the indirect pathway focuses on bad objects and suppresses saccades to them. Inactivation of CDt deteriorated the object choice, because saccades to bad objects were no longer suppressed. This suggests that the indirect pathway is important for object choice. However, the direct and indirect pathways for 'object choice', which aim at the same action (i.e., saccade), may not work for 'action choice'. One possibility is that circuits controlling different actions are connected through the indirect pathway. Additional connections of the indirect pathway with brain areas outside the basal ganglia may also provide a wider range of behavioral choice. In conclusion, basal ganglia circuits are composed of the basic direct/indirect pathways and additional connections and thus have acquired multiple functions.

Parallel, but Dissociable, Processing in Discrete Corticostriatal Inputs Encodes Skill Learning

Changes in cortical and striatal function underlie the transition from novel actions to refined motor skills. How discrete, anatomically defined corticostriatal projections function in vivo to encode skill learning remains unclear. Using novel fiber photometry approaches to assess real-time activity of associative inputs from medial prefrontal cortex to dorsomedial striatum and sensorimotor inputs from motor cortex to dorsolateral striatum, we show that associative and sensorimotor inputs co-engage early in action learning and disengage in a dissociable manner as actions are refined. Disengagement of associative, but not sensorimotor, inputs predicts individual differences in subsequent skill learning. Divergent somatic and presynaptic engagement in both projections during early action learning suggests potential learning-related in vivo modulation of presynaptic corticostriatal function. These findings reveal parallel processing within associative and sensorimotor circuits that challenges and refines existing views of corticostriatal function and expose neuronal projection- and compartment-specific activity dynamics that encode and predict action learning.

Indirect Pathway of Caudal Basal Ganglia for Rejection of Valueless Visual Objects

The striatum controls behavior in two ways: facilitation and suppression through the direct and indirect pathways, respectively. However, it is still unclear what information is processed in these pathways. To address this question, we studied two pathways originating from the primate caudate tail (CDt). We found that the CDt innervated the caudal-dorsal-lateral part of the substantia nigra pars reticulata (cdlSNr), directly or indirectly through the caudal-ventral part of the globus pallidus externus (cvGPe). Notably, cvGPe neurons receiving inputs from the CDt were mostly visual neurons that encoded stable reward values of visual objects based on long-past experiences. Their dominant response was inhibition by valueless objects, which generated disinhibition of cdlSNr neurons and inhibition of superior colliculus neurons. Our data suggest that low-value signals are sent by the CDt-indirect pathway to suppress saccades to valueless objects, whereas high-value signals are sent by the CDt-direct pathway to facilitate saccades to valuable objects.

Reward processing in the value-driven attention network: reward signals tracking cue identity and location

Through associative reward learning, arbitrary cues acquire the ability to automatically capture visual attention. Previous studies have examined the neural correlates of value-driven attentional orienting, revealing elevated activity within a network of brain regions encompassing the visual corticostriatal loop [caudate tail, lateral occipital complex (LOC) and early visual cortex] and intraparietal sulcus (IPS). Such attentional priority signals raise a broader question concerning how visual signals are combined with reward signals during learning to create a representation that is sensitive to the confluence of the two. This study examines reward signals during the cued reward training phase commonly used to generate value-driven attentional biases. High, compared with low, reward feedback preferentially activated the value-driven attention network, in addition to regions typically implicated in reward processing. Further examination of these reward signals within the visual system revealed information about the identity of the preceding cue in the caudate tail and LOC, and information about the location of the preceding cue in IPS, while early visual cortex represented both location and identity. The results reveal teaching signals within the value-driven attention network during associative reward learning, and further suggest functional specialization within different regions of this network during the acquisition of an integrated representation of stimulus value.

Parallel basal ganglia circuits for decision making

The basal ganglia control body movements, mainly, based on their values. Critical for this mechanism is dopamine neurons, which sends unpredicted value signals, mainly, to the striatum. This mechanism enables animals to change their behaviors flexibly, eventually choosing a valuable behavior. However, this may not be the best behavior, because the flexible choice is focused on recent, and, therefore, limited, experiences (i.e., short-term memories). Our old and recent studies suggest that the basal ganglia contain separate circuits that process value signals in a completely different manner. They are insensitive to recent changes in value, yet gradually accumulate the value of each behavior (i.e., movement or object choice). These stable circuits eventually encode values of many behaviors and then retain the value signals for a long time (i.e., long-term memories). They are innervated by a separate group of dopamine neurons that retain value signals, even when no reward is predicted. Importantly, the stable circuits can control motor behaviors (e.g., hand or eye) quickly and precisely, which allows animals to automatically acquire valuable outcomes based on historical life experiences. These behaviors would be called 'skills', which are crucial for survival. The stable circuits are localized in the posterior part of the basal ganglia, separately from the flexible circuits located in the anterior part. To summarize, the flexible and stable circuits in the basal ganglia, working together but independently, enable animals (and humans) to reach valuable goals in various contexts.

Motor Learning Enhances Use-Dependent Plasticity

Motor behaviors are shaped not only by current sensory signals but also by the history of recent experiences. For instance, repeated movements toward a particular target bias the subsequent movements toward that target direction. This process, called use-dependent plasticity (UDP), is considered a basic and goal-independent way of forming motor memories. Most studies consider movement history as the critical component that leads to UDP (Classen et al., 1998; Verstynen and Sabes, 2011). However, the effects of learning (i.e., improved performance) on UDP during movement repetition have not been investigated. Here, we used transcranial magnetic stimulation in two experiments to assess plasticity changes occurring in the primary motor cortex after individuals repeated reinforced and nonreinforced actions. The first experiment assessed whether learning a skill task modulates UDP. We found that a group that successfully learned the skill task showed greater UDP than a group that did not accumulate learning, but made comparable repeated actions. The second experiment aimed to understand the role of reinforcement learning in UDP while controlling for reward magnitude and action kinematics. We found that providing subjects with a binary reward without visual feedback of the cursor led to increased UDP effects. Subjects in the group that received comparable reward not associated with their actions maintained the previously induced UDP. Our findings illustrate how reinforcing consistent actions strengthens use-dependent memories and provide insight into operant mechanisms that modulate plastic changes in the motor cortex.SIGNIFICANCE STATEMENT Performing consistent motor actions induces use-dependent plastic changes in the motor cortex. This plasticity reflects one of the basic forms of human motor learning. Past studies assumed that this form of learning is exclusively affected by repetition of actions. However, here we showed that success-based reinforcement signals could affect the human use-dependent plasticity (UDP) process. Our results indicate that learning augments and interacts with UDP. This effect is important to the understanding of the interplay between the different forms of motor learning and suggests that reinforcement is not only important to learning new behaviors, but can shape our subsequent behavior via its interaction with UDP.

Neurons in the primate dorsal striatum signal the uncertainty of object-reward associations

To learn, obtain reward and survive, humans and other animals must monitor, approach and act on objects that are associated with variable or unknown rewards. However, the neuronal mechanisms that mediate behaviours aimed at uncertain objects are poorly understood. Here we demonstrate that a set of neurons in an internal-capsule bordering regions of the primate dorsal striatum, within the putamen and caudate nucleus, signal the uncertainty of object–reward associations. Their uncertainty responses depend on the presence of objects associated with reward uncertainty and evolve rapidly as monkeys learn novel object–reward associations. Therefore, beyond its established role in mediating actions aimed at known or certain rewards, the dorsal striatum also participates in behaviours aimed at reward-uncertain objects.

The dorsal striatum (DS) is a brain region that is thought to aim actions at certain or known rewards. Here, the authors show that an internal-capsule bordering region of the primate DS signals the uncertainty of object-reward associations, suggesting a novel role for the DS in behavior under uncertainty.

Ecological Origins of Object Salience: Reward, Uncertainty, Aversiveness, and Novelty

Among many objects around us, some are more salient than others (i.e., attract our attention automatically). Some objects may be inherently salient (e.g., brighter), while others may become salient by virtue of their ecological relevance through experience. However, the role of ecological experience in automatic attention has not been studied systematically. To address this question, we let subjects (macaque monkeys) view a large number of complex objects (>300), each experienced repeatedly (>5 days) with rewarding, aversive or no outcome association (mere-perceptual exposure). Test of salience was done on separate days using free viewing with no outcome. We found that gaze was biased among the objects from the outset, affecting saccades to objects or fixations within objects. When the outcome was rewarding, gaze preference was stronger (i.e., positive) for objects with larger or equal but uncertain rewards. The effects of aversive outcomes were variable. Gaze preference was positive for some outcome associations (e.g., airpuff), but negative for others (e.g., time-out), possibly due to differences in threat levels. Finally, novel objects attracted gaze, but mere perceptual exposure of objects reduced their salience (learned negative salience). Our results show that, in primates, object salience is strongly influenced by previous ecological experience and is supported by a large memory capacity. Owing to such high capacity for learned salience, the ability to rapidly choose important objects can grow during the entire life to promote biological fitness.

Object-finding skill created by repeated reward experience

For most animals, survival depends on rapid detection of rewarding objects, but search for an object surrounded by many others is known to be difficult and time consuming. However, there is neuronal evidence for robust and rapid differentiation of objects based on their reward history in primates (Hikosaka, Kim, Yasuda, & Yamamoto, 2014). We hypothesized that such robust coding should support efficient search for high-value objects, similar to a pop-out mechanism. To test this hypothesis, we let subjects (n = 4, macaque monkeys) view a large number of complex objects with consistently biased rewards with variable training durations (1, 5, or 30 + days). Following training, subjects searched for a high-value object (Good) among a variable number of low-value objects (Bad). Consistent with our hypothesis, we found that Good objects were accurately and quickly targeted, often by a single and direct saccade with a very short latency (<200 ms). The dependence of search times on display size reduced significantly with longer reward training, giving rise to a more efficient search (40 ms/item to 16 ms/item). This object-finding skill showed a large capacity for value-biased objects and was maintained in the long-term memory with no interference from reward learning with other objects. Such object-finding skill, and in particular its large capacity and long term retention, would be crucial for maximizing rewards and biological fitness throughout life where many objects are experienced continuously and/or intermittently.

Neural mechanisms of goal-contingent task disengagement: Response-irrelevant stimuli activate the default mode network

As we experience the world, we must decide not only when and how to act based on input from the environment, but also when to avoid responding in situations where acting could lead to a detrimental outcome. The ability to regulate behavior in this way requires flexible cognitive control, as the same stimulus may call for a response in one context but not in another. In this sense, explicit non-responding can be characterized as an active, goal-directed cognitive process. Little is known about the mechanisms by which a currently active goal state modulates information processing to support the avoidance of undesired responding. In the present study, participants executed or withheld responses to a color target based whether its color matched that of a cue at the beginning of each trial. Behavioral and neural responses to task-irrelevant stimuli appearing as distractors were examined as a function of their relationship to the currently response-relevant color indicated by the cue. We observed a robust pattern in which stimuli possessing the currently response-irrelevant feature activate the default mode network (DMN), which was associated with a behavioral cost on trials in which this stimulus competed with a response-relevant target. Our findings reveal a role for the DMN in goal-directed cognitive control, facilitating active disengagement based on contextually-specific task demands.

Motivational state, reward value, and Pavlovian cues differentially affect skilled forelimb grasping in rats

Motor skills represent high-precision movements performed at optimal speed and accuracy. Such motor skills are learned with practice over time. Besides practice, effects of motivation have also been shown to influence speed and accuracy of movements, suggesting that fast movements are performed to maximize gained reward over time as noted in previous studies. In rodents, skilled motor performance has been successfully modeled with the skilled grasping task, in which animals use their forepaw to grasp for sugar pellet rewards through a narrow window. Using sugar pellets, the skilled grasping task is inherently tied to motivation processes. In the present study, we performed three experiments modulating animals’ motivation during skilled grasping by changing the motivational state, presenting different reward value ratios, and displaying Pavlovian stimuli. We found in all three studies that motivation affected the speed of skilled grasping movements, with the strongest effects seen due to motivational state and reward value. Furthermore, accuracy of the movement, measured in success rate, showed a strong dependence on motivational state as well. Pavlovian cues had only minor effects on skilled grasping, but results indicate an inverse Pavlovian-instrumental transfer effect on movement speed. These findings have broad implications considering the increasing use of skilled grasping in studies of motor system structure, function, and recovery after injuries.

Dopamine Neurons Encoding Long-Term Memory of Object Value for Habitual Behavior

Dopamine neurons promote learning by processing recent changes in reward values, such that reward may be maximized. However, such a flexible signal is not suitable for habitual behaviors that are sustained regardless of recent changes in reward outcome. We discovered a type of dopamine neuron in the monkey substantia nigra pars compacta (SNc) that retains past learned reward values stably. After reward values of visual objects are learned, these neurons continue to respond differentially to the objects, even when reward is not expected. Responses are strengthened by repeated learning and are evoked upon presentation of the objects long after learning is completed. These "sustain-type" dopamine neurons are confined to the caudal-lateral SNc and project to the caudate tail, which encodes long-term value memories of visual objects and guides gaze automatically to stably valued objects. This population of dopamine neurons thus selectively promotes learning and retention of habitual behavior.

Information processing biases in the brain: Implications for decision-making and self-governance

To make behavioral choices that are in line with our goals and our moral beliefs, we need to gather and consider information about our current situation. Most information present in our environment is not relevant to the choices we need or would want to make and thus could interfere with our ability to behave in ways that reflect our underlying values. Certain sources of information could even lead us to make choices we later regret, and thus it would be beneficial to be able to ignore that information. Our ability to exert successful self-governance depends on our ability to attend to sources of information that we deem important to our decision-making processes. We generally assume that, at any moment, we have the ability to choose what we pay attention to. However, recent research indicates that what we pay attention to is influenced by our prior experiences, including reward history and past successes and failures, even when we are not aware of this history. Even momentary distractions can cause us to miss or discount information that should have a greater influence on our decisions given our values. Such biases in attention thus raise questions about the degree to which the choices that we make may be poorly informed and not truly reflect our ability to otherwise exert self-governance.

The Role of Dopamine in Value-Based Attentional Orienting

Reward learning gives rise to strong attentional biases. Stimuli previously associated with reward automatically capture visual attention regardless of intention. Dopamine signaling within the ventral striatum plays an important role in reward learning, representing the expected reward initiated by a cue. How dopamine and the striatum may be involved in maintaining behaviors that have been shaped by reward learning, even after reward expectancies have changed, is less well understood. Nonspecific measures of brain activity have implicated the striatum in value-based attention. However, the neurochemical mechanisms underlying the attentional priority of learned reward cues remain unexplored. Here, we investigated the contribution of dopamine to value-based attention using positron emission tomography (PET) with [(11)C]raclopride. We show that, in the explicit absence of reward, the magnitude of attentional capture by previously reward-associated but currently task-irrelevant distractors is correlated across individuals with changes in available D2/D3 dopamine receptors (presumably due to intrasynaptic dopamine) linked to distractor processing within the right caudate and posterior putamen. Our findings provide direct evidence linking dopamine signaling within the striatum to the involuntary orienting of attention, and specifically to the attention-grabbing quality of learned reward cues. These findings also shed light on the neurochemical basis of individual susceptibility to value-driven attentional capture, which is known to play a role in addiction. More broadly, the present study highlights the value and feasibility of using PET to relate changes in the release of a neurotransmitter to learning-dependent changes in healthy adults.

Counterintuitive effects of negative social feedback on attention

Which stimuli we pay attention to is strongly influenced by learning. Stimuli previously associated with reward outcomes, such as money and food, and stimuli previously associated with aversive outcomes, such as monetary loss and electric shock, automatically capture attention. Social reward (happy expressions) can bias attention towards associated stimuli, but the role of negative social feedback in biasing attentional selection remains unexplored. On the one hand, negative social feedback often serves to discourage particular behaviours. If attentional selection can be curbed much like any other behavioural preference, we might expect stimuli associated with negative social feedback to be more readily ignored. On the other hand, if negative social feedback influences attention in the same way that other aversive outcomes do, such feedback might ironically bias attention towards the stimuli it is intended to discourage selection of. In the present study, participants first completed a training phase in which colour targets were associated with negative social feedback. Then, in a subsequent test phase, these same colour stimuli served as task-irrelevant distractors during a visual search task. The results strongly support the latter interpretation in that stimuli previously associated with negative social feedback impaired search performance.

The attention habit: how reward learning shapes attentional selection

There is growing consensus that reward plays an important role in the control of attention. Until recently, reward was thought to influence attention indirectly by modulating task-specific motivation and its effects on voluntary control over selection. Such an account was consistent with the goal-directed (endogenous) versus stimulus-driven (exogenous) framework that had long dominated the field of attention research. Now, a different perspective is emerging. Demonstrations that previously reward-associated stimuli can automatically capture attention even when physically inconspicuous and task-irrelevant challenge previously held assumptions about attentional control. The idea that attentional selection can be value driven, reflecting a distinct and previously unrecognized control mechanism, has gained traction. Since these early demonstrations, the influence of reward learning on attention has rapidly become an area of intense investigation, sparking many new insights. The result is an emerging picture of how the reward system of the brain automatically biases information processing. Here, I review the progress that has been made in this area, synthesizing a wealth of recent evidence to provide an integrated, up-to-date account of value-driven attention and some of its broader implications.

Social reward shapes attentional biases

Paying attention to stimuli that predict a reward outcome is important for an organism to survive and thrive. When visual stimuli are associated with tangible, extrinsic rewards such as money or food, these stimuli acquire high attentional priority and come to automatically capture attention. In humans and other primates, however, many behaviors are not motivated directly by such extrinsic rewards, but rather by the social feedback that results from performing those behaviors. In the present study, I examine whether positive social feedback can similarly influence attentional bias. The results show that stimuli previously associated with a high probability of positive social feedback elicit value-driven attentional capture, much like stimuli associated with extrinsic rewards. Unlike with extrinsic rewards, however, such stimuli also influence task-specific motivation. My findings offer a potential mechanism by which social reward shapes the information that we prioritize when perceiving the world around us.

Value-based attentional capture influences context-dependent decision-making

Normative theories posit that value-based decision-making is context independent. However, decisions between two high-value options can be suboptimally biased by the introduction of a third low-value option. This context-dependent modulation is consistent with the divisive normalization of the value of each stimulus by the total value of all stimuli. In addition, an independent line of research demonstrates that pairing a stimulus with a high-value outcome can lead to attentional capture that can mediate the efficiency of visual information processing. Here we tested the hypothesis that value-based attentional capture interacts with value-based normalization to influence the optimality of decision-making. We used a binary-choice paradigm in which observers selected between two targets and the color of each target indicated the magnitude of their reward potential. Observers also had to simultaneously ignore a task-irrelevant distractor rendered in a color that was previously associated with a specific reward magnitude. When the color of the task-irrelevant distractor was previously associated with a high reward, observers responded more slowly and less optimally. Moreover, as the learned value of the distractor increased, electrophysiological data revealed an attenuation of the lateralized N1 and N2Pc responses evoked by the relevant choice stimuli and an attenuation of the late positive deflection (LPD). Collectively, these behavioral and electrophysiological data suggest that value-based attentional capture and value-based normalization jointly mediate the influence of context on free-choice decision-making.

Parallel basal ganglia circuits for voluntary and automatic behaviour to reach rewards

The basal ganglia control body movements, value processing and decision-making. Many studies have shown that the inputs and outputs of each basal ganglia structure are topographically organized, which suggests that the basal ganglia consist of separate circuits that serve distinct functions. A notable example is the circuits that originate from the rostral (head) and caudal (tail) regions of the caudate nucleus, both of which target the superior colliculus. These two caudate regions encode the reward values of visual objects differently: flexible (short-term) values by the caudate head and stable (long-term) values by the caudate tail. These value signals in the caudate guide the orienting of gaze differently: voluntary saccades by the caudate head circuit and automatic saccades by the caudate tail circuit. Moreover, separate groups of dopamine neurons innervate the caudate head and tail and may selectively guide the flexible and stable learning/memory in the caudate regions. Studies focusing on manual handling of objects also suggest that rostrocaudally separated circuits in the basal ganglia control the action differently. These results suggest that the basal ganglia contain parallel circuits for two steps of goal-directed behaviour: finding valuable objects and manipulating the valuable objects. These parallel circuits may underlie voluntary behaviour and automatic skills, enabling animals (including humans) to adapt to both volatile and stable environments. This understanding of the functions and mechanisms of the basal ganglia parallel circuits may inform the differential diagnosis and treatment of basal ganglia disorders.