Changes in neural connectivity underlie decision threshold modulation for reward maximization

Contributions of the Basal Ganglia to Visual Perceptual Decisions

The basal ganglia (BG) make up a prominent nexus between visual and motor-related brain regions. In contrast to the BG's well-established roles in movement control and value-based decision making, their contributions to the transformation of visual input into an action remain unclear, especially in the context of perceptual decisions based on uncertain visual evidence. This article reviews recent progress in our understanding of the BG's contributions to the formation, evaluation, and adjustment of such decisions. From theoretical and experimental perspectives, the review focuses on four key stations in the BG network, namely, the striatum, pallidum, subthalamic nucleus, and midbrain dopamine neurons, which can have different roles and together support the decision process.

Reward at encoding but not retrieval modulates memory for detailed events

Much of the evidence suggesting that rewards improve memory performance has focused on how explicit rewards facilitate encoding of simplistic stimuli. To expand beyond this focus, the current study tested how explicit rewards presented at encoding as well as retrieval facilitate memory for information contained within complex events. In a single experimental session, participants (N = 88) encoded videos depicting naturalistic events (e.g., getting dressed) and then completed a recognition test probing their memory for different detail types (i.e., event, perceptual, or contextual) from the video stimuli. We manipulated the explicit reward associated with each video, such that accurate memory responses for half the videos were associated with high monetary incentives and half were associated with low monetary incentives. This reward manipulation was presented at either encoding or retrieval during a recognition memory test. The reward manipulation only affected memory when presented at encoding and this effect did not depend on the type of detail probed. Drift Diffusion Modelling further revealed that presenting reward information at encoding engendered greater encoding fidelity-indexed by an increase in drift rate-but did not change response caution at the time of retrieval-indexed by response threshold. Together, our results suggest that presenting reward information when encoding but not retrieving complex events has a general facilitatory effect, likely via attentional processing, on the ability to later remember precise details from the event.

Basal impulses: findings from the last twenty years on impulsivity and reward pathways using deep brain stimulation

INTRODUCTION

Deep brain stimulation (DBS) is an important treatment modality for movement disorders. Its role in tasks and processes of higher cortical function continues to increase in importance and relevance. This systematic review investigates the impact of DBS on measures of impulsivity.

EVIDENCE ACQUISITION

A total of 45 studies were collated from PubMed (30 prospective, 8 animal, 4 questionnaire-based, and 3 computational models), excluding case reports and review articles. Two areas extensively studied are the subthalamic nucleus (STN) and nucleus accumbens (NAc).

EVIDENCE SYNTHESIS

While both are part of the basal ganglia, the STN and NAc have extensive connections to the prefrontal cortex, cingulate cortex, and limbic system. Therefore, understanding cause and treatment of impulsivity requires understanding motor pathways, learning, memory, and emotional processing. DBS of the STN and NAc shell can increase objective measures of impulsivity, as measured by reaction times or reward-based learning, independent from patient insight. The ability for DBS to treat impulse control disorders, and also cause and/or worsen impulsivity in Parkinson's disease, may be explained by the affected closely-related neuroanatomical areas with discrete and sometimes opposing functions.

CONCLUSIONS

As newer, more refined DBS technology emerges, large-scale prospective studies specifically aimed at treatment of impulsivity disorders are needed.

Neural processes during adolescent risky decision making are associated with conformity to peer influence

•

Adolescents’ neural responses to risky decisions may modulate their conformity to different types of peer influence.

•

Neural activity in the anterior cingulate cortex (ACC) predicted conformity to risky peers while driving.

•

Connectivity between VS and risk processing regions (including insula and ACC) predicted safer driving under risky influence.

Adolescents demonstrate both heightened sensitivity to peer influence and increased risk-taking. The current study provides a novel test of how these two phenomena are related at behavioral and neural levels. Adolescent males (N = 83, 16–17 years) completed the Balloon Analogue Risk Task (BART) in an fMRI scanner. One week later, participants completed a driving task in which they drove alone and with a safety- or risk-promoting peer passenger. Results showed that neural responses during BART were associated with participants’ behavioral conformity to safe vs. risky peer influence while later driving. First, the extent that neural activation in the anterior cingulate cortex (ACC) scaled with decision stakes in BART was associated with conformity to risky peer influence. Additionally, stake-modulated functional connectivity between ventral striatum (VS) and risk processing regions (including ACC and insula) was associated with safer driving under risky peer influence (i.e. resistance to risky peer influence), suggesting that connectivity between VS and ACC as well as insula may serve a protective role under risky peer influence. Together, these results suggest that adolescents’ neural responses to risky decision making may modulate their behavioral conformity to different types of peer influence on risk taking.

Characterizing the role of the pre-SMA in the control of speed/accuracy trade-off with directed functional connectivity mapping and multiple solution reduction

Several plausible theories of the neural implementation of speed/accuracy trade-off (SAT), the phenomenon in which individuals may alternately emphasize speed or accuracy during the performance of cognitive tasks, have been proposed, and multiple lines of evidence point to the involvement of the pre-supplemental motor area (pre-SMA). However, as the nature and directionality of the pre-SMA's functional connections to other regions involved in cognitive control and task processing are not known, its precise role in the top-down control of SAT remains unclear. Although recent advances in cross-sectional path modeling provide a promising way of characterizing these connections, such models are limited by their tendency to produce multiple equivalent solutions. In a sample of healthy adults (N = 18), the current study uses the novel approach of Group Iterative Multiple Model Estimation for Multiple Solutions (GIMME-MS) to assess directed functional connections between the pre-SMA, other regions previously linked to control of SAT, and regions putatively involved in evidence accumulation for the decision task. Results reveal a primary role of the pre-SMA for modulating activity in regions involved in the decision process but suggest that this region receives top-down input from the DLPFC. Findings also demonstrate the utility of GIMME-MS and solution-reduction methods for obtaining valid directional inferences from connectivity path models.

Not all observed actions are perceived equally

Action observation is the visual process analyzing the actions of others to determine their goals and how the actor's body (part) movements permit attaining those goals. Our recent psychophysical study demonstrated that 1) observed action (OA) perception differs from shape perception in viewpoint and duration dependence, and 2) accuracy and reaction times of OA discrimination are fitted by the proportional-rate diffusion model whereby a sensory stage provides noisy evidence that is accumulated up to a criterion or bound by a decision stage. That study was devoted to observation of manipulative actions, following a general trend of the field. Recent functional imaging studies of action observation, however, have established various OA classes as separate entities with processing routes involving distinct posterior parietal cortex (PPC) regions. Here, we show that the diffusion model applies to multiple OA classes. Even more importantly, the observers' ability to discriminate exemplars of a given class differs considerably between OA classes and these performance differences correspond to differences in model parameters. In particular, OA classes differ in the bound parameter which we propose may reflect an urgency signal originating in the PPC regions corresponding to the sensory stages of different OA classes.

Modelling ADHD: A review of ADHD theories through their predictions for computational models of decision-making and reinforcement learning

Attention deficit hyperactivity disorder (ADHD) is characterized by altered decision-making (DM) and reinforcement learning (RL), for which competing theories propose alternative explanations. Computational modelling contributes to understanding DM and RL by integrating behavioural and neurobiological findings, and could elucidate pathogenic mechanisms behind ADHD. This review of neurobiological theories of ADHD describes predictions for the effect of ADHD on DM and RL as described by the drift-diffusion model of DM (DDM) and a basic RL model. Empirical studies employing these models are also reviewed. While theories often agree on how ADHD should be reflected in model parameters, each theory implies a unique combination of predictions. Empirical studies agree with the theories' assumptions of a lowered DDM drift rate in ADHD, while findings are less conclusive for boundary separation. The few studies employing RL models support a lower choice sensitivity in ADHD, but not an altered learning rate. The discussion outlines research areas for further theoretical refinement in the ADHD field.

Subthalamic nucleus deep brain stimulation induces impulsive action when patients with Parkinson's disease act under speed pressure

The subthalamic nucleus (STN) is proposed to modulate response thresholds and speed–accuracy trade-offs. In situations of conflict, the STN is considered to raise response thresholds, allowing time for the accumulation of information to occur before a response is selected. Conversely, speed pressure is thought to reduce the activity of the STN and lower response thresholds, resulting in fast, errorful responses. In Parkinson’s disease (PD), subthalamic nucleus deep brain stimulation (STN-DBS) reduces the activity of the nucleus and improves motor symptoms. We predicted that the combined effects of STN stimulation and speed pressure would lower STN activity and lead to fast, errorful responses, hence resulting in impulsive action. We used the motion discrimination ‘moving-dots’ task to assess speed–accuracy trade-offs, under both speed and accuracy instructions. We assessed 12 patients with PD and bilateral STN-DBS and 12 age-matched healthy controls. Participants completed the task twice, and the patients completed it once with STN-DBS on and once with STN-DBS off, with order counterbalanced. We found that STN stimulation was associated with significantly faster reaction times but more errors under speed instructions. Application of the drift diffusion model showed that stimulation resulted in lower response thresholds when acting under speed pressure. These findings support the involvement of the STN in the modulation of speed–accuracy trade-offs and establish for the first time that speed pressure alone, even in the absence of conflict, can result in STN stimulation inducing impulsive action in PD.

Evidence Accumulation and Choice Maintenance Are Dissociated in Human Perceptual Decision Making

Perceptual decision making in monkeys relies on decision neurons, which accumulate evidence and maintain choices until a response is given. In humans, several brain regions have been proposed to accumulate evidence, but it is unknown if these regions also maintain choices. To test if accumulator regions in humans also maintain decisions we compared delayed and self-paced responses during a face/house discrimination decision making task. Computational modeling and fMRI results revealed dissociated processes of evidence accumulation and decision maintenance, with potential accumulator activations found in the dorsomedial prefrontal cortex, right inferior frontal gyrus and bilateral insula. Potential maintenance activation spanned the frontal pole, temporal gyri, precuneus and the lateral occipital and frontal orbital cortices. Results of a quantitative reverse inference meta-analysis performed to differentiate the functions associated with the identified regions did not narrow down potential accumulation regions, but suggested that response-maintenance might rely on a verbalization of the response.

Paying Attention to the Details of Attention

Attention selects behaviorally relevant stimuli for greater neural representation. In this issue of Neuron, Luo and Maunsell (2015) show that attention acts, in part, by boosting the signal-to-noise ratio (SNR) of sensory neurons.

Tuning the speed-accuracy trade-off to maximize reward rate in multisensory decision-making

For decisions made under time pressure, effective decision making based on uncertain or ambiguous evidence requires efficient accumulation of evidence over time, as well as appropriately balancing speed and accuracy, known as the speed/accuracy trade-off. For simple unimodal stimuli, previous studies have shown that human subjects set their speed/accuracy trade-off to maximize reward rate. We extend this analysis to situations in which information is provided by multiple sensory modalities. Analyzing previously collected data (Drugowitsch et al., 2014), we show that human subjects adjust their speed/accuracy trade-off to produce near-optimal reward rates. This trade-off can change rapidly across trials according to the sensory modalities involved, suggesting that it is represented by neural population codes rather than implemented by slow neuronal mechanisms such as gradual changes in synaptic weights. Furthermore, we show that deviations from the optimal speed/accuracy trade-off can be explained by assuming an incomplete gradient-based learning of these trade-offs.

Representation of accumulating evidence for a decision in two parietal areas

Decisions are often made by accumulating evidence for and against the alternatives. The momentary evidence represented by sensory neurons is accumulated by downstream structures to form a decision variable, linking the evolving decision to the formation of a motor plan. When decisions are communicated by eye movements, neurons in the lateral intraparietal area (LIP) represent the accumulation of evidence bearing on the potential targets for saccades. We now show that reach-related neurons from the medial intraparietal area (MIP) exhibit a gradual modulation of their firing rates consistent with the representation of an evolving decision variable. When decisions were communicated by saccades instead of reaches, decision-related activity was attenuated in MIP, whereas LIP neurons were active while monkeys communicated decisions by saccades or reaches. Thus, for decisions communicated by a hand movement, a parallel flow of sensory information is directed to parietal areas MIP and LIP during decision formation.

Value is in the eye of the beholder: early visual cortex codes monetary value of objects during a diverted attention task

A central concern in the study of learning and decision-making is the identification of neural signals associated with the values of choice alternatives. An important factor in understanding the neural correlates of value is the representation of the object itself, separate from the act of choosing. Is it the case that the representation of an object within visual areas will change if it is associated with a particular value? We used fMRI adaptation to measure the neural similarity of a set of novel objects before and after participants learned to associate monetary values with the objects. We used a range of both positive and negative values to allow us to distinguish effects of behavioral salience (i.e., large vs. small values) from effects of valence (i.e., positive vs. negative values). During the scanning session, participants made a perceptual judgment unrelated to value. Crucially, the similarity of the visual features of any pair of objects did not predict the similarity of their value, so we could distinguish adaptation effects due to each dimension of similarity. Within early visual areas, we found that value similarity modulated the neural response to the objects after training. These results show that an abstract dimension, in this case, monetary value, modulates neural response to an object in visual areas of the brain even when attention is diverted.

Neural dynamics implement a flexible decision bound with a fixed firing rate for choice: a model-based hypothesis

Decisions are faster and less accurate when conditions favor speed, and are slower and more accurate when they favor accuracy. This speed-accuracy trade-off (SAT) can be explained by the principles of bounded integration, where noisy evidence is integrated until it reaches a bound. Higher bounds reduce the impact of noise by increasing integration times, supporting higher accuracy (vice versa for speed). These computations are hypothesized to be implemented by feedback inhibition between neural populations selective for the decision alternatives, each of which corresponds to an attractor in the space of network states. Since decision-correlated neural activity typically reaches a fixed rate at the time of commitment to a choice, it has been hypothesized that the neural implementation of the bound is fixed, and that the SAT is supported by a common input to the populations integrating evidence. According to this hypothesis, a stronger common input reduces the difference between a baseline firing rate and a threshold rate for enacting a choice. In simulations of a two-choice decision task, we use a reduced version of a biophysically-based network model (Wong and Wang, 2006) to show that a common input can control the SAT, but that changes to the threshold-baseline difference are epiphenomenal. Rather, the SAT is controlled by changes to network dynamics. A stronger common input decreases the model's effective time constant of integration and changes the shape of the attractor landscape, so the initial state is in a more error-prone position. Thus, a stronger common input reduces decision time and lowers accuracy. The change in dynamics also renders firing rates higher under speed conditions at the time that an ideal observer can make a decision from network activity. The difference between this rate and the baseline rate is actually greater under speed conditions than accuracy conditions, suggesting that the bound is not implemented by firing rates per se.

On the neural implementation of the speed-accuracy trade-off

Decisions are faster and less accurate when conditions favor speed, and are slower and more accurate when they favor accuracy. This phenomenon is referred to as the speed-accuracy trade-off (SAT). Behavioral studies of the SAT have a long history, and the data from these studies are well characterized within the framework of bounded integration. According to this framework, decision makers accumulate noisy evidence until the running total for one of the alternatives reaches a bound. Lower and higher bounds favor speed and accuracy respectively, each at the expense of the other. Studies addressing the neural implementation of these computations are a recent development in neuroscience. In this review, we describe the experimental and theoretical evidence provided by these studies. We structure the review according to the framework of bounded integration, describing evidence for (1) the modulation of the encoding of evidence under conditions favoring speed or accuracy, (2) the modulation of the integration of encoded evidence, and (3) the modulation of the amount of integrated evidence sufficient to make a choice. We discuss commonalities and differences between the proposed neural mechanisms, some of their assumptions and simplifications, and open questions for future work. We close by offering a unifying hypothesis on the present state of play in this nascent research field.

Temporal characteristics of the influence of punishment on perceptual decision making in the human brain

Perceptual decision making is the process by which information from sensory systems is combined and used to influence our behavior. In addition to the sensory input, this process can be affected by other factors, such as reward and punishment for correct and incorrect responses. To investigate the temporal dynamics of how monetary punishment influences perceptual decision making in humans, we collected electroencephalography (EEG) data during a perceptual categorization task whereby the punishment level for incorrect responses was parametrically manipulated across blocks of trials. Behaviorally, we observed improved accuracy for high relative to low punishment levels. Using multivariate linear discriminant analysis of the EEG, we identified multiple punishment-induced discriminating components with spatially distinct scalp topographies. Compared with components related to sensory evidence, components discriminating punishment levels appeared later in the trial, suggesting that punishment affects primarily late postsensory, decision-related processing. Crucially, the amplitude of these punishment components across participants was predictive of the size of the behavioral improvements induced by punishment. Finally, trial-by-trial changes in prestimulus oscillatory activity in the alpha and gamma bands were good predictors of the amplitude of these components. We discuss these findings in the context of increased motivation/attention, resulting from increases in punishment, which in turn yields improved decision-related processing.