Assessing the neural basis of uncertainty in perceptual category learning through varying levels of distortion

Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta-analysis

Learning occurs when an outcome differs from expectations, generating a reward prediction error signal (RPE). The RPE signal has been hypothesized to simultaneously embody the valence of an outcome (better or worse than expected) and its surprise (how far from expectations). Nonetheless, growing evidence suggests that separate representations of the two RPE components exist in the human brain. Meta-analyses provide an opportunity to test this hypothesis and directly probe the extent to which the valence and surprise of the error signal are encoded in separate or overlapping networks. We carried out several meta-analyses on a large set of fMRI studies investigating the neural basis of RPE, locked at decision outcome. We identified two valence learning systems by pooling studies searching for differential neural activity in response to categorical positive-versus-negative outcomes. The first valence network (negative > positive) involved areas regulating alertness and switching behaviours such as the midcingulate cortex, the thalamus and the dorsolateral prefrontal cortex whereas the second valence network (positive > negative) encompassed regions of the human reward circuitry such as the ventral striatum and the ventromedial prefrontal cortex. We also found evidence of a largely distinct surprise-encoding network including the anterior cingulate cortex, anterior insula and dorsal striatum. Together with recent animal and electrophysiological evidence this meta-analysis points to a sequential and distributed encoding of different components of the RPE signal, with potentially distinct functional roles.

Dorsal striatum mediates deliberate decision making, not late-stage, stimulus-response learning

We investigated a controversy regarding the role of the dorsal striatum (DS) in deliberate decision-making versus late-stage, stimulus-response learning to the point of automatization. Participants learned to associate abstract images with right or left button presses explicitly before strengthening these associations through stimulus-response trials with (i.e., Session 1) and without (i.e., Session 2) feedback. In Session 1, trials were divided into response-selection and feedback events to separately assess decision versus learning processes. Session 3 evaluated stimulus-response automaticity using a location Stroop task. DS activity correlated with response-selection and not feedback events in Phase 1 (i.e., Blocks 1-3), Session 1. Longer response times (RTs), lower accuracy, and greater intertrial variability characterized Phase 1, suggesting deliberation. DS activity extinguished in Phase 2 (i.e., Blocks 4-12), Session 1, once RTs, response variability, and accuracy stabilized, though stimulus-response automatization continued. This was signaled by persisting improvements in RT and accuracy into Session 2. Distraction between Sessions 1 and 2 briefly reintroduced response uncertainty, and correspondingly, significant DS activity reappeared in Block 1 of Session 2 only. Once stimulus-response associations were again refamiliarized and deliberation unnecessary, DS activation disappeared for Blocks 2-8, Session 2. Interference from previously learned right or left button responses with incongruent location judgments in a location Stroop task provided evidence that automaticity of stimulus-specific button-press responses had developed by the end of Session 2. These results suggest that DS mediates decision making and not late-stage learning, reconciling two, independently evolving and well-supported literatures that implicate DS in different cognitive functions. Hum Brain Mapp 38:6133-6156, 2017. © 2017 Wiley Periodicals, Inc.

Human substantia nigra neurons encode decision outcome and are modulated by categorization uncertainty in an auditory categorization task

The ability to categorize stimuli – predator or prey, friend or foe – is an essential feature of the decision-making process. Underlying that ability is the development of an internally generated category boundary to generate decision outcomes. While classic temporal difference reinforcement models assume midbrain dopaminergic neurons underlie the prediction error required to learn boundary location, these neurons also demonstrate a robust response to nonreward incentive stimuli. More recent models suggest that this may reflect a motivational aspect to performing a task which should be accounted for when modeling dopaminergic neuronal behavior. To clarify the role of substantia nigra dopamine neurons in uncertain perceptual decision making, we investigated their behavior using single neuron extracellular recordings in patients with Parkinson's disease undergoing deep brain stimulation. Subjects underwent a simple auditory categorical decision-making task in which they had to classify a tone as either low- or high-pitched relative to an explicit threshold tone and received feedback but no reward. We demonstrate that the activity of human SN dopaminergic neurons is predictive of perceptual categorical decision outcome and is modulated by uncertainty. Neuronal activity was highest during difficult (uncertain) decisions that resulted in correct responses and lowest during easy decisions that resulted in incorrect responses. This pattern of results is more consistent with a “motivational” role with regards to perceptual categorization and suggests that dopamine neurons are most active when critical information – as represented by uncertainty – is available for learning decision boundaries.

Generalization in category learning: the roles of representational and decisional uncertainty

Effective generalization in a multiple-category situation involves both assessing potential membership in individual categories and resolving conflict between categories while implementing a decision bound. We separated generalization from decision bound implementation using an information integration task in which category exemplars varied over two incommensurable feature dimensions. Human subjects first learned to categorize stimuli within limited training regions, and then, during fMRI scanning, they also categorized transfer stimuli from new regions of perceptual space. Transfer stimuli differed both in distance from the training region prototype and distance from the decision bound, allowing us to independently assess neural systems sensitive to each. Across all stimulus regions, categorization was associated with activity in the extrastriate visual cortex, basal ganglia, and the bilateral intraparietal sulcus. Categorizing stimuli near the decision bound was associated with recruitment of the frontoinsular cortex and medial frontal cortex, regions often associated with conflict and which commonly coactivate within the salience network. Generalization was measured in terms of greater distance from the decision bound and greater distance from the category prototype (average training region stimulus). Distance from the decision bound was associated with activity in the superior parietal lobe, lingual gyri, and anterior hippocampus, whereas distance from the prototype was associated with left intraparietal sulcus activity. The results are interpreted as supporting the existence of different uncertainty resolution mechanisms for uncertainty about category membership (representational uncertainty) and uncertainty about decision bound (decisional uncertainty).

Frontoparietal networks involved in categorization and item working memory

Categorization and memory for specific items are fundamental processes that allow us to apply knowledge to novel stimuli. This study directly compares categorization and memory using delay match to category (DMC) and delay match to sample (DMS) tasks. In DMC participants view and categorize a stimulus, maintain the category across a delay, and at the probe phase view another stimulus and indicate whether it is in the same category or not. In DMS, a standard item working memory task, participants encode and maintain a specific individual item, and at probe decide if the stimulus is an exact match or not. Constrained Principal Components Analysis was used to identify and compare activity within neural networks associated with these tasks, and we relate these networks to those that have been identified with resting state-fMRI. We found that two frontoparietal networks of particular interest. The first network included regions associated with the dorsal attention network and frontoparietal salience network; this network showed patterns of activity consistent with a role in rapid orienting to and processing of complex stimuli. The second uniquely involved regions of the frontoparietal central-executive network; this network responded more slowly following each stimulus and showed a pattern of activity consistent with a general role in role in decision-making across tasks. Additional components were identified that were associated with visual, somatomotor and default mode networks.

Striatal and hippocampal involvement in motor sequence chunking depends on the learning strategy

Motor sequences can be learned using an incremental approach by starting with a few elements and then adding more as training evolves (e.g., learning a piano piece); conversely, one can use a global approach and practice the whole sequence in every training session (e.g., shifting gears in an automobile). Yet, the neural correlates associated with such learning strategies in motor sequence learning remain largely unexplored to date. Here we used functional magnetic resonance imaging to measure the cerebral activity of individuals executing the same 8-element sequence after they completed a 4-days training regimen (2 sessions each day) following either a global or incremental strategy. A network comprised of striatal and fronto-parietal regions was engaged significantly regardless of the learning strategy, whereas the global training regimen led to additional cerebellar and temporal lobe recruitment. Analysis of chunking/grouping of sequence elements revealed a common prefrontal network in both conditions during the chunk initiation phase, whereas execution of chunk cores led to higher mediotemporal activity (involving the hippocampus) after global than incremental training. The novelty of our results relate to the recruitment of mediotemporal regions conditional of the learning strategy. Thus, the present findings may have clinical implications suggesting that the ability of patients with lesions to the medial temporal lobe to learn and consolidate new motor sequences may benefit from using an incremental strategy.

You'd better think twice: post-decision perceptual confidence

Current findings suggest that confidence emerges only after decision making. However, the temporal and neural dynamics of the emergence of post-decision confidence--a metacognitive judgement--are not fully explored. To gain insight into the dynamics of post-decision confidence processing and to disentangle the processes underlying confidence judgements and decision making, we applied a tactile discrimination task during functional magnetic resonance imaging (fMRI). Our results revealed that reaction times to post-decision confidence depend on the level of confidence, suggesting that post-decision confidence in a perceptual choice is not processed in parallel to perceptual decision making. Moreover, we demonstrated by the parametric analysis of fMRI data that post-decisionally modelled confidence processing can be distinguished from processes related to decision making through anatomical location and through the pattern of neural activity. In contrast to perceptual decision making, post-decision confidence appears to be strictly allocated to a prefrontal network of brain regions, primarily the anterior and dorsolateral prefrontal cortex, areas that have been related to metacognition. Moreover, the processes underlying decision making and post-decision confidence may share recruitment of the dorsolateral prefrontal cortex, although the former probably has distinct functions with regard to processing of perceptual choices and post-decision confidence. Thus, this is the first fMRI study to disentangle the processes underlying post-decision confidence and decision making on behavioural, neuroanatomical, and neurofunctional levels. With regard to the temporal evolution of post-decision confidence, results of the present study provide strong support for the most recent theoretical models of human perceptual decision making, and thus provide implications for investigating confidence in perceptual paradigms.

Learning-induced uncertainty reduction in perceptual decisions is task-dependent

Perceptual decision-making in which decisions are reached primarily from extracting and evaluating sensory information requires close interactions between the sensory system and decision-related networks in the brain. Uncertainty pervades every aspect of this process and can be considered related to either the stimulus signal or decision criterion. Here, we investigated the learning-induced reduction of both the signal and criterion uncertainty in two perceptual decision tasks based on two Glass pattern stimulus sets. This was achieved by manipulating spiral angle and signal level of radial and concentric Glass patterns. The behavioral results showed that the participants trained with a task based on criterion comparison improved their categorization accuracy for both tasks, whereas the participants who were trained on a task based on signal detection improved their categorization accuracy only on their trained task. We fitted the behavioral data with a computational model that can dissociate the contribution of the signal and criterion uncertainties. The modeling results indicated that the participants who were trained on the criterion comparison task reduced both the criterion and signal uncertainty. By contrast, the participants who were trained on the signal detection task only reduced their signal uncertainty after training. Our results suggest that the signal uncertainty can be resolved by training participants to extract signals from noisy environments and to discriminate between clear signals, which are evidenced by reduced perception variance after both training procedures. Conversely, the criterion uncertainty can only be resolved by the training of fine discrimination. These findings demonstrate that uncertainty in perceptual decision-making can be reduced with training but that the reduction of different types of uncertainty is task-dependent.

Linking neuroscientific research on decision making to the educational context of novice students assigned to a multiple-choice scientific task involving common misconceptions about electrical circuits

Functional magnetic resonance imaging was used to identify the brain-based mechanisms of uncertainty and certainty associated with answers to multiple-choice questions involving common misconceptions about electric circuits. Twenty-two scientifically novice participants (humanities and arts college students) were asked, in an fMRI study, whether or not they thought the light bulbs in images presenting electric circuits were lighted up correctly, and if they were certain or uncertain of their answers. When participants reported that they were unsure of their responses, analyses revealed significant activations in brain areas typically involved in uncertainty (anterior cingulate cortex, anterior insula cortex, and superior/dorsomedial frontal cortex) and in the left middle/superior temporal lobe. Certainty was associated with large bilateral activations in the occipital and parietal regions usually involved in visuospatial processing. Correct-and-certain answers were associated with activations that suggest a stronger mobilization of visual attention resources when compared to incorrect-and-certain answers. These findings provide insights into brain-based mechanisms of uncertainty that are activated when common misconceptions, identified as such by science education research literature, interfere in decision making in a school-like task. We also discuss the implications of these results from an educational perspective.

Neural correlates of state- and strength-based perception

Perceptual judgments can be based on two kinds of information: state-based perception of specific, detailed visual information, or strength-based perception of global or relational information. State-based perception is discrete in the sense that it either occurs or fails, whereas strength-based perception is continuously graded from weak to strong. The functional characteristics of these types of perception have been examined in some detail, but whether state- and strength-based perception are supported by different brain regions has been largely unexplored. A consideration of empirical work and recent theoretical proposals suggests that parietal and occipito-temporal regions may be differentially associated with state- and strength-based signals, respectively. We tested this parietal/occipito-temporal state/strength hypothesis using fMRI and a visual perception task that allows separation of state- and strength-based perception. Participants made same/different judgments on pairs of faces and scenes using a 6-point confidence scale where "6" responses indicated a state of perceiving specific details that had changed, and "1" to "5" responses indicated judgments based on varying strength of relational match/mismatch. Regions in the lateral and medial posterior parietal cortex (supramarginal gyrus, posterior cingulate cortex, and precuneus) were sensitive to state-based perception and were not modulated by varying levels of strength-based perception. In contrast, bilateral fusiform gyrus activation was increased for strength-based "different" responses compared with misses and did not show state-based effects. Finally, the lateral occipital complex showed increased activation for state-based responses and additionally showed graded activation across levels of strength-based perception. These results offer support for a state/strength distinction between parietal and temporal regions, with the lateral occipital complex at the intersection of state- and strength-based processing.

Medial frontal cortex and anterior insula are less sensitive to outcome predictability when monetary stakes are higher

Prior research links greater activation of posterior medial frontal cortex (pMFC) and anterior insula (AI) with decreasing outcome predictability during decision making, as measured by decreasing probability for the more likely outcome out of two or increasing outcome variance. In addition to predictability, much work indicates that the magnitude or 'stakes' of the outcome is also important. Despite the interest in the neural correlates of these decision variables, it is unknown whether pMFC and AI are differentially sensitive to predictability when magnitude is varied. This study examined brain activity during decision making in relation to decreasing outcome predictability for low as compared with high magnitude decisions. For low magnitude decisions, reduced predictability of the outcome was associated with greater activity in pMFC and bilateral AI, replicating prior studies. In contrast, there was no relationship between predictability and brain activity for high magnitude decisions, which tended to elicit greater pMFC and AI activity than low magnitude decisions for more predictable outcomes. These data indicate that the relationship between outcome predictability and pMFC and AI activity during decision making depends on magnitude, and suggest that these regions may be responding to the motivational salience of the decision rather than predictability information per se.

Categorization = decision making + generalization

We rarely, if ever, repeatedly encounter exactly the same situation. This makes generalization crucial for real world decision making. We argue that categorization, the study of generalizable representations, is a type of decision making, and that categorization learning research would benefit from approaches developed to study the neuroscience of decision making. Similarly, methods developed to examine generalization and learning within the field of categorization may enhance decision making research. We first discuss perceptual information processing and integration, with an emphasis on accumulator models. We then examine learning the value of different decision making choices via experience, emphasizing reinforcement learning modeling approaches. Next we discuss how value is combined with other factors in decision making, emphasizing the effects of uncertainty. Finally, we describe how a final decision is selected via thresholding processes implemented by the basal ganglia and related regions. We also consider how memory related functions in the hippocampus may be integrated with decision making mechanisms and contribute to categorization.

Striatal and hippocampal entropy and recognition signals in category learning: simultaneous processes revealed by model-based fMRI

Category learning is a complex phenomenon that engages multiple cognitive processes, many of which occur simultaneously and unfold dynamically over time. For example, as people encounter objects in the world, they simultaneously engage processes to determine their fit with current knowledge structures, gather new information about the objects, and adjust their representations to support behavior in future encounters. Many techniques that are available to understand the neural basis of category learning assume that the multiple processes that subserve it can be neatly separated between different trials of an experiment. Model-based functional magnetic resonance imaging offers a promising tool to separate multiple, simultaneously occurring processes and bring the analysis of neuroimaging data more in line with category learning's dynamic and multifaceted nature. We use model-based imaging to explore the neural basis of recognition and entropy signals in the medial temporal lobe and striatum that are engaged while participants learn to categorize novel stimuli. Consistent with theories suggesting a role for the anterior hippocampus and ventral striatum in motivated learning in response to uncertainty, we find that activation in both regions correlates with a model-based measure of entropy. Simultaneously, separate subregions of the hippocampus and striatum exhibit activation correlated with a model-based recognition strength measure. Our results suggest that model-based analyses are exceptionally useful for extracting information about cognitive processes from neuroimaging data. Models provide a basis for identifying the multiple neural processes that contribute to behavior, and neuroimaging data can provide a powerful test bed for constraining and testing model predictions.

Knowing how much you don't know: a neural organization of uncertainty estimates

How we estimate uncertainty is important in decision neuroscience and has wide-ranging implications in basic and clinical neuroscience, from computational models of optimality to ideas on psychopathological disorders including anxiety, depression and schizophrenia. Empirical research in neuroscience, which has been based on divergent theoretical assumptions, has focused on the fundamental question of how uncertainty is encoded in the brain and how it influences behaviour. Here, we integrate several theoretical concepts about uncertainty into a decision-making framework. We conclude that the currently available evidence indicates that distinct neural encoding (including summary statistic-type representations) of uncertainty occurs in distinct neural systems.

Object recognition in clutter: cortical responses depend on the type of learning

Theoretical studies suggest that the visual system uses prior knowledge of visual objects to recognize them in visual clutter, and posit that the strategies for recognizing objects in clutter may differ depending on whether or not the object was learned in clutter to begin with. We tested this hypothesis using functional magnetic resonance imaging (fMRI) of human subjects. We trained subjects to recognize naturalistic, yet novel objects in strong or weak clutter. We then tested subjects' recognition performance for both sets of objects in strong clutter. We found many brain regions that were differentially responsive to objects during object recognition depending on whether they were learned in strong or weak clutter. In particular, the responses of the left fusiform gyrus (FG) reliably reflected, on a trial-to-trial basis, subjects' object recognition performance for objects learned in the presence of strong clutter. These results indicate that the visual system does not use a single, general-purpose mechanism to cope with clutter. Instead, there are two distinct spatial patterns of activation whose responses are attributable not to the visual context in which the objects were seen, but to the context in which the objects were learned.

Striatal activations signal prediction errors on confidence in the absence of external feedback

Research on the neural bases of learning has mainly focused on reinforcement learning where the central role of the dopaminergic system is well established. However, in everyday life many decisions are not followed by feedback, in which case humans have been shown to code the most probable outcome into memory. We used functional magnetic resonance imaging (fMRI) to examine the neural basis of internally generated signals on correctness and decision confidence in the complete absence of feedback in a categorization task. During test trials after observational training activation in dopaminergic target regions was modulated by the correctness of the answer similarly as during feedback-based training. Moreover, activation in the nucleus accumbens and putamen was correlated with the prediction error on confidence as estimated by a reinforcement learning model. In this model subjective confidence ratings acquired after each trial served as outcome measure. Activation in the striatum therefore follows a similar pattern in response to prediction errors on confidence as it does during reinforcement learning in response to reward prediction errors, but with respect to internally generated signals based on knowledge of the structure of the environment. Furthermore, ventral striatal activation decreased with stimulus novelty, which might support the allocation of attention to unfamiliar stimuli. These results provide a parsimonious account for the neural bases of learning, indicating overlapping neural substrates of reinforcement learning and learning when outcome information has to be internally constructed.

Learning shapes spatiotemporal brain patterns for flexible categorical decisions

Learning is thought to facilitate our ability to perform complex perceptual tasks and optimize brain circuits involved in decision making. However, little is known about the experience-dependent mechanisms in the human brain that support our ability to make fine categorical judgments. Previous work has focused on identifying spatial brain patterns (i.e., areas) that change with learning. Here, we take advantage of the complementary high spatial and temporal resolution of simultaneous electroencephalography-functional magnetic resonance imaging (EEG-fMRI) to identify the spatiotemporal dynamics between cortical networks involved in flexible category learning. Observers were trained to use different decision criteria (i.e., category boundaries) when making fine categorical judgments on morphed stimuli (i.e., radial vs. concentric patterns). Our findings demonstrate that learning acts on a feedback-based circuit that supports fine categorical judgments. Experience-dependent changes in the behavioral decision criterion were associated with changes in later perceptual processes engaging higher occipitotemporal and frontoparietal circuits. In contrast, category learning did not modulate early processes in a medial frontotemporal network that are thought to support the coarse interpretation of visual scenes. These findings provide evidence that learning flexible criteria for fine categorical judgments acts on distinct spatiotemporal brain circuits and shapes the readout of sensory signals that provide evidence for categorical decisions.

Differential effects of dopaminergic therapies on dorsal and ventral striatum in Parkinson's disease: implications for cognitive function

Cognitive abnormalities are a feature of Parkinson's disease (PD). Unlike motor symptoms that are clearly improved by dopaminergic therapy, the effect of dopamine replacement on cognition seems paradoxical. Some cognitive functions are improved whereas others are unaltered or even hindered. Our aim was to understand the effect of dopamine replacement therapy on various aspects of cognition. Whereas dorsal striatum receives dopamine input from the substantia nigra (SN), ventral striatum is innervated by dopamine-producing cells in the ventral tegmental area (VTA). In PD, degeneration of SN is substantially greater than cell loss in VTA and hence dopamine-deficiency is significantly greater in dorsal compared to ventral striatum. We suggest that dopamine supplementation improves functions mediated by dorsal striatum and impairs, or heightens to a pathological degree, operations ascribed to ventral striatum. We consider the extant literature in light of this principle. We also survey the effect of dopamine replacement on functional neuroimaging in PD relating the findings to this framework. This paper highlights the fact that currently, titration of therapy in PD is geared to optimizing dorsal striatum-mediated motor symptoms, at the expense of ventral striatum operations. Increased awareness of contrasting effects of dopamine replacement on dorsal versus ventral striatum functions will lead clinicians to survey a broader range of symptoms in determining optimal therapy, taking into account both those aspects of cognition that will be helped versus those that will be hindered by dopaminergic treatment.