Striatal activity underlies novelty-based choice in humans

Novelty enhances visual salience independently of reward in the parietal lobe

Novelty modulates sensory and reward processes, but it remains unknown how these effects interact, i.e., how the visual effects of novelty are related to its motivational effects. A widespread hypothesis, based on findings that novelty activates reward-related structures, is that all the effects of novelty are explained in terms of reward. According to this idea, a novel stimulus is by default assigned high reward value and hence high salience, but this salience rapidly decreases if the stimulus signals a negative outcome. Here we show that, contrary to this idea, novelty affects visual salience in the monkey lateral intraparietal area (LIP) in ways that are independent of expected reward. Monkeys viewed peripheral visual cues that were novel or familiar (received few or many exposures) and predicted whether the trial will have a positive or a negative outcome--i.e., end in a reward or a lack of reward. We used a saccade-based assay to detect whether the cues automatically attracted or repelled attention from their visual field location. We show that salience--measured in saccades and LIP responses--was enhanced by both novelty and positive reward associations, but these factors were dissociable and habituated on different timescales. The monkeys rapidly recognized that a novel stimulus signaled a negative outcome (and withheld anticipatory licking within the first few presentations), but the salience of that stimulus remained high for multiple subsequent presentations. Therefore, novelty can provide an intrinsic bonus for attention that extends beyond the first presentation and is independent of physical rewards.

Dopamine modulates novelty seeking behavior during decision making

Novelty seeking refers to the tendency of humans and animals to explore novel and unfamiliar stimuli and environments. The idea that dopamine modulates novelty seeking is supported by evidence that novel stimuli excite dopamine neurons and activate brain regions receiving dopaminergic input. In addition, dopamine is shown to drive exploratory behavior in novel environments. It is not clear whether dopamine promotes novelty seeking when it is framed as the decision to explore novel options versus the exploitation of familiar options. To test this hypothesis, we administered systemic injections of saline or GBR-12909, a selective dopamine transporter (DAT) inhibitor, to monkeys and assessed their novelty seeking behavior during a probabilistic decision making task. The task involved pseudorandom introductions of novel choice options. This allowed monkeys the opportunity to explore novel options or to exploit familiar options that they had already sampled. We found that DAT blockade increased the monkeys' preference for novel options. A reinforcement learning (RL) model fit to the monkeys' choice data showed that increased novelty seeking after DAT blockade was driven by an increase in the initial value the monkeys assigned to novel options. However, blocking DAT did not modulate the rate at which the monkeys learned which cues were most predictive of reward or their tendency to exploit that knowledge. These data demonstrate that dopamine enhances novelty-driven value and imply that excessive novelty seeking-characteristic of impulsivity and behavioral addictions-might be caused by increases in dopamine, stemming from less reuptake.

Cerebellar and prefrontal cortex contributions to adaptation, strategies, and reinforcement learning

Traditionally, motor learning has been studied as an implicit learning process, one in which movement errors are used to improve performance in a continuous, gradual manner. The cerebellum figures prominently in this literature given well-established ideas about the role of this system in error-based learning and the production of automatized skills. Recent developments have brought into focus the relevance of multiple learning mechanisms for sensorimotor learning. These include processes involving repetition, reinforcement learning, and strategy utilization. We examine these developments, considering their implications for understanding cerebellar function and how this structure interacts with other neural systems to support motor learning. Converging lines of evidence from behavioral, computational, and neuropsychological studies suggest a fundamental distinction between processes that use error information to improve action execution or action selection. While the cerebellum is clearly linked to the former, its role in the latter remains an open question.

Novelty or surprise?

Novelty and surprise play significant roles in animal behavior and in attempts to understand the neural mechanisms underlying it. They also play important roles in technology, where detecting observations that are novel or surprising is central to many applications, such as medical diagnosis, text processing, surveillance, and security. Theories of motivation, particularly of intrinsic motivation, place novelty and surprise among the primary factors that arouse interest, motivate exploratory or avoidance behavior, and drive learning. In many of these studies, novelty and surprise are not distinguished from one another: the words are used more-or-less interchangeably. However, while undeniably closely related, novelty and surprise are very different. The purpose of this article is first to highlight the differences between novelty and surprise and to discuss how they are related by presenting an extensive review of mathematical and computational proposals related to them, and then to explore the implications of this for understanding behavioral and neuroscience data. We argue that opportunities for improved understanding of behavior and its neural basis are likely being missed by failing to distinguish between novelty and surprise.

Cortical and hippocampal correlates of deliberation during model-based decisions for rewards in humans

How do we use our memories of the past to guide decisions we've never had to make before? Although extensive work describes how the brain learns to repeat rewarded actions, decisions can also be influenced by associations between stimuli or events not directly involving reward — such as when planning routes using a cognitive map or chess moves using predicted countermoves — and these sorts of associations are critical when deciding among novel options. This process is known as model-based decision making. While the learning of environmental relations that might support model-based decisions is well studied, and separately this sort of information has been inferred to impact decisions, there is little evidence concerning the full cycle by which such associations are acquired and drive choices. Of particular interest is whether decisions are directly supported by the same mnemonic systems characterized for relational learning more generally, or instead rely on other, specialized representations. Here, building on our previous work, which isolated dual representations underlying sequential predictive learning, we directly demonstrate that one such representation, encoded by the hippocampal memory system and adjacent cortical structures, supports goal-directed decisions. Using interleaved learning and decision tasks, we monitor predictive learning directly and also trace its influence on decisions for reward. We quantitatively compare the learning processes underlying multiple behavioral and fMRI observables using computational model fits. Across both tasks, a quantitatively consistent learning process explains reaction times, choices, and both expectation- and surprise-related neural activity. The same hippocampal and ventral stream regions engaged in anticipating stimuli during learning are also engaged in proportion to the difficulty of decisions. These results support a role for predictive associations learned by the hippocampal memory system to be recalled during choice formation.

We are always learning regularities in the world around us: where things are, and in what order we might find them. Our knowledge of these contingencies can be relied upon if we later want to use them to make decisions. However, there is little agreement about the neurobiological mechanism by which learned contingencies are deployed for decision making. These are different kinds of decisions than simple habits, in which we take actions that have in the past given us reward. Neural mechanisms of habitual decisions are well-described by computational reinforcement learning approaches, but have not often been applied to ‘model-based’ decisions that depend on learned contingencies. In this article, we apply reinforcement learning to investigate model-based decisions. We tested participants on a serial reaction time task with changing sequential contingencies, and choice probes that depend on these contingencies. Fitting computational models to reaction times, we show that two sets of predictions drive simple response behavior, only one of which is used to make choices. Using fMRI, we observed learning and decision-related activity in hippocampal and ventral cortical areas that is computationally linked to the learned contingencies used to make choices. These results suggest a critical role for a hippocampal-cortical network in model-based decisions for reward.

Impulsive and compulsive behaviors in Parkinson's disease

Impulsive-compulsive behaviors (ICBs) in Parkinson's disease (PD) are a common and devastating side effect of dopamine replacement therapy. In this review we describe the phenomenology, prevalence, and risk factors of patients with PD. Results of behavioral studies assessing the neuropsychological profile of patients with PD emphasize that the ICBs, which are behavioral addictions, are not hedonically motivated. Rather, other factors such as the inability to cope with uncertainty may be triggering ICBs. New insights from functional imaging studies, strengthening the incentive salience hypothesis, are discussed, and therapeutic guidelines for the management of ICBs in PD are given.

Which is the best intrinsic motivation signal for learning multiple skills?

Humans and other biological agents are able to autonomously learn and cache different skills in the absence of any biological pressure or any assigned task. In this respect, Intrinsic Motivations (i.e., motivations not connected to reward-related stimuli) play a cardinal role in animal learning, and can be considered as a fundamental tool for developing more autonomous and more adaptive artificial agents. In this work, we provide an exhaustive analysis of a scarcely investigated problem: which kind of IM reinforcement signal is the most suitable for driving the acquisition of multiple skills in the shortest time? To this purpose we implemented an artificial agent with a hierarchical architecture that allows to learn and cache different skills. We tested the system in a setup with continuous states and actions, in particular, with a kinematic robotic arm that has to learn different reaching tasks. We compare the results of different versions of the system driven by several different intrinsic motivation signals. The results show (a) that intrinsic reinforcements purely based on the knowledge of the system are not appropriate to guide the acquisition of multiple skills, and (b) that the stronger the link between the IM signal and the competence of the system, the better the performance.

The effect of visual salience on memory-based choices

Deciding whether a stimulus is the "same" or "different" from a previous presented one involves integrating among the incoming sensory information, working memory, and perceptual decision making. Visual selective attention plays a crucial role in selecting the relevant information that informs a subsequent course of action. Previous studies have mainly investigated the role of visual attention during the encoding phase of working memory tasks. In this study, we investigate whether manipulation of bottom-up attention by changing stimulus visual salience impacts on later stages of memory-based decisions. In two experiments, we asked subjects to identify whether a stimulus had either the same or a different feature to that of a memorized sample. We manipulated visual salience of the test stimuli by varying a task-irrelevant feature contrast. Subjects chose a visually salient item more often when they looked for matching features and less often so when they looked for a nonmatch. This pattern of results indicates that salient items are more likely to be identified as a match. We interpret the findings in terms of capacity limitations at a comparison stage where a visually salient item is more likely to exhaust resources leading it to be prematurely parsed as a match.

Behavioral endophenotypes of drug addiction: Etiological insights from neuroimaging studies

This article reviews recent advances in the elucidation of neurobehavioral endophenotypes associated with drug addiction made possible by the translational neuroimaging techniques magnetic resonance imaging (MRI) and positron emission tomography (PET). Increasingly, these non-invasive imaging approaches have been the catalyst for advancing our understanding of the etiology of drug addiction as a brain disorder involving complex interactions between pre-disposing behavioral traits, environmental influences and neural perturbations arising from the chronic abuse of licit and illicit drugs. In this article we discuss the causal role of trait markers associated with impulsivity and novelty-/sensation-seeking in speeding the development of compulsive drug administration and in facilitating relapse. We also discuss the striking convergence of imaging findings from these behavioural traits and addiction in rats, monkeys and humans with a focus on biomarkers of dopamine neurotransmission, and highlight areas where further research is needed to disambiguate underlying causal mechanisms. This article is part of a Special Issue entitled 'NIDA 40th Anniversary Issue'.

Neural correlates of the perception for novel objects

Perception of novel objects is of enormous importance in our lives. People have to perceive or understand novel objects when seeing an original painting, admiring an unconventional construction, and using an inventive device. However, very little is known about neural mechanisms underlying the perception for novel objects. Perception of novel objects relies on the integration of unusual features of novel objects in order to identify what such objects are. In the present study, functional Magnetic Resonance Imaging (MRI) was employed to investigate neural correlates of perception of novel objects. The neuroimaging data on participants engaged in novel object viewing versus ordinary object viewing revealed that perception of novel objects involves significant activation in the left precuneus (Brodmann area 7) and the right visual cortex. The results suggest that the left precuneus is associated with the integration of unusual features of novel objects, while the right visual cortex is sensitive to the detection of such features. Our findings highlight the left precuneus as a crucial component of the neural circuitry underlying perception of novel objects.

Impulsivity and apathy in Parkinson's disease

Impulse control disorders (ICDs) and apathy are recognized as two important neuropsychiatric syndromes associated with Parkinson's disease (PD), but as yet we understand very little about the cognitive mechanisms underlying them. Here, we review emerging findings, from both human and animal studies, that suggest that impulsivity and apathy are opposite extremes of a dopamine-dependent spectrum of motivated decision making. We first argue that there is strong support for a hypodopaminergic state in PD patients with apathy, as well as for an association between dopamine therapy and development of ICDs. However, there is little evidence for a clear dose-response relationship, and great heterogeneity of findings. We argue that dopaminergic state on its own is an insufficient explanation, and suggest instead that there is now substantial evidence that both apathy and impulsivity are in fact multi-dimensional syndromes, with separate, dissociable mechanisms underlying their 'surface' manifestations. Some of these mechanisms might be dopamine-dependent. According to this view, individuals diagnosed as impulsive or apathetic may have very different mechanisms underlying their clinical states. We propose that impulsivity and apathy can arise from dissociable deficits in option generation, option selection, action initiation or inhibition and learning. Review of the behavioural and neurobiological evidence leads us to a new conceptual framework that might help understand the variety of functional deficits seen in PD.

Reputational priors magnify striatal responses to violations of trust

Humans learn to trust each other by evaluating the outcomes of repeated interpersonal interactions. However, available prior information on the reputation of traders may alter the way outcomes affect learning. Our functional magnetic resonance imaging study is the first to allow the direct comparison of interaction-based and prior-based learning. Twenty participants played repeated trust games with anonymous counterparts. We manipulated two experimental conditions: whether or not reputational priors were provided, and whether counterparts were generally trustworthy or untrustworthy. When no prior information is available our results are consistent with previous studies in showing that striatal activation patterns correlate with behaviorally estimated reinforcement learning measures. However, our study additionally shows that this correlation is disrupted when reputational priors on counterparts are provided. Indeed participants continue to rely on priors even when experience sheds doubt on their accuracy. Notably, violations of trust from a cooperative counterpart elicited stronger caudate deactivations when priors were available than when they were not. However, tolerance to such violations appeared to be mediated by prior-enhanced connectivity between the caudate nucleus and ventrolateral prefrontal cortex, which anticorrelated with retaliation rates. Moreover, on top of affecting learning mechanisms, priors also clearly oriented initial decisions to trust, reflected in medial prefrontal cortex activity.

Uncertainty about mapping future actions into rewards may underlie performance on multiple measures of impulsivity in behavioral addiction: evidence from Parkinson's disease

A subset of patients with Parkinson's disease (PD) develops behavioral addictions, which may be due to their dopamine replacement therapy. Recently, several groups have been comparing PD patients with and without behavioral addictions on tasks that are thought to measure aspects of impulsivity. Several of these experiments, including information sampling, a bias toward novel stimuli and temporal discounting, have shown differences between PD patients with and without behavioral addictions. We have developed a unifying theoretical framework that allows us to model behavior in all three of these tasks. By exploring the performance of the patient groups on the three tasks with a single framework, we can ask questions about common mechanisms that underlie all three. Our results suggest that the effects seen in all three tasks can be accounted for by uncertainty about the ability to map future actions into rewards. More specifically, the modeling is consistent with the hypothesis that the group with behavioral addictions behaves as if they cannot use information provided within the experimental context to improve future reward guided actions. Future studies will be necessary to more firmly establish (or refute) this hypothesis. We discuss this result in light of what is known about the pathology that underlies the behavioral addictions in the PD patients.

Dissociating hippocampal and striatal contributions to sequential prediction learning

Behavior may be generated on the basis of many different kinds of learned contingencies. For instance, responses could be guided by the direct association between a stimulus and response, or by sequential stimulus-stimulus relationships (as in model-based reinforcement learning or goal-directed actions). However, the neural architecture underlying sequential predictive learning is not well understood, in part because it is difficult to isolate its effect on choice behavior. To track such learning more directly, we examined reaction times (RTs) in a probabilistic sequential picture identification task in healthy individuals. We used computational learning models to isolate trial-by-trial effects of two distinct learning processes in behavior, and used these as signatures to analyse the separate neural substrates of each process. RTs were best explained via the combination of two delta rule learning processes with different learning rates. To examine neural manifestations of these learning processes, we used functional magnetic resonance imaging to seek correlates of time-series related to expectancy or surprise. We observed such correlates in two regions, hippocampus and striatum. By estimating the learning rates best explaining each signal, we verified that they were uniquely associated with one of the two distinct processes identified behaviorally. These differential correlates suggest that complementary anticipatory functions drive each region's effect on behavior. Our results provide novel insights as to the quantitative computational distinctions between medial temporal and basal ganglia learning networks and enable experiments that exploit trial-by-trial measurement of the unique contributions of both hippocampus and striatum to response behavior.

Generalization of value in reinforcement learning by humans

Research in decision-making has focused on the role of dopamine and its striatal targets in guiding choices via learned stimulus-reward or stimulus-response associations, behavior that is well described by reinforcement learning theories. However, basic reinforcement learning is relatively limited in scope and does not explain how learning about stimulus regularities or relations may guide decision-making. A candidate mechanism for this type of learning comes from the domain of memory, which has highlighted a role for the hippocampus in learning of stimulus-stimulus relations, typically dissociated from the role of the striatum in stimulus-response learning. Here, we used functional magnetic resonance imaging and computational model-based analyses to examine the joint contributions of these mechanisms to reinforcement learning. Humans performed a reinforcement learning task with added relational structure, modeled after tasks used to isolate hippocampal contributions to memory. On each trial participants chose one of four options, but the reward probabilities for pairs of options were correlated across trials. This (uninstructed) relationship between pairs of options potentially enabled an observer to learn about option values based on experience with the other options and to generalize across them. We observed blood oxygen level-dependent (BOLD) activity related to learning in the striatum and also in the hippocampus. By comparing a basic reinforcement learning model to one augmented to allow feedback to generalize between correlated options, we tested whether choice behavior and BOLD activity were influenced by the opportunity to generalize across correlated options. Although such generalization goes beyond standard computational accounts of reinforcement learning and striatal BOLD, both choices and striatal BOLD activity were better explained by the augmented model. Consistent with the hypothesized role for the hippocampus in this generalization, functional connectivity between the ventral striatum and hippocampus was modulated, across participants, by the ability of the augmented model to capture participants' choice. Our results thus point toward an interactive model in which striatal reinforcement learning systems may employ relational representations typically associated with the hippocampus.

The effect of novelty on reinforcement learning

Recent research suggests that novelty has an influence on reward-related learning. Here, we showed that novel stimuli presented from a pre-familiarized category can accelerate or decelerate learning of the most rewarding category, depending on the condition. The extent of this influence depended on the individual trait of novelty seeking. Different reinforcement learning models were developed to quantify subjects' choices. We introduced a bias parameter to model explorative behavior toward novel stimuli and characterize individual variation in novelty response. The theoretical framework allowed us to test different assumptions, concerning the motivational value of novelty. The best fitting-model combined all novelty components and had a significant positive correlation with both the experimentally measured novelty bias and the independent novelty-seeking trait. Altogether, we have not only shown that novelty by itself enhances behavioral responses underlying reward processing, but also that novelty has a direct influence on reward-dependent learning processes, consistently with computational predictions.

Stimulant drugs trigger transient volumetric changes in the human ventral striatum

The ventral striatum (VStr) integrates mesolimbic dopaminergic and corticolimbic glutamatergic afferents and forms an essential component of the neural circuitry regulating impulsive behaviour. This structure represents a primary target of psychostimulant medication, the first-choice treatment for attention-deficit/hyperactivity disorder (ADHD), and is biochemically modified by these drugs in animals. However, the effects of stimulants on the human VStr remain to be determined. We acquired anatomical brain MRI scans from 23 never-medicated adult patients with ADHD, 31 adult patients with a history of stimulant treatment and 32 control subjects, and VStr volumes were determined using individual rater-blinded region of interest delineation on high-resolution neuroanatomical scans. Furthermore, we also extracted VStr volumes before and after methylphenidate treatment in a subsample of the medication-naïve adult patients as well as in 20 never-medicated children with ADHD. We observed smaller VStr volumes in adult patients with a history of stimulant treatment in comparison to never-medicated patients. Moreover, our longitudinal analyses uncovered a reduction of grey matter volume in the bilateral VStr in adult patients after exposure to methylphenidate, which was followed by volumetric recovery to control level. In children, the same pattern of VStr volume changes was observed after treatment with methylphenidate. These findings suggest that the altered VStr volumes previously observed in patients with ADHD may represent a transitory effect of stimulant exposure rather than an intrinsic feature of the disorder. More generally, these data show that stimulant drugs can render plastic volume changes in human VStr neuroanatomy.

Striatal and hippocampal entropy and recognition signals in category learning: simultaneous processes revealed by model-based fMRI

Category learning is a complex phenomenon that engages multiple cognitive processes, many of which occur simultaneously and unfold dynamically over time. For example, as people encounter objects in the world, they simultaneously engage processes to determine their fit with current knowledge structures, gather new information about the objects, and adjust their representations to support behavior in future encounters. Many techniques that are available to understand the neural basis of category learning assume that the multiple processes that subserve it can be neatly separated between different trials of an experiment. Model-based functional magnetic resonance imaging offers a promising tool to separate multiple, simultaneously occurring processes and bring the analysis of neuroimaging data more in line with category learning's dynamic and multifaceted nature. We use model-based imaging to explore the neural basis of recognition and entropy signals in the medial temporal lobe and striatum that are engaged while participants learn to categorize novel stimuli. Consistent with theories suggesting a role for the anterior hippocampus and ventral striatum in motivated learning in response to uncertainty, we find that activation in both regions correlates with a model-based measure of entropy. Simultaneously, separate subregions of the hippocampus and striatum exhibit activation correlated with a model-based recognition strength measure. Our results suggest that model-based analyses are exceptionally useful for extracting information about cognitive processes from neuroimaging data. Models provide a basis for identifying the multiple neural processes that contribute to behavior, and neuroimaging data can provide a powerful test bed for constraining and testing model predictions.

Striatal contributions to declarative memory retrieval

Declarative memory is known to depend on the medial temporal lobe memory system. Recently, there has been renewed focus on the relationship between the basal ganglia and declarative memory, including the involvement of striatum. However, the contribution of striatum to declarative memory retrieval remains unknown. Here, we review neuroimaging and neuropsychological evidence for the involvement of the striatum in declarative memory retrieval. From this review, we propose that, along with the prefrontal cortex (PFC), the striatum primarily supports cognitive control of memory retrieval. We conclude by proposing three hypotheses for the specific role of striatum in retrieval: (1) striatum modulates the re-encoding of retrieved items in accord with their expected utility (adaptive encoding), (2) striatum selectively admits information into working memory that is expected to increase the likelihood of successful retrieval (adaptive gating), and (3) striatum enacts adjustments in cognitive control based on the outcome of retrieval (reinforcement learning).

Attention, learning, and the value of information

Despite many studies on selective attention, fundamental questions remain about its nature and neural mechanisms. Here I draw from the animal and machine learning fields that describe attention as a mechanism for active learning and uncertainty reduction and explore the implications of this view for understanding visual attention and eye movement control. I propose that a closer integration of these different views has the potential greatly to expand our understanding of oculomotor control and our ability to use this system as a window into high level but poorly understood cognitive functions, including the capacity for curiosity and exploration and for inferring internal models of the external world.

Contributions of the striatum to learning, motivation, and performance: an associative account

It has long been recognized that the striatum is composed of distinct functional sub-units that are part of multiple cortico-striatal-thalamic circuits. Contemporary research has focused on the contribution of striatal sub-regions to three main phenomena: learning of associations between stimuli, actions and rewards; selection between competing response alternatives; and motivational modulation of motor behavior. Recent proposals have argued for a functional division of the striatum along these lines, attributing, for example, learning to one region and performance to another. Here, we consider empirical data from human and animal studies, as well as theoretical notions from both the psychological and computational literatures, and conclude that striatal sub-regions instead differ most clearly in terms of the associations being encoded in each region.

Functional brain networks associated with cognitive control, cocaine dependence, and treatment outcome

Individuals with cocaine dependence often evidence poor cognitive control. The purpose of this exploratory study was to investigate networks of functional connectivity underlying cognitive control in cocaine dependence and examine the relationship of the networks to the disorder and its treatment. Independent component analysis (ICA) was applied to fMRI data to investigate if regional activations underlying cognitive control processes operate in functional networks, and whether these networks relate to performance and treatment outcome measures in cocaine dependence. Twenty patients completed a Stroop task during fMRI prior to entering outpatient treatment and were compared to 20 control participants. ICA identified five distinct functional networks related to cognitive control interference events. Cocaine-dependent patients displayed differences in performance-related recruitment of three networks. Reduced involvement of a "top-down" fronto-cingular network contributing to conflict monitoring correlated with better treatment retention. Greater engagement of two "bottom-up" subcortical and ventral prefrontal networks related to cue-elicited motivational processing correlated with abstinence during treatment. The identification of subcortical networks linked to cocaine abstinence and cortical networks to treatment retention suggests that specific circuits may represent important, complementary targets in treatment development for cocaine dependence.

Neural correlates of dynamically evolving interpersonal ties predict prosocial behavior

There is a growing interest for the determinants of human choice behavior in social settings. Upon initial contact, investment choices in social settings can be inherently risky, as the degree to which the other person will reciprocate is unknown. Nevertheless, people have been shown to exhibit prosocial behavior even in one-shot laboratory settings where all interaction has been taken away. A logical step has been to link such behavior to trait empathy-related neurobiological networks. However, as a social interaction unfolds, the degree of uncertainty with respect to the expected payoff of choice behavior may change as a function of the interaction. Here we attempt to capture this factor. We show that the interpersonal tie one develops with another person during interaction – rather than trait empathy – motivates investment in a public good that is shared with an anonymous interaction partner. We examined how individual differences in trait empathy and interpersonal ties modulate neural responses to imposed monetary sharing. After, but not before interaction in a public good game, sharing prompted activation of neural systems associated with reward (striatum), empathy (anterior insular cortex and anterior cingulate cortex) as well as altruism, and social significance [posterior superior temporal sulcus (pSTS)]. Although these activations could be linked to both empathy and interpersonal ties, only tie-related pSTS activation predicted prosocial behavior during subsequent interaction, suggesting a neural substrate for keeping track of social relevance.

Distinction between Externally vs. Internally Guided Decision-Making: Operational Differences, Meta-Analytical Comparisons and Their Theoretical Implications

Most experimental studies of decision-making have specifically examined situations in which a single less-predictable correct answer exists (externally guided decision-making under uncertainty). Along with such externally guided decision-making, there are instances of decision-making in which no correct answer based on external circumstances is available for the subject (internally guided decision-making). Such decisions are usually made in the context of moral decision-making as well as in preference judgment, where the answer depends on the subject's own, i.e., internal, preferences rather than on external, i.e., circumstantial, criteria. The neuronal and psychological mechanisms that allow guidance of decisions based on more internally oriented criteria in the absence of external ones remain unclear. This study was undertaken to compare decision-making of these two kinds empirically and theoretically. First, we reviewed studies of decision-making to clarify experimental-operational differences between externally guided and internally guided decision-making. Second, using multi-level kernel density analysis, a whole-brain-based quantitative meta-analysis of neuroimaging studies was performed. Our meta-analysis revealed that the neural network used predominantly for internally guided decision-making differs from that for externally guided decision-making under uncertainty. This result suggests that studying only externally guided decision-making under uncertainty is insufficient to account for decision-making processes in the brain. Finally, based on the review and results of the meta-analysis, we discuss the differences and relations between decision-making of these two types in terms of their operational, neuronal, and theoretical characteristics.

Neural mechanisms underlying the induction and relief of perceptual curiosity

Curiosity is one of the most basic biological drives in both animals and humans, and has been identified as a key motive for learning and discovery. Despite the importance of curiosity and related behaviors, the topic has been largely neglected in human neuroscience; hence little is known about the neurobiological mechanisms underlying curiosity. We used functional magnetic resonance imaging (fMRI) to investigate what happens in our brain during the induction and subsequent relief of perceptual curiosity. Our core findings were that (1) the induction of perceptual curiosity, through the presentation of ambiguous visual input, activated the anterior insula and anterior cingulate cortex (ACC), brain regions sensitive to conflict and arousal; (2) the relief of perceptual curiosity, through visual disambiguation, activated regions of the striatum that have been related to reward processing; and (3) the relief of perceptual curiosity was associated with hippocampal activation and enhanced incidental memory. These findings provide the first demonstration of the neural basis of human perceptual curiosity. Our results provide neurobiological support for a classic psychological theory of curiosity, which holds that curiosity is an aversive condition of increased arousal whose termination is rewarding and facilitates memory.

Positive outcomes enhance incidental learning for both younger and older adults

Previous studies suggest that memory encoding is enhanced when people are anticipating a potential reward, consistent with the idea that dopaminergic systems that respond to motivationally relevant information also enhance memory for that information. In the current study, we examined how anticipating and receiving rewards versus losses affect incidental learning of information. In addition, we compared the modulatory effects of reward anticipation and outcome on memory for younger and older adults. Forty-two younger (aged 18–33 years) and 44 older (aged 66–92 years) adults played a game involving pressing a button as soon as they saw a target. Gain trials began with a cue that they would win $0.25 if they pressed the button fast enough, loss trials began with a cue that they would avoid losing $0.25 if they pressed the button fast enough, and no-outcome trials began with a cue indicating no monetary outcome. The target was a different photo-object on each trial (e.g., balloon, dolphin) and performance outcomes were displayed after the photo disappeared. Both younger and older adults recalled and recognized pictures from trials with positive outcomes (either rewarding or loss avoiding) better than from trials with negative outcomes. Positive outcomes were associated with not only enhanced memory for the picture just seen in that trial, but also with enhanced memory for the pictures shown in the next two trials. Although anticipating a reward also enhanced incidental memory, this effect was seen only in recognition memory of positive pictures and was a smaller effect than the outcome effect. The fact that older adults showed similar incidental memory effects of reward anticipation and outcome as younger adults suggests that reward–memory system interactions remain intact in older age.

Striatal topography of probability and magnitude information for decisions under uncertainty

Most decisions involve some element of uncertainty. When the outcomes of these decisions have different likelihoods of occurrence, the decision-maker must consider both the magnitude of each outcome and the probability of its occurrence, but how do individual decision makers combine the two dimensions of magnitude and probability? Here, we approach the problem by separating in time the presentation of magnitude and probability information, and focus the analysis of fMRI activations on the first piece of information only. Thus, we are able to identify distinct neural circuits for the two dimensions without the confounding effect of divided attention or the cognitive operation of combining them. We find that magnitude information correlates with the size of the response of the ventral striatum while probability information correlates with the response in the dorsal striatum. The relative responsiveness of these two striatal regions correlates with the behavioral tendency to weight one more than the other. The results are consistent with a second-order process of information aggregation in which individuals make separate judgments for magnitude and probability and then integrate those judgments.

Novelty seeking behaviour in Parkinson's disease

Novelty seeking can be a positive trait leading to creativity and innovation, but it is also related to increased risk of damaging addictive behaviour. We have assessed novelty seeking with a three armed bandit task, in which novel stimuli were occasionally introduced, replacing choice options from which the participants had been choosing. This allowed us to assess whether or not they would be prone to selecting novel stimuli. We tested 25 non impulsive patients with Parkinson's disease (PD) and 27 PD patients with impulsive compulsive behaviours (ICBs). Both patient groups were examined "on" and "off" dopaminergic medication in a counterbalanced order and their behaviour was compared with 24 healthy controls. We found that PD patients with ICBs were significantly more prone to choose novel options than either non impulsive PD patients or controls, regardless of medication status. Our findings suggest that attraction to novelty is a personality trait in all PD patients with ICBs which is independent of medication status.

Novelty increases the mesolimbic functional connectivity of the substantia nigra/ventral tegmental area (SN/VTA) during reward anticipation: Evidence from high-resolution fMRI

Reward and novelty are potent learning signals that critically rely on dopaminergic midbrain responses. Recent findings suggest that although reward and novelty are likely to interact, both functions may be subserved by distinct neuronal clusters. We used high-resolution functional magnetic resonance imaging (fMRI) to isolate neural responses to reward and novelty within the human substantia nigra/ventral tegmental area (SN/VTA) complex to investigate the spatial delineation and integration of reward- and novelty-related activity clusters. We demonstrate that distinct clusters within the caudal portion of the medial SN/VTA and the lateral portion of the right SN are predominantly modulated by the anticipation of reward, while a more rostral part of the medial SN/VTA was exclusively modulated by novelty. In addition, the caudal medial SN/VTA cluster embodied an interaction between novelty and reward where novelty selectively increased reward-anticipation responses. This interaction, in turn, was paralleled by differences in the functional-connectivity patterns of these SN/VTA regions. Specifically, novel as compared to familiar reward-predictive stimuli increased the functional connectivity of the medial SN/VTA with mesolimbic regions, including the nucleus accumbens and the hippocampus, as well as with the primary visual cortex. This functional correlation may highlight how afferents of the medial SN/VTA provide integrative information about novelty and reward, or, alternatively, how medial SN/VTA activity may modulate memory processes for novel events associated with rewards.

Neural correlates of forward planning in a spatial decision task in humans

Although reinforcement learning (RL) theories have been influential in characterizing the mechanisms for reward-guided choice in the brain, the predominant temporal difference (TD) algorithm cannot explain many flexible or goal-directed actions that have been demonstrated behaviorally. We investigate such actions by contrasting an RL algorithm that is model based, in that it relies on learning a map or model of the task and planning within it, to traditional model-free TD learning. To distinguish these approaches in humans, we used functional magnetic resonance imaging in a continuous spatial navigation task, in which frequent changes to the layout of the maze forced subjects continually to relearn their favored routes, thereby exposing the RL mechanisms used. We sought evidence for the neural substrates of such mechanisms by comparing choice behavior and blood oxygen level-dependent (BOLD) signals to decision variables extracted from simulations of either algorithm. Both choices and value-related BOLD signals in striatum, although most often associated with TD learning, were better explained by the model-based theory. Furthermore, predecessor quantities for the model-based value computation were correlated with BOLD signals in the medial temporal lobe and frontal cortex. These results point to a significant extension of both the computational and anatomical substrates for RL in the brain.

Neural correlates of sequence learning with stochastic feedback

Although much is known about decision making under uncertainty when only a single step is required in the decision process, less is known about sequential decision making. We carried out a stochastic sequence learning task in which subjects had to use noisy feedback to learn sequences of button presses. We compared flat and hierarchical behavioral models and found that although both models predicted the choices of the group of subjects equally well, only the hierarchical model correlated significantly with learning-related changes in the magneto-encephalographic response. The significant modulations in the magneto-encephalographic signal occurred 83 msec before button press and 67 msec after button press. We also localized the sources of these effects and found that the early effect localized to the insula, whereas the late effect localized to the premotor cortex.

Sleep deprivation amplifies reactivity of brain reward networks, biasing the appraisal of positive emotional experiences

Appropriate interpretation of pleasurable, rewarding experiences favors decisions that enhance survival. Conversely, dysfunctional affective brain processing can lead to life-threatening risk behaviors (e.g., addiction) and emotion imbalance (e.g., mood disorders). The state of sleep deprivation continues to be associated with maladaptive emotional regulation, leading to exaggerated neural and behavioral reactivity to negative, aversive experiences. However, such detrimental consequences are paradoxically aligned with the perplexing antidepressant benefit of sleep deprivation, elevating mood in a proportion of patients with major depression. Nevertheless, it remains unknown how sleep loss alters the dynamics of brain and behavioral reactivity to rewarding, positive emotional experiences. Using functional magnetic resonance imaging (fMRI), here we demonstrate that sleep deprivation amplifies reactivity throughout human mesolimbic reward brain networks in response to pleasure-evoking stimuli. In addition, this amplified reactivity was associated with enhanced connectivity in early primary visual processing pathways and extended limbic regions, yet with a reduction in coupling with medial frontal and orbitofrontal regions. These neural changes were accompanied by a biased increase in the number of emotional stimuli judged as pleasant in the sleep-deprived group, the extent of which exclusively correlated with activity in mesolimbic regions. Together, these data support a view that sleep deprivation not only is associated with enhanced reactivity toward negative stimuli, but imposes a bidirectional nature of affective imbalance, associated with amplified reward-relevant reactivity toward pleasure-evoking stimuli also. Such findings may offer a neural foundation on which to consider interactions between sleep loss and emotional reactivity in a variety of clinical mood disorders.

Motor Planning under Unpredictable Reward: Modulations of Movement Vigor and Primate Striatum Activity

Although reward probability is an important factor that shapes animal's behavior, it is not well understood how the brain translates reward expectation into the vigor of movement [reaction time (RT) and speed]. To address this question, we trained two monkeys in a RT task that required wrist movements in response to vibrotactile and visual stimuli, with a variable reward schedule. Correct performance was rewarded in 75% of the trials. Monkeys were certain that they would be rewarded only in the trials immediately following withheld rewards. In these trials, the animals responded sooner and moved faster. Single-unit recordings from the dorsal striatum revealed modulations in neural firing that reflected changes in movement vigor. First, in the trials with certain rewards, striatal neurons modulated their firing rates earlier. Second, magnitudes of changes in neuronal firing rates depended on whether or not monkeys were certain about the reward. Third, these modulations depended on the sensory modality of the cue (visual vs. vibratory) and/or movement direction (flexions vs. extensions). We conclude that dorsal striatum may be a part of the mechanism responsible for the modulation of movement vigor in response to changes of reward predictability.

Contextual interaction between novelty and reward processing within the mesolimbic system

Medial temporal lobe (MTL) dependent long‐term memory for novel events is modulated by a circuitry that also responds to reward and includes the ventral striatum, dopaminergic midbrain, and medial orbitofrontal cortex (mOFC). This common neural network may reflect a functional link between novelty and reward whereby novelty motivates exploration in the search for rewards; a link also termed novelty “exploration bonus.” We used fMRI in a scene encoding paradigm to investigate the interaction between novelty and reward with a focus on neural signals akin to an exploration bonus. As expected, reward related long‐term memory for the scenes (after 24 hours) strongly correlated with activity of MTL, ventral striatum, and substantia nigra/ventral tegmental area (SN/VTA). Furthermore, the hippocampus showed a main effect of novelty, the striatum showed a main effect of reward, and the mOFC signalled both novelty and reward. An interaction between novelty and reward akin to an exploration bonus was found in the hippocampus. These data suggest that MTL novelty signals are interpreted in terms of their reward‐predicting properties in the mOFC, which biases striatal reward responses. The striatum together with the SN/VTA then regulates MTL‐dependent long‐term memory formation and contextual exploration bonus signals in the hippocampus. Hum Brain Mapp, 2011. © 2011 Wiley‐Liss, Inc.

Model-based influences on humans' choices and striatal prediction errors

The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based planning, and the interaction between model-based and model-free values, prediction errors, and preferences is underexplored. We designed a multistep decision task in which model-based and model-free influences on human choice behavior could be distinguished. By showing that choices reflected both influences we could then test the purity of the ventral striatal BOLD signal as a model-free report. Contrary to expectations, the signal reflected both model-free and model-based predictions in proportions matching those that best explained choice behavior. These results challenge the notion of a separate model-free learner and suggest a more integrated computational architecture for high-level human decision-making.

Dopamine D₁ receptors and nonlinear probability weighting in risky choice

Misestimating risk could lead to disadvantaged choices such as initiation of drug use (or gambling) and transition to regular drug use (or gambling). Although the normative theory in decision-making under risks assumes that people typically take the probability-weighted expectation over possible utilities, experimental studies of choices among risks suggest that outcome probabilities are transformed nonlinearly into subjective decision weights by a nonlinear weighting function that overweights low probabilities and underweights high probabilities. Recent studies have revealed the neurocognitive mechanism of decision-making under risk. However, the role of modulatory neurotransmission in this process remains unclear. Using positron emission tomography, we directly investigated whether dopamine D₁ and D₂ receptors in the brain are associated with transformation of probabilities into decision weights in healthy volunteers. The binding of striatal D₁ receptors is negatively correlated with the degree of nonlinearity of weighting function. Individuals with lower striatal D₁ receptor density showed more pronounced overestimation of low probabilities and underestimation of high probabilities. This finding should contribute to a better understanding of the molecular mechanism of risky choice, and extreme or impaired decision-making observed in drug and gambling addiction.

Striatal responses to negative monetary outcomes differ between temperamentally inhibited and non-inhibited adolescents

The present study compared blood oxygen level dependent (BOLD) response in behaviorally inhibited and behaviorally non-inhibited adolescents to positive and negative feedback following their choice in a reward task. Previous data in these same subjects showed enhanced activation in striatal areas in behaviorally inhibited subjects to cues predicting gain or a loss. However, no analyses had examined responses following actual gains or losses. Relative to non-inhibited subjects, behaviorally inhibited subjects in the current study showed enhanced caudate response to negative but not positive feedback, indicating that striatal sensitivity to feedback may be specific to aversive information. In addition, compared to non-inhibited subjects, behaviorally inhibited subjects exhibited reduced differentiation between positive and negative feedback in ventromedial prefrontal cortex (vmPFC). This suggests a perturbed ability to encode reward value.

Risk taking in avalanche terrain: a study of the human factor contribution

OBJECTIVE

To discover possible associations between the human factors and avalanche incidents.

DESIGN

Self-report, intercept, and Web-based, 1-year retrospective, cross-sectional study.

SETTING

Mountain Equipment Co-op stores in Calgary and Vancouver, Canada.

PARTICIPANTS

People shopping at the store and who had entered avalanche terrain in the past 12 months were invited to complete the survey (n = 447).

INDEPENDENT VARIABLES

Sex, age, sport activity, days of exposure, years of experience, socioeconomic status, level of training, risk propensity, and motivation.

MAIN OUTCOME MEASURES

Experiencing an avalanche incident.

RESULTS

Women and those traveling with women were less likely to experience an avalanche incident [odds ratio (OR) = 0.45; 95% confidence interval (CI), 0.21-0.96]. Those with the most training were more likely to report experiencing an avalanche incident (OR = 6.86; 95% CI, 2.37-19.83), but this difference was attenuated (OR = 2.25) and not statistically significant (95% CI, 0.57-8.81) after adjustment for exposure. Experience was not found to be a factor. Being motivated to seek intense experiences was found to be a factor (OR = 2.19; 95% CI, 1.03-4.66), whereas being motivated to create memorable experiences was protective (OR = 0.29; 95% CI, 0.10-0.86).

CONCLUSIONS

The results of this study suggest that people exposing themselves to avalanche risk do so to satisfy inherited and learned motivational needs and that some motivations are associated with higher or lower risk taking than others. Training appears to be exploited so as to increase access to these benefits rather than reduce risk. Within this risk/reward paradigm, risk taking among men is moderated by the presence of a woman in the group.

Testing the reward prediction error hypothesis with an axiomatic model

Neuroimaging studies typically identify neural activity correlated with the predictions of highly parameterized models, like the many reward prediction error (RPE) models used to study reinforcement learning. Identified brain areas might encode RPEs or, alternatively, only have activity correlated with RPE model predictions. Here, we use an alternate axiomatic approach rooted in economic theory to formally test the entire class of RPE models on neural data. We show that measurements of human neural activity from the striatum, medial prefrontal cortex, amygdala, and posterior cingulate cortex satisfy necessary and sufficient conditions for the entire class of RPE models. However, activity measured from the anterior insula falsifies the axiomatic model, and therefore no RPE model can account for measured activity. Further analysis suggests the anterior insula might instead encode something related to the salience of an outcome. As cognitive neuroscience matures and models proliferate, formal approaches of this kind that assess entire model classes rather than specific model exemplars may take on increased significance.

Computational and dynamic models in neuroimaging

This article reviews the substantial impact computational neuroscience has had on neuroimaging over the past years. It builds on the distinction between models of the brain as a computational machine and computational models of neuronal dynamics per se; i.e., models of brain function and biophysics. Both sorts of model borrow heavily from computational neuroscience, and both have enriched the analysis of neuroimaging data and the type of questions we address. To illustrate the role of functional models in imaging neuroscience, we focus on optimal control and decision (game) theory; the models used here provide a mechanistic account of neuronal computations and the latent (mental) states represent by the brain. In terms of biophysical modelling, we focus on dynamic causal modelling, with a special emphasis on recent advances in neural-mass models for hemodynamic and electrophysiological time series. Each example emphasises the role of generative models, which embed our hypotheses or questions, and the importance of model comparison (i.e., hypothesis testing). We will refer to this theme, when trying to contextualise recent trends in relation to each other.

Alterations in choice behavior by manipulations of world model

How to compute initially unknown reward values makes up one of the key problems in reinforcement learning theory, with two basic approaches being used. Model-free algorithms rely on the accumulation of substantial amounts of experience to compute the value of actions, whereas in model-based learning, the agent seeks to learn the generative process for outcomes from which the value of actions can be predicted. Here we show that (i) "probability matching"-a consistent example of suboptimal choice behavior seen in humans-occurs in an optimal Bayesian model-based learner using a max decision rule that is initialized with ecologically plausible, but incorrect beliefs about the generative process for outcomes and (ii) human behavior can be strongly and predictably altered by the presence of cues suggestive of various generative processes, despite statistically identical outcome generation. These results suggest human decision making is rational and model based and not consistent with model-free learning.

The human basal ganglia modulate frontal-posterior connectivity during attention shifting

Current models of flexible cognitive control emphasize the role of the prefrontal cortex. This region has been shown to control attention by biasing information processing in favor of task-relevant representations. However, the prefrontal cortex does not act in isolation. We used functional magnetic resonance imaging combined with nonlinear dynamic causal modeling to demonstrate that the basal ganglia play a role in modulating the top-down influence of the prefrontal cortex on visual processing in humans. Specifically, our results reveal that connectivity between the prefrontal cortex and stimulus-specific visual association areas depends on activity in the ventral striatopallidum, elicited by salient events leading to shifts in attention. These data integrate disparate literatures on top-down control by the prefrontal cortex and selective gating by the basal ganglia and highlight the importance of the basal ganglia for high-level cognitive control.

Striatal prediction error modulates cortical coupling

Both perceptual inference and motor responses are shaped by learned probabilities. For example, stimulus-induced responses in sensory cortices and preparatory activity in premotor cortex reflect how (un)expected a stimulus is. This is in accordance with predictive coding accounts of brain function, which posit a fundamental role of prediction errors for learning and adaptive behavior. We used functional magnetic resonance imaging and recent advances in computational modeling to investigate how (failures of) learned predictions about visual stimuli influence subsequent motor responses. Healthy volunteers discriminated visual stimuli that were differentially predicted by auditory cues. Critically, the predictive strengths of cues varied over time, requiring subjects to continuously update estimates of stimulus probabilities. This online inference, modeled using a hierarchical Bayesian learner, was reflected behaviorally: speed and accuracy of motor responses increased significantly with predictability of the stimuli. We used nonlinear dynamic causal modeling to demonstrate that striatal prediction errors are used to tune functional coupling in cortical networks during learning. Specifically, the degree of striatal trial-by-trial prediction error activity controls the efficacy of visuomotor connections and thus the influence of surprising stimuli on premotor activity. This finding substantially advances our understanding of striatal function and provides direct empirical evidence for formal learning theories that posit a central role for prediction error-dependent plasticity.

Neural mechanisms of acquired phasic dopamine responses in learning

What biological mechanisms underlie the reward-predictive firing properties of midbrain dopaminergic neurons, and how do they relate to the complex constellation of empirical findings understood as Pavlovian and instrumental conditioning? We previously presented PVLV, a biologically inspired Pavlovian learning algorithm accounting for DA activity in terms of two interrelated systems: a primary value (PV) system, which governs how DA cells respond to a US (reward) and; a learned value (LV) system, which governs how DA cells respond to a CS. Here, we provide a more extensive review of the biological mechanisms supporting phasic DA firing and their relation to the spate of Pavlovian conditioning phenomena and their sensitivity to focal brain lesions. We further extend the model by incorporating a new NV (novelty value) component reflecting the ability of novel stimuli to trigger phasic DA firing, providing "novelty bonuses" which encourages exploratory working memory updating and in turn speeds learning in trace conditioning and other working memory-dependent paradigms. The evolving PVLV model builds upon insights developed in many earlier computational models, especially reinforcement learning models based on the ideas of Sutton and Barto, biological models, and the psychological model developed by Savastano and Miller. The PVLV framework synthesizes these various approaches, overcoming important shortcomings of each by providing a coherent and specific mapping to much of the relevant empirical data at both the micro- and macro-levels, and examines their relevance for higher order cognitive functions.

Intracranial EEG correlates of expectancy and memory formation in the human hippocampus and nucleus accumbens

The human brain is adept at anticipating upcoming events, but in a rapidly changing world, it is essential to detect and encode events that violate these expectancies. Unexpected events are more likely to be remembered than predictable events, but the underlying neural mechanisms for these effects remain unclear. We report intracranial EEG recordings from the hippocampus of epilepsy patients, and from the nucleus accumbens of depression patients. We found that unexpected stimuli enhance an early (187 ms) and a late (482 ms) hippocampal potential, and that the late potential is associated with successful memory encoding for these stimuli. Recordings from the nucleus accumbens revealed a late potential (peak at 475 ms), which increases in magnitude during unexpected items, but no subsequent memory effect and no early component. These results are consistent with the hypothesis that activity in a loop involving the hippocampus and the nucleus accumbens promotes encoding of unexpected events.

Contextual novelty changes reward representations in the striatum

Reward representation in ventral striatum is boosted by perceptual novelty, although the mechanism of this effect remains elusive. Animal studies indicate a functional loop (Lisman and Grace, 2005) that includes hippocampus, ventral striatum, and midbrain as being important in regulating salience attribution within the context of novel stimuli. According to this model, reward responses in ventral striatum or midbrain should be enhanced in the context of novelty even if reward and novelty constitute unrelated, independent events. Using fMRI, we show that trials with reward-predictive cues and subsequent outcomes elicit higher responses in the striatum if preceded by an unrelated novel picture, indicating that reward representation is enhanced in the context of novelty. Notably, this effect was observed solely when reward occurrence, and hence reward-related salience, was low. These findings support a view that contextual novelty enhances neural responses underlying reward representation in the striatum and concur with the effects of novelty processing as predicted by the model of Lisman and Grace (2005).