Does reward frequency or magnitude drive reinforcement-learning in attention-deficit/hyperactivity disorder?

Impaired flexible reward learning in ADHD patients is associated with blunted reinforcement sensitivity and neural signals in ventral striatum and parietal cortex

Reward-based learning and decision-making are prime candidates to understand symptoms of attention deficit hyperactivity disorder (ADHD). However, only limited evidence is available regarding the neurocomputational underpinnings of the alterations seen in ADHD. This concerns flexible behavioral adaption in dynamically changing environments, which is challenging for individuals with ADHD. One previous study points to elevated choice switching in adolescent ADHD, which was accompanied by disrupted learning signals in medial prefrontal cortex. Here, we investigated young adults with ADHD (n = 17) as compared to age- and sex-matched controls (n = 17) using a probabilistic reversal learning experiment during functional magnetic resonance imaging (fMRI). The task requires continuous learning to guide flexible behavioral adaptation to changing reward contingencies. To disentangle the neurocomputational underpinnings of the behavioral data, we used reinforcement learning (RL) models, which informed the analysis of fMRI data. ADHD patients performed worse than controls particularly in trials before reversals, i.e., when reward contingencies were stable. This pattern resulted from 'noisy' choice switching regardless of previous feedback. RL modelling showed decreased reinforcement sensitivity and enhanced learning rates for negative feedback in ADHD patients. At the neural level, this was reflected in a diminished representation of choice probability in the left posterior parietal cortex in ADHD. Moreover, modelling showed a marginal reduction of learning about the unchosen option, which was paralleled by a marginal reduction in learning signals incorporating the unchosen option in the left ventral striatum. Taken together, we show that impaired flexible behavior in ADHD is due to excessive choice switching ('hyper-flexibility'), which can be detrimental or beneficial depending on the learning environment. Computationally, this resulted from blunted sensitivity to reinforcement of which we detected neural correlates in the attention-control network, specifically in the parietal cortex. These neurocomputational findings remain preliminary due to the relatively small sample size.

Executive and motor function in children with autism spectrum disorder

The purpose of this study was to investigate the relationship between executive function, mainly attention and impulse control, and motor function impairment in children with ASD. Fifteen children with ASD (ages 8-14 years) were examined on their executive function using the Conners continuous performance test-3 and motor function using the movement assessment battery for children-2 (MABC-2). It was found that children with ASD were delayed on executive and motor function. This means that those effects were also observed in male children with Autism Spectrum Disorder (ASD) who had similar characteristics as the participants in the study. This study may have practical implications in the relationship between executive function and motor performance in children with ASD and for clinicians and educators working with children with ASD. Furthermore, the findings suggest that variability in reaction time may be a useful predictor of motor skill performance in children with ASD and may reflect underlying difficulties in executive function and future research is suggested.

Distinct reinforcement learning profiles distinguish between language and attentional neurodevelopmental disorders

BACKGROUND

Theoretical models posit abnormalities in cortico-striatal pathways in two of the most common neurodevelopmental disorders (Developmental dyslexia, DD, and Attention deficit hyperactive disorder, ADHD), but it is still unclear what distinct cortico-striatal dysfunction might distinguish language disorders from others that exhibit very different symptomatology. Although impairments in tasks that depend on the cortico-striatal network, including reinforcement learning (RL), have been implicated in both disorders, there has been little attempt to dissociate between different types of RL or to compare learning processes in these two types of disorders. The present study builds upon prior research indicating the existence of two learning manifestations of RL and evaluates whether these processes can be differentiated in language and attention deficit disorders. We used a two-step RL task shown to dissociate model-based from model-free learning in human learners.

RESULTS

Our results show that, relative to neurotypicals, DD individuals showed an impairment in model-free but not in model-based learning, whereas in ADHD the ability to use both model-free and model-based learning strategies was significantly compromised.

CONCLUSIONS

Thus, learning impairments in DD may be linked to a selective deficit in the ability to form action-outcome associations based on previous history, whereas in ADHD some learning deficits may be related to an incapacity to pursue rewards based on the tasks' structure. Our results indicate how different patterns of learning deficits may underlie different disorders, and how computation-minded experimental approaches can differentiate between them.

The impact of ADHD symptom severity on reinforcement and punishment learning among adults

Introduction: Aberrations in feedback learning are hypothesised to contribute to the behavioural disruptions and impairment of attention-deficit/hyperactivity disorder (ADHD). However, few studies have evaluated the relation of reward/punishment feedback and ADHD symptom severity on learning. The current study evaluates the differential effects of reward and punishment feedback on learning among adults with elevated ADHD. Methods: One hundred five participants self-reported their level of current ADHD symptoms and completed an innovative instrumental learning task. Results: Consistent with predictions, participants with low self-reported ADHD symptom severity benefitted equally from reward and punishment feedback during the learning task, whereas participants with high self-reported symptom severity performed better (indexed by accuracy on learning task) from reward than punishment feedback trials. Conclusions: Overall, adults with high self-reported symptom severity of ADHD learned more from reward-based feedback, which provides critical implications for motivational theories about ADHD, as well as for treatment protocols. Future work should examine the translatability of results within a treatment setting.

Decision-making in autism: A narrative review

LAY SUMMARY

Many autistic people report difficulties with real-life decision-making. However, when doing decision-making tests in laboratory experiments, autistic people often perform as well or better than non-autistic people. We review previously published studies on autistic people's decision-making, across different types of tests, to understand what type of decision-making is more challenging. To do this, we searched four databases of research papers. We found 104 studies that tested, in total, 2712 autistic and 3189 comparison participants on different decision-making tasks. We found that there were four categories of decision-making tests that were used in these experiments: perceptual (e.g. deciding which image has the most dots); reward learning (e.g. learning which deck of cards gives the best reward); metacognition (e.g. knowing how well you perform or what you want); and value-based (e.g. making a decision based on a choice between two outcomes that differ in value to you). Overall, these studies suggest that autistic and comparison participants tend to perform similarly well at perceptual and reward-learning decisions. However, autistic participants tended to decide differently from comparison participants on metacognition and value-based paradigms. This suggests that autistic people might differ from typically developing controls in how they evaluate their own performance and in how they make decisions based on weighing up the subjective value of two different options. We suggest these reflect more general differences in metacognition, thinking about thinking, in autism.

Reward modality modulates striatal responses to reward anticipation in ADHD: Effects of affiliative and food stimuli

Altered reward sensitivity has been proposed to underlie symptoms of attention deficit hyperactivity disorder (ADHD). Functional magnetic resonance imaging (fMRI) studies have reported hypoactivation to reward-predicting cues in the ventral striatum among individuals with ADHD, using experimental designs with and without behavioral response requirements. These studies have typically used monetary incentives as rewards; however, it is unclear if these findings extend to other reward types. The current study examined striatal responses to anticipation and delivery of both affiliative and food reward images using a classical conditioning paradigm. Data from 20 typically developing young adults, and 20 individuals diagnosed with ADHD were included in a region-of-interest analysis for a priori striatal regions. Consistent with findings from studies using monetary rewards, individuals with ADHD showed decreased activation to cues predicting affiliative rewards in the bilateral ventral and dorsal striatum and increased activation to the delivery of affiliative rewards in the ventral striatum. No group differences were found in striatal responses to food reward cues or images. These results suggest hyposensitivity to reward-predicting cues in ADHD extends to affiliative rewards, with important implications for understanding and managing the learning and social functioning of those with ADHD.

Responsivity of the Striatal Dopamine System to Methylphenidate-A Within-Subject I-123-β-CIT-SPECT Study in Male Children and Adolescents With Attention-Deficit/Hyperactivity Disorder

BACKGROUND

Methylphenidate (MPH) is the first-line pharmacological treatment of attention-deficit/hyperactivity disorder (ADHD). MPH binds to the dopamine (DA) transporter (DAT), which has high density in the striatum. Assessments of the striatal dopamine transporter by single positron emission computed tomography (SPECT) in childhood and adolescent patients are rare but can provide insight on how the effects of MPH affect DAT availability. The aim of our within-subject study was to investigate the effect of MPH on DAT availability and how responsivity to MPH in DAT availability is linked to clinical symptoms and cognitive functioning.

METHODS

Thirteen adolescent male patients (9-16 years) with a diagnosis of ADHD according to the DSM-IV and long-term stimulant medication (for at least 6 months) with MPH were assessed twice within 7 days using SPECT after application of I-123-β-CIT to examine DAT binding potential (DAT BP). SPECT measures took place in an on- and off-MPH status balanced for order across participants. A virtual reality continuous performance test was performed at each time point. Further clinical symptoms were assessed for baseline off-MPH.

RESULTS

On-MPH status was associated with a highly significant change (-29.9%) of striatal DAT BP as compared to off-MPH (t = -4.12, p = 0.002). A more pronounced change in striatal DAT BP was associated with higher off-MPH attentional and externalizing symptom ratings (Pearson r = 0.68, p = 0.01). Striatal DAT BP off-MPH, but not on-MPH, was associated with higher symptom ratings (Pearson r = 0.56, p = 0.04).

CONCLUSION

Our findings corroborate previous reports from mainly adult samples that MPH changes striatal DAT BP availability and suggest higher off-MPH DAT BP, likely reflecting low baseline DA levels, as a marker of symptom severity.

Systematic Review: Attention-Deficit/Hyperactivity Disorder and Instrumental Learning

OBJECTIVE

Although instrumental learning deficits are, among other deficits, assumed to contribute to attention-deficit/hyperactivity disorder (ADHD), no comprehensive systematic review of instrumental learning deficits in ADHD exists. This review examines differences between ADHD and typically developing (TD) children in basic instrumental learning and the effects of reinforcement form, magnitude, schedule, and complexity, as well as effects of medication, on instrumental learning in children with ADHD.

METHOD

A systematic search of PubMed, PsyINFO, CINAHL, EMBASE+EMBASE CLASSIC, ERIC, and Web of Science was conducted for articles up to March 16, 2020. Experimental studies comparing instrumental learning between groups (ADHD versus TD) or a manipulation of reinforcement/medication within an ADHD sample were included. Quality of studies was assessed with an adapted version of the Hombrados and Waddington criteria to assess risk of bias in (quasi-) experimental studies.

RESULTS

A total of 19 studies from among 3,384 non-duplicate screened articles were included. No difference in basic instrumental learning was found between children with ADHD and TD children, nor effects of form or magnitude of reinforcement. Results regarding reinforcement schedule and reversal learning were mixed, but children with ADHD seemed to show deficits in conditional discrimination learning compared to TD children. Methylphenidate improved instrumental learning in children with ADHD. Quality assessment showed poor quality of studies with respect to sample sizes and outcome and missing data reporting.

CONCLUSION

The review identified very few and highly heterogenous studies, with inconsistent findings. No clear deficit was found in instrumental learning under laboratory conditions. Children with ADHD do show deficits in complex forms of learning, that is, conditional discrimination learning. Clearly more research is needed, using more similar task designs and manipulations.

Probabilistic Learning in Children With Attention-Deficit/Hyperactivity Disorder

Objective: The current study examined instrumental learning in ADHD. Method: A total of 58 children with ADHD and 58 typically developing (TD) children performed a probabilistic learning task using three reward probability conditions (100%, 85%, 70% reward). After a learning phase, application of what was learned was assessed in a test phase. Results: Results showed that children with ADHD performed less accurate compared with TD children during the learning phase, particularly in the 100% and 85% reward probability conditions. These findings were accompanied by a blunted learning rate in the first few task trials. Furthermore, children with ADHD showed poorer application of what was learned. Conclusion: To conclude, children with ADHD show initial learning problems, but increased performance in a similar manner as TD children independent of the probability of reward, although they fail to apply their knowledge. Findings are of clinical relevance as the application of knowledge is important to successfully adapt to daily challenges in life.

A Systematic Review of the Positive Valence System in Autism Spectrum Disorder

This review synthesized current literature of behavioral and cognitive studies targeting reward processing in autism spectrum disorder (ASD). The National Institute of Mental Health's Research Domain Criteria (RDoC) Positive Valence System (PVS) domain was used as an overarching framework. The objectives were to determine which component operations of reward processing may be atypical in ASD and consequently postulate a heuristic model of reward processing in ASD that could be evaluated with future research. 34 studies were identified from the Embase, PubMed, PsycINFO, and Web of Science databases and included in the review using guidelines from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (also known as PRISMA guidelines). The extant literature suggested potential relationships between social symptoms of ASD and PVS sub-constructs of reward anticipation, probabilistic and reinforcement learning, reward prediction error, reward (probability), delay, and effort as well as between restricted and repetitive behaviors and interests (RRBIs) and PVS-sub constructs of initial response to reward, reward anticipation, reward (probability), delay, and effort. However, these findings are limited by a sparse and mixed literature for some sub-constructs. We put forward a developmentally informed heuristic model that posits how these component reward processes may be implicated in early ASD behaviors as well as later emerging and more intransigent symptoms. Future research is needed to comprehensively evaluate the proposed model.

Fathers' and Mothers' Early Mind-Mindedness Predicts Social Competence and Behavior Problems in Childhood

Parental mind-mindedness, the parent's propensity to treat the child as an intentional agent, has repeatedly shown to promote children's development of social understanding and secure attachment. Less is known about whether the impact of maternal and paternal mind-mindedness extends to children's social and behavior problems. We investigated the combined effect of mothers' and fathers' (N = 104) mind-mindedness at 4, 12, and 30 months on children's social competence and externalizing and internalizing behavior problems at 4.5 years. Besides, we examined the stability, continuity, parental concordance, and inter-parental differences in the use of mind-related comments. Appropriate mind-mindedness (i.e., correct interpretations of the child's mental states) and nonattuned mind-mindedness (i.e., misinterpretations of the child's mental states) were observed during parent-child free-play interactions. Social competence, internalizing and externalizing behavior problems were assessed using both parents' reports. Hierarchical multiple regression analyses showed that, at 12 months, infrequent use of appropriate mind-related comments of both parents predicted children's externalizing problems, while their frequent use of nonattuned comments predicted children's low social competence. Furthermore, mothers' frequent use of nonattuned comments at 12 and 30 months and fathers' nonattuned comments at 30 months predicted children's externalizing behavior. The findings suggest that both parents' low use of mind-related comments, and frequent misinterpretations of their child's mind, may act as risk factors for later social and behavior problems of their child.

Methylphenidate increases willingness to perform effort in adults with ADHD

BACKGROUND

A reduced willingness to perform effort based on the magnitude and probability of potential rewards has been associated with diminished dopamine function and may be relevant to attention-deficit/hyperactivity disorder (ADHD). Here, we investigated the influence of ADHD status and methylphenidate on effort-based decisions. We hypothesized that ADHD participants would make fewer high-effort selections than non-ADHD subjects, and that methylphenidate would increase the number of high-effort selections. Furthermore, we hypothesized there would be associations among ADHD severity and methylphenidate-related changes in effort-based and attentional performance across all participants.

METHODS AND PARTICIPANTS

ADHD (n = 23) and non-ADHD (n = 23) adults completed the Effort Expenditure for Rewards Task in which participants select between low-effort and high-effort options to receive monetary rewards at varying levels of reward magnitude and probability. A test of attentional performance was also completed.

RESULTS

Overall, participants made more high-effort selections as potential reward magnitude and probability increased. ADHD participants did not make fewer high-effort selections than non-ADHD participants, but ADHD participants showed greater methylphenidate-related increases in high-effort selections. ADHD participants had worse attentional performance than non-ADHD participants. ADHD severity was associated with methylphenidate-related changes in high-effort selections, but not changes in attentional performance.

CONCLUSIONS

These results indicate that methylphenidate increases the willingness to perform effort in individuals with ADHD, possibly due to disorder-related motivational deficits. This provides support for theories of insufficient effort allocation among individuals with ADHD.

TRIAL REGISTRATION

Clinicaltrials.gov Identifier, NCT02630017.

Feedback Timing Modulates Probabilistic Learning in Adults with ADHD

Attention deficit hyperactivity disorder (ADHD) has been associated primarily with executive function deficits. Emerging findings suggest, however, that procedural learning may be compromised as well. To this effect, we recently showed that feedback-based procedural learning is selectively impaired in ADHD, results that coincide with dopaminergic alterations associated with ADHD. Key questions, however, remain unresolved, among which are the learning conditions that may improve procedural learning in ADHD. Here we examined feedback-based probabilistic learning during conditions that engage procedural and declarative learning systems to different degrees, depending on feedback timing. ADHD and control participants carried out a probabilistic learning task in which they were required to learn to associate between cues and outcomes, where outcomes were presented either immediately or with a short/long delays. Whereas performance in probabilistic learning in ADHD participants was comparable to controls in delayed feedback conditions, during both learning and test phases, their performance diminished when feedback was immediate. Furthermore, ADHD symptom severity was negatively correlated with the ability to learn from immediate feedback. These results suggest that feedback-based probabilistic learning can be improved in ADHD, provided appropriate conditions. By shifting the load from midbrain/striatal systems to declarative memory mechanisms, behavioral performance in ADHD populations can be remediated.

Behavioral adjustment to asymmetric reward availability among children with and without ADHD: effects of past and current reinforcement contingencies

Altered reinforcement sensitivity is hypothesized to underlie symptoms of attention deficit hyperactivity disorder (ADHD). Here we evaluate the behavioral sensitivity of Brazilian children with and without ADHD to a change in reward availability. Forty typically developing children and 32 diagnosed with DSM-IV ADHD completed a signal-detection task in which correct discriminations between two stimuli were associated with different frequencies of reinforcement. The response alternative associated with the higher rate of reinforcement switched, without warning, after 30 rewards were delivered. The task continued until another 30 rewards were delivered. Both groups of children developed a response bias toward the initially more frequently reinforced alternative. This effect was larger in the control group. The response allocation of the two groups changed following the shift in reward availability. Over time the ADHD group developed a significant response bias toward the now more frequently reinforced alternative. In contrast, the bias of the control group stayed near zero after an initial decline following the contingency change. The overall shift in bias was similar for the two groups. The behavior of both groups of children was sensitive to the asymmetric reward distribution and to the change in reward availability. Subtle group differences in response patterns emerged, possibly reflecting differences in the time frame of reward effects and sensitivity to reward exposure.

Delay discounting of monetary gains and losses in adolescents with ADHD: Contribution of delay aversion to choice

Adolescents with attention-deficit/hyperactivity disorder (ADHD) are known to have stronger preferences for smaller immediate rewards over larger delayed rewards in delay discounting tasks than their peers, which has been argued to reflect delay aversion. Here, participants performed a delay discounting task with gains and losses. In this latter condition, participants were asked whether they were willing to wait in order to lose less money. Following the core assumption of the delay aversion model that individuals with ADHD have a general aversion to delay, one would predict adolescents with ADHD to avoid waiting in both conditions. Adolescents (12-17 years) with ADHD (n = 29) and controls (n = 28) made choices between smaller immediate and larger delayed gains, and between larger immediate and smaller delayed losses. All delays (5-25 s) and gains/losses (2-10 cents) were experienced. In addition to an area under the curve approach, a mixed-model analysis was conducted to disentangle the contributions of delay duration and immediate gain/delayed loss amount to choice. The ADHD group chose the immediate option more often than controls in the gain condition, but not in the loss condition. The contribution of delay duration to immediate choices was stronger for the ADHD group than the control group in the gain condition only. In addition, the ADHD group scored higher on self-reported delay aversion, and delay aversion was associated with delay sensitivity in the gain condition, but not in the loss condition. In sum, we found no clear evidence for a general aversion to delay in adolescents with ADHD.

Reward Processing Deficits During a Spatial Attention Task in Patients With ADHD: An fMRI Study

OBJECTIVE

In this study, we aimed to explore how cues signaling rewards and feedbacks about rewards are processed in ADHD.

METHOD

Inside the scanner, 16 healthy children and 19 children with ADHD completed a spatial attention paradigm where cues informed about the availability of reward and feedbacks were provided about the earned reward.

RESULTS

In ventral anterior thalamus (VA), the controls exhibited greater activation in response to reward-predicting cues, as compared with no-reward cues, whereby in the ADHD group, the reverse pattern was observed (nonreward > reward). For feedbacks; absence of rewards produced greater activation than presence in the left caudate and frontal eye field for the control group, whereas for the ADHD group, the reverse pattern was again observed (reward > nonreward).

DISCUSSION

The present findings indicate that ADHD is associated with difficulty integrating reward contingency information with the orienting and regulatory phases of attention.

Development and validation of the Canine Reward Responsiveness Scale -Examining individual differences in reward responsiveness of the domestic dog

Although there is ample data indicating that reward processing plays an important role in human psychopathologies and pharmaco- and psychotherapy treatment response, the corresponding animal-model research needs to be extended to models whose motivational and social dispositions are better generalizable than those of the traditional models. Accordingly, our aim was to develop and assess the reliability and validity of an owner-report rating scale of reward responsiveness in domestic dogs (N = 2149) and then to examine individual differences in reward responsiveness. Responsiveness was categorisable by reward type (ball/toy and food) and exhibited individual variability manifesting in age- and breed-related differences. Rating scale scores were associated with behavioural observation of reward processing, indicating evidence of convergent validity. Ball/toy and food reward responsiveness were associated with owner-rated hyperactivity-impulsivity‚ inattention and with differences in training, indicating evidence of concurrent validity. Extreme (vs. average) reward responsiveness was also predicted by dogs’ hyperactivity-impulsivity and inattention‚ and extreme responsiveness was associated with increased likelihood of physical health and/or social problems. These findings are informative with regard to the dog as an animal model for various human behavioural and cognitive functions‚ and also for the dog in its own right as they are relevant to training and welfare.

Attention-Deficit/Hyperactivity Disorder: A Historical Neuropsychological Perspective

The behavior patterns of hyperactivity, impulsivity and inattention that would ultimately become recognized as Attention-Deficit Hyperactivity Disorder (ADHD) have been described for centuries. Nevertheless, in the past 35 years, advances in diagnostic methods, identification of biomarkers, and treatments have advanced at an exponential rate. ADHD is now recognized as the most common behavioral disorder of childhood, with risks extending well into adulthood for both males and females, leading to its identification as a significant public health issue. This historical neuropsychological review of ADHD emphasizes scientific highlights in the past 35 years related to ADHD, including the evolution of the diagnosis (from Hyperkinetic Reaction of Childhood to ADHD), influential theories (executive functions, cognitive-energetic, delay aversion), landmark treatment studies (Multimodal Treatment of ADHD [MTA] and Preschool ADHD Treatment Study [PATS]), and advances in brain mapping techniques (anatomic, functional, and resting state magnetic resonance imaging, diffusion tensor imaging). The review concludes by highlighting the challenges of studying and treating a heterogeneous neurodevelopmental disorder like ADHD, with emphasis on associated disorders and conditions (learning disabilities, sluggish cognitive tempo), special populations (girls, preschoolers, adults), and recommendations for scientific inquiry in the next 35 years. Neuropsychologists are well positioned to address the clinical and research challenges of the next generation of studies, especially involving advances in understanding the sexual dimor.phism, full developmental course, and dynamic risks associated with ADHD. (JINS, 2017, 23, 916-929).

Neural substrates of trait impulsivity, anhedonia, and irritability: Mechanisms of heterotypic comorbidity between externalizing disorders and unipolar depression

Trait impulsivity, which is often defined as a strong preference for immediate over delayed rewards and results in behaviors that are socially inappropriate, maladaptive, and short-sighted, is a predisposing vulnerability to all externalizing spectrum disorders. In contrast, anhedonia is characterized by chronically low motivation and reduced capacity to experience pleasure, and is common to depressive disorders. Although externalizing and depressive disorders have virtually nonoverlapping diagnostic criteria in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders, heterotypic comorbidity between them is common. Here, we review common neural substrates of trait impulsivity, anhedonia, and irritability, which include both low tonic mesolimbic dopamine activity and low phasic mesolimbic dopamine responding to incentives during reward anticipation and associative learning. We also consider how other neural networks, including bottom-up emotion generation systems and top-down emotion regulation systems, interact with mesolimbic dysfunction to result in alternative manifestations of psychiatric illness. Finally, we present a model that emphasizes a translational, transdiagnostic approach to understanding externalizing/depression comorbidity. This model should refine ways in which internalizing and externalizing disorders are studied, classified, and treated.

Low Cognitive Impulsivity Is Associated with Better Gain and Loss Learning in a Probabilistic Decision-Making Task

Many advances have been made over the last decades in describing, on the one hand, the link between reward-based learning and decision-making, and on the other hand, the link between impulsivity and decision-making. However, the association between reward-based learning and impulsivity remains poorly understood. In this study, we evaluated the association between individual differences in loss-minimizing and gain-maximizing behavior in a learning-based probabilistic decision-making task and individual differences in cognitive impulsivity. We found that low cognitive impulsivity was associated both with a better performance minimizing losses and maximizing gains during the task. These associations remained significant after controlling for mathematical skills and gender as potential confounders. We discuss potential mechanisms through which cognitive impulsivity might interact with reward-based learning and decision-making.

Response time variability under slow and fast-incentive conditions in children with ASD, ADHD and ASD+ADHD

BACKGROUND

Attention deficit hyperactivity disorder (ADHD) and autism spectrum disorder (ASD) show significant behavioural and genetic overlap. Both ADHD and ASD are characterised by poor performance on a range of cognitive tasks. In particular, increased response time variability (RTV) is a promising indicator of risk for both ADHD and ASD. However, it is not clear whether different indices of RTV and changes to RTV according to task conditions are able to discriminate between the two disorders.

METHODS

Children with ASD (n = 19), ADHD (n = 18), ASD + ADHD (n = 29) and typically developing controls (TDC; n = 26) performed a four-choice RT task with slow-baseline and fast-incentive conditions. Performance was characterised by mean RT (MRT), standard deviation of RT (SD-RT), coefficient of variation (CV) and ex-Gaussian distribution measures of Mu, Sigma and Tau.

RESULTS

In the slow-baseline condition, categorical diagnoses and trait measures converged to indicate that children with ADHD-only and ASD + ADHD demonstrated increased MRT, SD-RT, CV and Tau compared to TDC and ASD-only. Importantly, greater improvement in MRT, SD-RT and Tau was demonstrated in ADHD and ASD + ADHD from slow-baseline to fast-incentive conditions compared to TDC and ASD-only.

CONCLUSIONS

Slower and more variable RTs are markers of ADHD compared to ASD and typically developing controls during slow and less rewarding conditions. Energetic factors and rewards improve task performance to a greater extent in children with ADHD compared to children with ASD. These findings suggest that RTV can be distinguished in ASD, ADHD and ASD + ADHD based on the indices of variability used and the conditions in which they are elicited. Further work identifying neural processes underlying increased RTV is warranted, in order to elucidate disorder-specific and disorder-convergent aetiological pathways.

How can Continuous Performance Test help to assess inattention when mood and ADHD symptoms coexist?

Depression and attention-deficit/hyperactivity disorder (ADHD) are prevalent, and often comorbid, disorders, with varying severity levels among patients. Inattention is a symptom present in both disorders, which often makes their differential diagnosis difficult in clinical practice (depression only versus comorbidity). This study aimed to investigate the influence of depressive symptoms on attention performance using one of the most common tasks in clinical practice, the continuous performance test (CPT). Ninety-three college students (60 men, 33 women) with a mean age of 24 years old were investigated with self-reports and semi-structured interviews for ADHD; the Beck Depression Inventory (BDI) was used for depression ratings. Attention measures were derived from the CPT. There was no correlation between depression and ADHD symptoms; in addition, depression was not correlated with any of the CPT scores; ADHD symptomatology was the only predictor of changes in those CPT variables (commission and omission errors and d prime). ADHD-associated impairment on the CPT was not augmented by the presence of depressive symptoms, making neuropsychological results on this test helpful for the differential diagnosis. When attention deficits are observed in individuals with mild or moderate depression, they are most likely not attributed to depression.

Modelling ADHD: A review of ADHD theories through their predictions for computational models of decision-making and reinforcement learning

Attention deficit hyperactivity disorder (ADHD) is characterized by altered decision-making (DM) and reinforcement learning (RL), for which competing theories propose alternative explanations. Computational modelling contributes to understanding DM and RL by integrating behavioural and neurobiological findings, and could elucidate pathogenic mechanisms behind ADHD. This review of neurobiological theories of ADHD describes predictions for the effect of ADHD on DM and RL as described by the drift-diffusion model of DM (DDM) and a basic RL model. Empirical studies employing these models are also reviewed. While theories often agree on how ADHD should be reflected in model parameters, each theory implies a unique combination of predictions. Empirical studies agree with the theories' assumptions of a lowered DDM drift rate in ADHD, while findings are less conclusive for boundary separation. The few studies employing RL models support a lower choice sensitivity in ADHD, but not an altered learning rate. The discussion outlines research areas for further theoretical refinement in the ADHD field.

Electrophysiological correlates of reinforcement learning in young people with Tourette syndrome with and without co-occurring ADHD symptoms

Altered reinforcement learning is implicated in the causes of Tourette syndrome (TS) and attention-deficit/hyperactivity disorder (ADHD). TS and ADHD frequently co-occur but how this affects reinforcement learning has not been investigated. We examined the ability of young people with TS (n=18), TS+ADHD (N=17), ADHD (n=13) and typically developing controls (n=20) to learn and reverse stimulus-response (S-R) associations based on positive and negative reinforcement feedback. We used a 2 (TS-yes, TS-no)×2 (ADHD-yes, ADHD-no) factorial design to assess the effects of TS, ADHD, and their interaction on behavioural (accuracy, RT) and event-related potential (stimulus-locked P3, feedback-locked P2, feedback-related negativity, FRN) indices of learning and reversing the S-R associations. TS was associated with intact learning and reversal performance and largely typical ERP amplitudes. ADHD was associated with lower accuracy during S-R learning and impaired reversal learning (significantly reduced accuracy and a trend for smaller P3 amplitude). The results indicate that co-occurring ADHD symptoms impair reversal learning in TS+ADHD. The implications of these findings for behavioural tic therapies are discussed.

WITHDRAWN: Electrophysiological correlates of reinforcement learning in young people with Tourette syndrome with and without co-occurring ADHD symptoms

The Publisher regrets that this article is an accidental duplication of an article that has already been published, http://dx.doi.org/10.1016/j.ijdevneu.2016.04.006. The duplicate article has therefore been withdrawn. The full Elsevier Policy on Article Withdrawal can be found at http://www.elsevier.com/locate/withdrawalpolicy.

Instrumental learning in ADHD in a context of reward: intact learning curves and performance improvement with methylphenidate

We tested the hypothesis that instrumental learning is impaired in children with attention deficit/hyperactivity disorder (ADHD) and tested whether this deficiency can be ameliorated by the use of three different doses of methylphenidate (MPH): a low, a medium, and a high dose. Fifty children (23 with ADHD, 27 typically developing) performed an instrumental learning task in which they had to map four stimuli to two responses using feedback that differed in information value and that was coupled to monetary gain (<euro>0.2). Feedback was either contingent, thus fully informative on performance, or probabilistic, and informative in only 88 % of the trials. Dependent variables were response accuracy and latency. Contrary to predictions, children with ADHD and typically developing controls did not differ in their ability to acquire the stimulus-response mappings, in either condition. This indicates that with a simple instrumental learning task, children with ADHD are able to learn from rewarded feedback, as controls do, suggesting that children with ADHD are capable of learning both at home and in class when the environment is adequate. Possibly, learning problems are more evident in children with comorbid ODD, as exploratory analyses indicated that accuracy scores were significantly reduced when children with ADHD exhibited high levels of parent-rated ODD symptoms. Medication analysis showed that children with ADHD completed the task more accurately and faster when taking MPH compared to placebo, particularly when the dose was high.

Trait-related decision making impairment in obsessive-compulsive disorder: evidence from decision making under ambiguity but not decision making under risk

This study aimed to investigate whether deficits in decision making were potential endophenotype markers for OCD considering different phases of the disease. Fifty-seven non-medicated OCD patients (nmOCD), 77 medicated OCD patients (mOCD), 48 remitted patients with OCD (rOCD) and 115 healthy controls were assessed with the Iowa Gambling Task (IGT), which measured decision making under ambiguity, and the Game of Dice Task (GDT), which measured decision making under risk. While the three patients groups showed impaired performance on the IGT compared with healthy controls, all patients showed intact performance on the GDT. Furthermore, the rOCD patients showed a preference for deck B, indicating that they showed more sensitivity to the frequency of loss than to the magnitude of loss, whereas the mOCD patients showed a preference for deck A, indicating that they had more sensitivity to the magnitude of loss than to the frequency of loss. These data suggested that OCD patients had trait-related impairments in decision making under ambiguity but not under risk, and that dissociation of decision making under ambiguity and under risk is an appropriate potential neurocognitive endophenotype for OCD. The subtle but meaningful differences in decision making performance between the OCD groups require further study.

Reinforcement enhances vigilance among children with ADHD: comparisons to typically developing children and to the effects of methylphenidate

Sustained attention and reinforcement are posited as causal mechanisms in Attention-Deficit/Hyperactivity Disorder (ADHD), but their interaction has received little empirical study. In two studies, we examined the impact of performance-based reinforcement on sustained attention over time, or vigilance, among 9- to 12-year-old children. Study 1 demonstrated the expected vigilance deficit among children with ADHD (n = 25; 12% female) compared to typically developing (TD) controls (n = 33; 22% female) on a standard continuous performance task (CPT). During a subsequent visit, reinforcement improved attention more among children with ADHD than controls. Study 2 examined the separate and combined effects of reinforcement and acute methylphenidate (MPH) on CPT performance in children with ADHD (n = 19; 21% female). Both reinforcement and MPH enhanced overall target detection and attenuated the vigilance decrement that occurred in no-reinforcement, placebo condition. Cross-study comparisons suggested that the combination of MPH and reinforcement eliminated the vigilance deficit in children with ADHD, normalizing sustained attention. This work highlights the clinically and theoretically interesting intersection of reinforcement and sustained attention.

Probabilistic reward learning in adults with Attention Deficit Hyperactivity Disorder--an electrophysiological study

Attention Deficit Hyperactivity Disorder (ADHD) is hypothesized to be characterized by altered reinforcement sensitivity. The main aim of the present study was to assess alterations in the electrophysiological correlates of monetary reward processing in adult patients with ADHD of the combined subtype. Fourteen adults with ADHD of the combined subtype and 14 healthy control participants performed an active and an observational probabilistic reward-based learning task while an electroencephalogramm (EEG) was recorded. Regardless of feedback valence, there was a general feedback-related negativity (FRN) enhancement in combination with reduced learning performance during both active and observational reward learning in patients with ADHD relative to healthy controls. Other feedback-locked potentials such as the P200 and P300 and response-locked potentials were unaltered in the patients. There were no significant correlations between learning performance, FRN amplitudes and clinical symptoms, neither in the overall group involving all participants, nor in patients or controls considered separately. This pattern of findings might reflect generally impaired reward prediction in adults with ADHD of the combined subtype. We demonstrated for the first time that patients with ADHD of the combined subtype show not only deficient active reward learning but are also impaired when learning by observing other people׳s outcomes.

ADHD, altered dopamine neurotransmission, and disrupted reinforcement processes: implications for smoking and nicotine dependence

Attention deficit hyperactivity disorder (ADHD) is a common and impairing disorder affecting millions of children, adolescents, and adults. Individuals with ADHD smoke cigarettes at rates significantly higher than their non-diagnosed peers and the disorder also confers risk for a number of related adverse smoking outcomes including earlier age of initiation, faster progression to regular use, heavier smoking/greater dependence, and more difficulty quitting. Progress in our understanding of dopamine neurotransmission and basic behavioral reinforcement processes in ADHD may help increase our understanding of the ADHD-smoking comorbidity. This review will examine how these areas have been studied and how further work may aid in the development of better prevention and treatment for smoking in those with ADHD.

Goal-directed action control in children with autism spectrum disorders

Repetitive behavior is a key characteristic of autism spectrum disorders. Our aim was to investigate the hypothesis that this abnormal behavioral repetition results from a tendency to over-rely on habits at the expense of flexible, goal-directed action. Twenty-four children with autism spectrum disorders and 24 age- and gender-matched controls (8-12 years) initially learned to give specific responses to different pictorial stimuli in order to gain valuable outcomes. Subsequently, in the "slips-of-action" test, some of these outcomes were no longer valuable. Children needed to refrain from responding when stimuli were shown that signaled the availability of those outcomes while continuing to respond for the still-valuable outcomes. Reliance on habits should lead to "slips of action" toward no longer valuable outcomes. Therefore, the children's ability to respond selectively for still-valuable outcomes provides a measure of relative habitual versus goal-directed control. Two additional tasks were included to control for general task characteristics (i.e. working memory and inhibition). Children with autism spectrum disorders learned equally well as controls and were not impaired at flexibly adjusting their behavior to devaluation of the outcomes or stimuli. We found no evidence for a disruption in the balance between goal-directed and habitual behavioral control in children with autism spectrum disorders.

A behavioral neuroenergetics theory of ADHD

Energetic insufficiency in neurons due to inadequate lactate supply is implicated in several neuropathologies, including attention-deficit/hyperactivity disorder (ADHD). By formalizing the mechanism and implications of such constraints on function, the behavioral Neuroenergetics Theory (NeT) predicts the results of many neuropsychological tasks involving individuals with ADHD and kindred dysfunctions, and entails many novel predictions. The associated diffusion model predicts that response times will follow a mixture of Wald distributions from the attentive state, and ex-Wald distributions after attentional lapses. It is inferred from the model that ADHD participants can bring only 75-85% of the neurocognitive energy to bear on tasks, and allocate only about 85% of the cognitive resources of comparison groups. Parameters derived from the model in specific tasks predict performance in other tasks, and in clinical conditions often associated with ADHD. The primary action of therapeutic stimulants is to increase norepinephrine in active regions of the brain. This activates glial adrenoceptors, increasing the release of lactate from astrocytes to fuel depleted neurons. The theory is aligned with other approaches and integrated with more general theories of ADHD. Therapeutic implications are explored.

ADHD and autism: differential diagnosis or overlapping traits? A selective review

According to DSM-IV TR and ICD-10, a diagnosis of autism or Asperger Syndrome precludes a diagnosis of attention-deficit/hyperactivity disorder (ADHD). However, despite the different conceptualization, population-based twin studies reported symptom overlap, and a recent epidemiologically based study reported a high rate of ADHD in autism and autism spectrum disorders (ASD). In the planned revision of the DSM-IV TR, dsm5 (www.dsm5.org), the diagnoses of autistic disorder and ADHD will not be mutually exclusive any longer. This provides the basis of more differentiated studies on overlap and distinction between both disorders. This review presents data on comorbidity rates and symptom overlap and discusses common and disorder-specific risk factors, including recent proteomic studies. Neuropsychological findings in the areas of attention, reward processing, and social cognition are then compared between both disorders, as these cognitive abilities show overlapping as well as specific impairment for one of both disorders. In addition, selective brain imaging findings are reported. Therapeutic options are summarized, and new approaches are discussed. The review concludes with a prospectus on open questions for research and clinical practice.

Neuroeconomics of attention-deficit/hyperactivity disorder: differential influences of medial, dorsal, and ventral prefrontal brain networks on suboptimal decision making?

Psychiatric neuroeconomics offers an alternative approach to understanding mental disorders by studying the way disorder-related neurobiological alterations constrain economic agency, as revealed through decisions about choices between future goods. In this article, we apply this perspective to understand suboptimal decision making in attention-deficit/hyperactivity disorder (ADHD) by integrating recent advances in the neuroscience of decision making and studies of the pathophysiology of ADHD. We identify three brain networks as candidates for further study and develop specific hypotheses about how these could be implicated in ADHD. First, we postulate that altered patterns of connectivity within a network linking medial prefrontal cortex and posterior cingulate cortex (i.e., the default mode network) disrupts ordering of utilities, prospection about desired future states, setting of future goals, and implementation of aims. Second, we hypothesize that deficits in dorsal frontostriatal networks, including the dorsolateral prefrontal cortex and dorsal striatum, produce executive dysfunction-mediated impairments in the ability to compare outcome options and make choices. Third, we propose that dopaminergic dysregulation in a ventral frontostriatal network encompassing the orbitofrontal cortex, ventral striatum, and amygdala disrupts processing of cues of future utility, evaluation of experienced outcomes (feedback), and learning of associations between cues and outcomes. Finally, we extend this perspective to consider three contemporary themes in ADHD research.

Reinforcement, dopamine and rodent models in drug development for ADHD

Attention deficit hyperactivity disorder (ADHD) presents special challenges for drug development. Current treatment with psychostimulants and nonstimulants is effective, but their mechanism of action beyond the cellular level is incompletely understood. We review evidence suggesting that altered reinforcement mechanisms are a fundamental characteristic of ADHD. We show that a deficit in the transfer of dopamine signals from established positive reinforcers to cues that predict such reinforcers may underlie these altered reinforcement mechanisms, and in turn explain key symptoms of ADHD. We argue that the neural substrates controlling the excitation and inhibition of dopamine neurons during the transfer process are a promising target for future drug development. There is a need to develop animal models and behavioral paradigms that can be used to experimentally investigate these mechanisms and their effects on sensitivity to reinforcement. More specific and selective targeting of drug development may be possible through this approach.

Can motivation normalize working memory and task persistence in children with attention-deficit/hyperactivity disorder? The effects of money and computer-gaming

Visual-spatial Working Memory (WM) is the most impaired executive function in children with Attention-Deficit/Hyperactivity Disorder (ADHD). Some suggest that deficits in executive functioning are caused by motivational deficits. However, there are no studies that investigate the effects of motivation on the visual-spatial WM of children with- and without ADHD. Studies examining this in executive functions other than WM, show inconsistent results. These inconsistencies may be related to differences in the reinforcement used. The effects of different reinforcers on WM performance were investigated in 30 children with ADHD and 31 non-ADHD controls. A visual-spatial WM task was administered in four reinforcement conditions: Feedback-only, 1 euro, 10 euros, and a computer-game version of the task. In the Feedback-only condition, children with ADHD performed worse on the WM measure than controls. Although incentives significantly improved the WM performance of children with ADHD, even the strongest incentives (10 euros and Gaming) were unable to normalize their performance. Feedback-only provided sufficient reinforcement for controls to reach optimal performance, while children with ADHD required extra reinforcement. Only children with ADHD showed a decrease in performance over time. Importantly, the strongest incentives (10 euros and Gaming) normalized persistence of performance in these children, whereas 1 euro had no such effect. Both executive and motivational deficits give rise to visual-spatial WM deficits in ADHD. Problems with task-persistence in ADHD result from motivational deficits. In ADHD-reinforcement studies and clinical practice (e.g., assessment), reinforcement intensity can be a confounding factor and should be taken into account. Gaming can be a cost-effective way to maximize performance in ADHD.

Adults with autism spectrum disorders exhibit decreased sensitivity to reward parameters when making effort-based decisions

BACKGROUND

Efficient effort expenditure to obtain rewards is critical for optimal goal-directed behavior and learning. Clinical observation suggests that individuals with autism spectrum disorders (ASD) may show dysregulated reward-based effort expenditure, but no behavioral study to date has assessed effort-based decision-making in ASD.

METHODS

The current study compared a group of adults with ASD to a group of typically developing adults on the Effort Expenditure for Rewards Task (EEfRT), a behavioral measure of effort-based decision-making. In this task, participants were provided with the probability of receiving a monetary reward on a particular trial and asked to choose between either an "easy task" (less motoric effort) for a small, stable reward or a "hard task" (greater motoric effort) for a variable but consistently larger reward.

RESULTS

Participants with ASD chose the hard task more frequently than did the control group, yet were less influenced by differences in reward value and probability than the control group. Additionally, effort-based decision-making was related to repetitive behavior symptoms across both groups.

CONCLUSIONS

These results suggest that individuals with ASD may be more willing to expend effort to obtain a monetary reward regardless of the reward contingencies. More broadly, results suggest that behavioral choices may be less influenced by information about reward contingencies in individuals with ASD. This atypical pattern of effort-based decision-making may be relevant for understanding the heightened reward motivation for circumscribed interests in ASD.

Reward processing in male adults with childhood ADHD--a comparison between drug-naïve and methylphenidate-treated subjects

RATIONALE

Dysfunctional reward processing has been proposed as a main deficit in attention-deficit/hyperactivity disorder (ADHD), which could be modulated by treatment with methylphenidate (MPH).

OBJECTIVES

We examined differences in reward processing in adulthood (independent of actual ADHD) depending on MPH treatment during childhood.

METHODS

Eleven males with childhood ADHD treated with MPH, 12 drug-naïve males with childhood ADHD, and 12 controls matched by age, handedness, and smoking behavior were studied drug-free using functional magnetic resonance imaging. BOLD-responses were compared during a monetary incentive delay task using an ANOVA design focusing on the ventral striatum during anticipation and the orbitofrontal cortex during outcome.

RESULTS

Controls, drug-naïve, and treated subjects did not differ significantly in their activations in the ventral striatum and orbitofrontal cortex. Explorative analyses revealed decreased insula activation during outcome of loss avoidance in drug-naïve subjects in comparison to both groups, while treated subjects did not differ from controls. Insula activation correlated significantly positive with harm avoidance in the treated group. Furthermore, comparing subjects with actual ADHD symptoms, remitters and controls we observed decreased putamen activition in ADHD persisters.

CONCLUSIONS

Basal ganglia reward processing seemed to be unrelated to MPH pretreatment, but was related to remission. On the other hand, the revealed differences between treated and drug-naïve subjects with childhood ADHD, i.e., in the insula, give evidence for more pronounced abnormal activation in reward-associated brain regions in untreated subjects with childhood ADHD and underpin the need of prospective studies on long-term effects of psychostimulant treatment.

Identifying the neurobiology of altered reinforcement sensitivity in ADHD: a review and research agenda

ADHD is associated with altered reinforcement sensitivity, despite a number of inconsistent findings. This review focuses on the overlap and differences between seven neurobiologically valid models and lists 15 predictions assessing reinforcement sensitivity in ADHD. When comparing the models it becomes clear that there are great differences in the level of explanation. For example, some models try to explain a single core deficit in terms lower-level reinforcement systems, such as the dopamine transfer to reward back in time. Other models explain multiple deficits, by describing higher-level systems, such as impaired bottom-up prefrontal activation. When reviewing the available experimental evidence in support of the predictions, most experimental studies have been focusing on behavioral changes in the face of reward and response cost over no-reward, and on delay discounting. There is currently a lack in studies that focus on explaining underlying cognitive or neural mechanisms of altered reinforcement sensitivity in ADHD. Additionally, there is a lack in studies that try to understand what subgroup of children with ADHD shows alterations in reinforcement sensitivity. The scarcity in studies testing the neurobiological predictions is explained partly by a lack in knowledge how to test some of these predictions in humans. Nevertheless, we believe that these predictions can serve as a useful guide to the systematic evaluation of altered reinforcement sensitivity in ADHD.