Focus on the positive: computational simulations implicate asymmetrical reward prediction error signals in childhood attention-deficit/hyperactivity disorder

Electrophysiological correlates of perceptual prediction error are attenuated in dyslexia

A perceptual adaptation deficit often accompanies reading difficulty in dyslexia, manifesting in poor perceptual learning of consistent stimuli and reduced neurophysiological adaptation to stimulus repetition. However, it is not known how adaptation deficits relate to differences in feedforward or feedback processes in the brain. Here we used electroencephalography (EEG) to interrogate the feedforward and feedback contributions to neural adaptation as adults with and without dyslexia viewed pairs of faces and words in a paradigm that manipulated whether there was a high probability of stimulus repetition versus a high probability of stimulus change. We measured three neural dependent variables: expectation (the difference between prestimulus EEG power with and without the expectation of stimulus repetition), feedforward repetition (the difference between event-related potentials (ERPs) evoked by an expected change and an unexpected repetition), and feedback-mediated prediction error (the difference between ERPs evoked by an unexpected change and an expected repetition). Expectation significantly modulated prestimulus theta- and alpha-band EEG in both groups. Unexpected repetitions of words, but not faces, also led to significant feedforward repetition effects in the ERPs of both groups. However, neural prediction error when an unexpected change occurred instead of an expected repetition was significantly weaker in dyslexia than the control group for both faces and words. These results suggest that the neural and perceptual adaptation deficits observed in dyslexia reflect the failure to effectively integrate perceptual predictions with feedforward sensory processing. In addition to reducing perceptual efficiency, the attenuation of neural prediction error signals would also be deleterious to the wide range of perceptual and procedural learning abilities that are critical for developing accurate and fluent reading skills.

Quantifying mechanisms of cognition with an experiment and modeling ecosystem

Although there have been major strides toward uncovering the neurobehavioral mechanisms involved in cognitive functions like memory and decision making, methods for measuring behavior and accessing latent processes through computational means remain limited. To this end, we have created SUPREME (Sensing to Understanding and Prediction Realized via an Experiment and Modeling Ecosystem): a toolbox for comprehensive cognitive assessment, provided by a combination of construct-targeted tasks and corresponding computational models. SUPREME includes four tasks, each developed symbiotically with a mechanistic model, which together provide quantified assessments of perception, cognitive control, declarative memory, reward valuation, and frustrative nonreward. In this study, we provide validation analyses for each task using two sessions of data from a cohort of cognitively normal participants (N = 65). Measures of test-retest reliability (r: 0.58-0.75), stability of individual differences (ρ: 0.56-0.70), and internal consistency (α: 0.80-0.86) support the validity of our tasks. After fitting the models to data from individual subjects, we demonstrate each model's ability to capture observed patterns of behavioral results across task conditions. Our computational approaches allow us to decompose behavior into cognitively interpretable subprocesses, which we can compare both within and between participants. We discuss potential future applications of SUPREME, including clinical assessments, longitudinal tracking of cognitive functions, and insight into compensatory mechanisms.

Modelling ADHD: A review of ADHD theories through their predictions for computational models of decision-making and reinforcement learning

Attention deficit hyperactivity disorder (ADHD) is characterized by altered decision-making (DM) and reinforcement learning (RL), for which competing theories propose alternative explanations. Computational modelling contributes to understanding DM and RL by integrating behavioural and neurobiological findings, and could elucidate pathogenic mechanisms behind ADHD. This review of neurobiological theories of ADHD describes predictions for the effect of ADHD on DM and RL as described by the drift-diffusion model of DM (DDM) and a basic RL model. Empirical studies employing these models are also reviewed. While theories often agree on how ADHD should be reflected in model parameters, each theory implies a unique combination of predictions. Empirical studies agree with the theories' assumptions of a lowered DDM drift rate in ADHD, while findings are less conclusive for boundary separation. The few studies employing RL models support a lower choice sensitivity in ADHD, but not an altered learning rate. The discussion outlines research areas for further theoretical refinement in the ADHD field.

Adaptive effort investment in cognitive and physical tasks: a neurocomputational model

Despite its importance in everyday life, the computational nature of effort investment remains poorly understood. We propose an effort model obtained from optimality considerations, and a neurocomputational approximation to the optimal model. Both are couched in the framework of reinforcement learning. It is shown that choosing when or when not to exert effort can be adaptively learned, depending on rewards, costs, and task difficulty. In the neurocomputational model, the limbic loop comprising anterior cingulate cortex (ACC) and ventral striatum in the basal ganglia allocates effort to cortical stimulus-action pathways whenever this is valuable. We demonstrate that the model approximates optimality. Next, we consider two hallmark effects from the cognitive control literature, namely proportion congruency and sequential congruency effects. It is shown that the model exerts both proactive and reactive cognitive control. Then, we simulate two physical effort tasks. In line with empirical work, impairing the model's dopaminergic pathway leads to apathetic behavior. Thus, we conceptually unify the exertion of cognitive and physical effort, studied across a variety of literatures (e.g., motivation and cognitive control) and animal species.

Impaired reward processing by anterior cingulate cortex in children with attention deficit hyperactivity disorder

Decades of research have examined the neurocognitive mechanisms of cognitive control, but the motivational factors underlying task selection and performance remain to be elucidated. We recently proposed that anterior cingulate cortex (ACC) utilizes reward prediction error signals carried by the midbrain dopamine system to learn the value of tasks according to the principles of hierarchical reinforcement learning. According to this position, disruption of the ACC-dopamine interface can disrupt the selection and execution of extended, task-related behaviors. To investigate this issue, we recorded the event-related brain potential (ERP) from children with attention deficit hyperactivity disorder (ADHD), which is strongly associated with ACC-dopamine dysfunction, and from typically developing children while they navigated a simple "virtual T-maze" to find rewards. Depending on the condition, the feedback stimuli on each trial indicated that the children earned or failed to earn either money or points. We found that the reward positivity, an ERP component proposed to index the impact of dopamine-related reward signals on ACC, was significantly larger with money feedback than with points feedback for the children with ADHD, but not for the typically developing children. These results suggest that disruption of the ACC-dopamine interface may underlie the impairments in motivational control observed in childhood ADHD.

Motivational incentives and methylphenidate enhance electrophysiological correlates of error monitoring in children with attention deficit/hyperactivity disorder

BACKGROUND

Children with attention deficit hyperactivity disorder (ADHD) are characterised by developmentally inappropriate levels of hyperactivity, impulsivity and/or inattention and are particularly impaired when performing tasks that require a high level of cognitive control. Methylphenidate (MPH) and motivational incentives may help improve cognitive control by enhancing the ability to monitor response accuracy and regulate performance accordingly.

METHODS

Twenty-eight children with DSM-IV ADHD (combined type) aged 9-15 years and pairwise-matched typically developing children (CTRL) performed a go/no-go task in which the incentives attached to performance on no-go trials were manipulated. The ADHD group performed the task off and on their usual dose of MPH. CTRL children performed the task twice but were never medicated. EEG data were recorded simultaneously and two electrophysiological indices of error monitoring, the error-related negativity (ERN) and error positivity (Pe) were measured. Amplitudes of each ERP were compared between diagnostic groups (CTRL, ADHD), medication days (Off MPH, On MPH) and motivational conditions (baseline - low incentive, reward, response cost).

RESULTS

Error rates were lower in the reward and response cost conditions compared with baseline across diagnostic groups and medication days. ERN and Pe amplitudes were significantly reduced in ADHD compared with CTRL, and were significantly enhanced by MPH. Incentives significantly increased ERN and Pe amplitudes in the ADHD group but had no effect in CTRL. The effects of incentives did not interact with the effects of MPH on either ERP. Effect sizes were computed and revealed larger effects of MPH than incentives on ERN and Pe amplitudes.

CONCLUSIONS

The findings reveal independent effects of motivational incentives and MPH on two electrophysiological markers of error monitoring in children with ADHD, suggesting that each may be important tools for enhancing or restoring cognitive control in these children.

Deficient reinforcement learning in medial frontal cortex as a model of dopamine-related motivational deficits in ADHD

Attention Deficit/Hyperactivity Disorder (ADHD) is a pathophysiologically complex and heterogeneous condition with both cognitive and motivational components. We propose a novel computational hypothesis of motivational deficits in ADHD, drawing together recent evidence on the role of anterior cingulate cortex (ACC) and associated mesolimbic dopamine circuits in both reinforcement learning and ADHD. Based on findings of dopamine dysregulation and ACC involvement in ADHD we simulated a lesion in a previously validated computational model of ACC (Reward Value and Prediction Model, RVPM). We explored the effects of the lesion on the processing of reinforcement signals. We tested specific behavioral predictions about the profile of reinforcement-related deficits in ADHD in three experimental contexts; probability tracking task, partial and continuous reward schedules, and immediate versus delayed rewards. In addition, predictions were made at the neurophysiological level. Behavioral and neurophysiological predictions from the RVPM-based lesion-model of motivational dysfunction in ADHD were confirmed by data from previously published studies. RVPM represents a promising model of ADHD reinforcement learning suggesting that ACC dysregulation might play a role in the pathogenesis of motivational deficits in ADHD. However, more behavioral and neurophysiological studies are required to test core predictions of the model. In addition, the interaction with different brain networks underpinning other aspects of ADHD neuropathology (i.e., executive function) needs to be better understood.

Neuroeconomics of attention-deficit/hyperactivity disorder: differential influences of medial, dorsal, and ventral prefrontal brain networks on suboptimal decision making?

Psychiatric neuroeconomics offers an alternative approach to understanding mental disorders by studying the way disorder-related neurobiological alterations constrain economic agency, as revealed through decisions about choices between future goods. In this article, we apply this perspective to understand suboptimal decision making in attention-deficit/hyperactivity disorder (ADHD) by integrating recent advances in the neuroscience of decision making and studies of the pathophysiology of ADHD. We identify three brain networks as candidates for further study and develop specific hypotheses about how these could be implicated in ADHD. First, we postulate that altered patterns of connectivity within a network linking medial prefrontal cortex and posterior cingulate cortex (i.e., the default mode network) disrupts ordering of utilities, prospection about desired future states, setting of future goals, and implementation of aims. Second, we hypothesize that deficits in dorsal frontostriatal networks, including the dorsolateral prefrontal cortex and dorsal striatum, produce executive dysfunction-mediated impairments in the ability to compare outcome options and make choices. Third, we propose that dopaminergic dysregulation in a ventral frontostriatal network encompassing the orbitofrontal cortex, ventral striatum, and amygdala disrupts processing of cues of future utility, evaluation of experienced outcomes (feedback), and learning of associations between cues and outcomes. Finally, we extend this perspective to consider three contemporary themes in ADHD research.

Dissociation of response and feedback negativity in schizophrenia: electrophysiological and computational evidence for a deficit in the representation of value

Contrasting theories of schizophrenia propose that the disorder is characterized by a deficit in phasic changes in dopamine activity in response to ongoing events or, alternatively, by a weakness in the representation of the value of responses. Schizophrenia patients have reliably reduced brain activity following incorrect responses but other research suggests that they may have intact feedback-related potentials, indicating that the impairment may be specifically response-related. We used event-related brain potentials and computational modeling to examine this issue by comparing the neural response to outcomes with the neural response to behaviors that predict outcomes in patients with schizophrenia and psychiatrically healthy comparison subjects. We recorded feedback-related activity in a passive gambling task and a time estimation task and error-related activity in a flanker task. Patients' brain activity following an erroneous response was reduced compared to comparison subjects but feedback-related activity did not differ between groups. To test hypotheses about the possible causes of this pattern of results, we used computational modeling of the electrophysiological data to simulate the effects of an overall reduction in patients' sensitivity to feedback, selective insensitivity to positive or negative feedback, reduced learning rate, and a decreased representation of the value of the response given the stimulus on each trial. The results of the computational modeling suggest that schizophrenia patients exhibit weakened representation of response values, possibly due to failure of the basal ganglia to strongly associate stimuli with appropriate response alternatives.