Transcranial direct current stimulation over the left prefrontal cortex increases randomness of choice in instrumental learning

The neurocognitive role of working memory load when Pavlovian motivational control affects instrumental learning

Research suggests that a fast, capacity-limited working memory (WM) system and a slow, incremental reinforcement learning (RL) system jointly contribute to instrumental learning. Thus, situations that strain WM resources alter instrumental learning: under WM loads, learning becomes slow and incremental, the reliance on computationally efficient learning increases, and action selection becomes more random. It is also suggested that Pavlovian learning influences people's behavior during instrumental learning by providing hard-wired instinctive responses including approach to reward predictors and avoidance of punishment predictors. However, it remains unknown how constraints on WM resources affect instrumental learning under Pavlovian influence. Thus, we conducted a functional magnetic resonance imaging (fMRI) study (N = 49) in which participants completed an instrumental learning task with Pavlovian-instrumental conflict (the orthogonalized go/no-go task) both with and without extra WM load. Behavioral and computational modeling analyses revealed that WM load reduced the learning rate and increased random choice, without affecting Pavlovian bias. Model-based fMRI analysis revealed that WM load strengthened RPE signaling in the striatum. Moreover, under WM load, the striatum showed weakened connectivity with the ventromedial and dorsolateral prefrontal cortex when computing reward expectations. These results suggest that the limitation of cognitive resources by WM load promotes slow and incremental learning through the weakened cooperation between WM and RL; such limitation also makes action selection more random, but it does not directly affect the balance between instrumental and Pavlovian systems.

Cathodal Transcranial Direct Current Stimulation Over the Right Temporoparietal Junction Suppresses Its Functional Connectivity and Reduces Contralateral Spatial and Temporal Perception

The temporoparietal junction plays key roles in vestibular function, motor-sensory ability, and attitude stability. Conventional approaches to studying the temporoparietal junction have drawbacks, and previous studies have focused on self-motion rather than on vestibular spatial perception. Using transcranial direct current stimulation, we explored the temporoparietal junction’s effects on vestibular-guided orientation for self-motion and vestibular spatial perception. Twenty participants underwent position, motion, and time tasks, as well as functional magnetic resonance imaging scans. In the position task, cathodal transcranial direct current stimulation yielded a significantly lower response in the −6, −7, −8, −9, −10, −11, and −12 stimulus conditions for leftward rotations (P < 0.05). In the time task, the temporal bias for real transcranial direct current stimulation significantly differed from that for sham stimulation (P < 0.01). Functional magnetic resonance imaging showed that cathodal transcranial direct current stimulation suppressed functional connectivity between the temporoparietal junction, right insular cortex, and right supplementary motor area. Moreover, the change in connectivity between the right temporoparietal junction seed and the right insular cortex was positively correlated with temporal bias under stimulation. The above mentioned results show that cathodal transcranial direct current stimulation induces immediate and extended vestibular effects, which could suppress the functional connectivity of the temporoparietal junction and in turn reduce contralateral spatial and temporal perception. The consistent variation in temporal and spatial bias suggested that the temporoparietal junction may be the cortical temporal integrator for the internal model. Moreover, transcranial direct current stimulation could modulate the integration process and may thus have potential clinical applications in vestibular disorders caused by temporoparietal junction dysfunction.

Multi-session Transcranial Direct Current Stimulation Over Primary Motor Cortex Facilitates Sequence Learning, Chunking, and One Year Retention

Transcranial direct current stimulation (tDCS) over the primary motor cortex (M1) can facilitate motor learning, but it has not been established how stimulation to other brain regions impacts online and offline motor sequence learning, as well as long-term retention. Here, we completed three experiments comparing the effects of tDCS and sham stimulation to the prefrontal cortex (PFC), M1, and the supplementary motor area complex to understand the contributions of these brain regions to motor sequence learning. In Experiment 1, we found that both left and right PFC tDCS groups displayed a slowing in learning in both reaction time and number of chunks, whereas stimulation over M1 improved both metrics over the course of three sessions. To better understand the sequence learning impairment of left PFC anodal stimulation, we tested a left PFC cathodal tDCS group in Experiment 2. The cathodal group demonstrated learning impairments similar to the left PFC anodal stimulation group. In Experiment 3, a subset of participants from the left PFC, M1, and sham tDCS groups of Experiment 1 returned to complete a single session without tDCS on the same sequences assigned to them 1 year previously. We found that the M1 tDCS group reduced reaction time at a faster rate relative to the sham and left PFC groups, demonstrating faster relearning after a one-year delay. Thus, our findings suggest that, regardless of the polarity of stimulation, tDCS to PFC impairs sequence learning, whereas stimulation to M1 facilitates learning and relearning, especially in terms of chunk formation.

The Role of Frontostriatal Systems in Instructed Reinforcement Learning: Evidence From Genetic and Experimentally-Induced Variation

Instructions have a powerful effect on learning and decision-making, biasing choice even in the face of disconfirming feedback. Detrimental biasing effects have been reported in a number of studies in which instruction was given prior to trial-and-error learning. Previous work has attributed individual differences in instructional bias to variations in prefrontal and striatal dopaminergic genes, suggesting a role for prefrontally-mediated cognitive control processes in biasing learning. The current study replicates and extends these findings. Human subjects performed a probabilistic reinforcement learning task after receiving inaccurate instructions about the quality of one of the options. In order to establish a causal relationship between prefrontal cortical mechanisms and instructional bias, we applied transcranial direct current stimulation over dorsolateral prefrontal cortex (anodal, cathodal, or sham) while subjects performed the task. We additionally genotyped subjects for the COMT Val158Met genetic polymorphism, which influences the breakdown of prefrontal dopamine, and for the DAT1/SLC6A3 variable number tandem repeat, which affects expression of striatal dopamine transporter. We replicated the finding that the COMT Met allele is associated with increased instructional bias and further demonstrated that variation in DAT1 has similar effects to variation in COMT, with 9-repeat carriers demonstrating increased bias relative to 10-repeat homozygotes. Consistent with increased top-down regulation of reinforcement learning, anodal subjects demonstrated greater bias relative to sham, though this effect was present only early in training. In contrast, there was no effect of cathodal stimulation. Finally, we fit computational models to subjects' data to better characterize the mechanisms underlying instruction bias. A novel choice bias model, in which instructions influence decision-making rather than learning, was found to best account for subjects' behavior. Overall, these data provide further evidence for the role of frontostriatal interactions in biasing instructed reinforcement learning, which adds to the growing literature documenting both costs and benefits of cognitive control.

Age Effects on Spatiotemporal Dynamics of Response Inhibition: An MEG Study

Inhibition, the ability to suppress irrelevant information, thoughts or movements, is crucial for humans to perform context-appropriate behaviors. It was suggested that declined cognitive performance in older adults might be attributed to inhibitory deficiencies. Although previous studies have shown an age-associated reduction in inhibitory ability, the understanding regarding its cortical spatiotemporal maps remained limited. Thus, we used a whole-head magnetoencephalography (MEG) to elucidate the age effects on response inhibition, and to explore the brain activation differences in high- and low-performing seniors. We recruited 22 younger and 22 older adults to participate in the visual Go/No-go task. Both behavioral performance and neuromagnetic responses to No-go stimuli were analyzed. The behavioral results showed that the older adults made more false alarm (FA) errors than the younger adults did. The MEG results showed that the seniors exhibited declined cortical activities in middle temporal gyrus (MTG) and delayed activation in MTG, prefrontal cortex (PFC) and pre-supplementary motor area (pre-SMA). Furthermore, among the older adults, more recruitment of the left PFC was found in the high-performers than in the lower-performers. In conclusion, age-related deficiencies in response inhibition were observed in both behavioral performance and neurophysiological measurement. Our results also suggested that frontal recruitment plays a compensatory role in successful inhibition.

Placebo Intervention Enhances Reward Learning in Healthy Individuals

According to the placebo-reward hypothesis, placebo is a reward-anticipation process that increases midbrain dopamine (DA) levels. Reward-based learning processes, such as reinforcement learning, involves a large part of the DA-ergic network that is also activated by the placebo intervention. Given the neurochemical overlap between placebo and reward learning, we investigated whether verbal instructions in conjunction with a placebo intervention are capable of enhancing reward learning in healthy individuals by using a monetary reward-based reinforcement-learning task. Placebo intervention was performed with non-invasive brain stimulation techniques. In a randomized, triple-blind, cross-over study we investigated this cognitive placebo effect in healthy individuals by manipulating the participants’ perceived uncertainty about the intervention’s efficacy. Volunteers in the purportedly low- and high-uncertainty conditions earned more money, responded more quickly and had a higher learning rate from monetary rewards relative to baseline. Participants in the purportedly high-uncertainty conditions showed enhanced reward learning, and a model-free computational analysis revealed a higher learning rate from monetary rewards compared to the purportedly low-uncertainty and baseline conditions. Our results indicate that the placebo response is able to enhance reward learning in healthy individuals, opening up exciting avenues for future research in placebo effects on other cognitive functions.

The effect of the interval-between-sessions on prefrontal transcranial direct current stimulation (tDCS) on cognitive outcomes: a systematic review and meta-analysis

Recently, there has been wide interest in the effects of transcranial direct current stimulation (tDCS) of the dorsolateral prefrontal cortex (DLPFC) on cognitive functioning. However, many methodological questions remain unanswered. One of them is whether the time interval between active and sham-controlled stimulation sessions, i.e. the interval between sessions (IBS), influences DLPFC tDCS effects on cognitive functioning. Therefore, a systematic review and meta-analysis was performed of experimental studies published in PubMed, Science Direct, and other databases from the first data available to February 2016. Single session sham-controlled within-subject studies reporting the effects of tDCS of the DLPFC on cognitive functioning in healthy controls and neuropsychiatric patients were included. Cognitive tasks were categorized in tasks assessing memory, attention, and executive functioning. Evaluation of 188 trials showed that anodal vs. sham tDCS significantly decreased response times and increased accuracy, and specifically for the executive functioning tasks, in a sample of healthy participants and neuropsychiatric patients (although a slightly different pattern of improvement was found in analyses for both samples separately). The effects of cathodal vs. sham tDCS (45 trials), on the other hand, were not significant. IBS ranged from less than 1 h to up to 1 week (i.e. cathodal tDCS) or 2 weeks (i.e. anodal tDCS). This IBS length had no influence on the estimated effect size when performing a meta-regression of IBS on reaction time and accuracy outcomes in all three cognitive categories, both for anodal and cathodal stimulation. Practical recommendations and limitations of the study are further discussed.

A Systematic Review and Meta-Analysis of the Effects of Transcranial Direct Current Stimulation (tDCS) Over the Dorsolateral Prefrontal Cortex in Healthy and Neuropsychiatric Samples: Influence of Stimulation Parameters

BACKGROUND

Research into the effects of transcranial direct current stimulation of the dorsolateral prefrontal cortex on cognitive functioning is increasing rapidly. However, methodological heterogeneity in prefrontal tDCS research is also increasing, particularly in technical stimulation parameters that might influence tDCS effects.

OBJECTIVE

To systematically examine the influence of technical stimulation parameters on DLPFC-tDCS effects.

METHODS

We performed a systematic review and meta-analysis of tDCS studies targeting the DLPFC published from the first data available to February 2016. Only single-session, sham-controlled, within-subject studies reporting the effects of tDCS on cognition in healthy controls and neuropsychiatric patients were included.

RESULTS

Evaluation of 61 studies showed that after single-session a-tDCS, but not c-tDCS, participants responded faster and more accurately on cognitive tasks. Sub-analyses specified that following a-tDCS, healthy subjects responded faster, while neuropsychiatric patients responded more accurately. Importantly, different stimulation parameters affected a-tDCS effects, but not c-tDCS effects, on accuracy in healthy samples vs.

PATIENTS

increased current density and density charge resulted in improved accuracy in healthy samples, most prominently in females; for neuropsychiatric patients, task performance during a-tDCS resulted in stronger increases in accuracy rates compared to task performance following a-tDCS.

CONCLUSIONS

Healthy participants respond faster, but not more accurate on cognitive tasks after a-tDCS. However, increasing the current density and/or charge might be able to enhance response accuracy, particularly in females. In contrast, online task performance leads to greater increases in response accuracy than offline task performance in neuropsychiatric patients. Possible implications and practical recommendations are discussed.

Using fMRI to study reward processing in humans: past, present, and future

Functional magnetic resonance imaging (fMRI) is a noninvasive tool used to probe cognitive and affective processes. Although fMRI provides indirect measures of neural activity, the advent of fMRI has allowed for1) the corroboration of significant animal findings in the human brain, and2) the expansion of models to include more common human attributes that inform behavior. In this review, we briefly consider the neural basis of the blood oxygenation level dependent signal to set up a discussion of how fMRI studies have applied it in examining cognitive models in humans and the promise of using fMRI to advance such models. Specifically, we illustrate the contribution that fMRI has made to the study of reward processing, focusing on the role of the striatum in encoding reward-related learning signals that drive anticipatory and consummatory behaviors. For instance, we discuss how fMRI can be used to link neural signals (e.g., striatal responses to rewards) to individual differences in behavior and traits. While this functional segregation approach has been constructive to our understanding of reward-related functions, many fMRI studies have also benefitted from a functional integration approach that takes into account how interconnected regions (e.g., corticostriatal circuits) contribute to reward processing. We contend that future work using fMRI will profit from using a multimodal approach, such as combining fMRI with noninvasive brain stimulation tools (e.g., transcranial electrical stimulation), that can identify causal mechanisms underlying reward processing. Consequently, advancements in implementing fMRI will promise new translational opportunities to inform our understanding of psychopathologies.

A Randomized, Double-Blind, Sham-Controlled Trial of Transcranial Direct Current Stimulation in Attention-Deficit/Hyperactivity Disorder

Background

Current standardized treatments for cognitive impairment in attention-deficit/hyperactivity disorder remain limited and their efficacy restricted. Transcranial direct current stimulation (tDCS) is a promising tool for enhancing cognitive performance in several neuropsychiatric disorders. Nevertheless, the effects of tDCS in reducing cognitive impairment in patients with attention-deficit/hyperactivity disorder (ADHD) have not yet been investigated.

Methods

A parallel, randomized, double-blind, sham-controlled trial was conducted to examine the efficacy of tDCS on the modulation of inhibitory control in adults with ADHD. Thirty patients were randomly allocated to each group and performed a go/no-go task before and after a single session of either anodal stimulation (1 mA) over the left dorsolateral prefrontal cortex or sham stimulation.

Results

A nonparametric two-sample Wilcoxon rank-sum (Mann-Whitney) test revealed no significant differences between the two groups of individuals with ADHD (tDCS vs. sham) in regard to behavioral performance in the go/no go tasks. Furthermore, the effect sizes of group differences after treatment for the primary outcome measures—correct responses, impulsivity and omission errors—were small. No adverse events resulting from stimulation were reported.

Conclusion

According to these findings, there is no evidence in support of the use of anodal stimulation over the left dorsolateral prefrontal cortex as an approach for improving inhibitory control in ADHD patients. To the best of our knowledge, this is the first clinical study to assess the cognitive effects of tDCS in individuals with ADHD. Further research is needed to assess the clinical efficacy of tDCS in this population.

Trial Registration

ClinicalTrials.gov NCT01968512