Predictive value and reward in implicit classification learning

The Role of the Human Cerebellum for Learning from and Processing of External Feedback in Non-Motor Learning: A Systematic Review

This review aimed to systematically identify and comprehensively review the role of the cerebellum in performance monitoring, focusing on learning from and on processing of external feedback in non-motor learning. While 1078 articles were screened for eligibility, ultimately 36 studies were included in which external feedback was delivered in cognitive tasks and which referenced the cerebellum. These included studies in patient populations with cerebellar damage and studies in healthy subjects applying neuroimaging. Learning performance in patients with different cerebellar diseases was heterogeneous, with only about half of all patients showing alterations. One patient study using EEG demonstrated that damage to the cerebellum was associated with altered neural processing of external feedback. Studies assessing brain activity with task-based fMRI or PET and one resting-state functional imaging study that investigated connectivity changes following feedback-based learning in healthy participants revealed involvement particularly of lateral and posterior cerebellar regions in processing of and learning from external feedback. Cerebellar involvement was found at different stages, e.g., during feedback anticipation and following the onset of the feedback stimuli, substantiating the cerebellum's relevance for different aspects of performance monitoring such as feedback prediction. Future research will need to further elucidate precisely how, where, and when the cerebellum modulates the prediction and processing of external feedback information, which cerebellar subregions are particularly relevant, and to what extent cerebellar diseases alter these processes.

Excitatory cerebellar transcranial direct current stimulation boosts the leverage of prior knowledge for predicting actions

The cerebellum causally supports social processing by generating internal models of social events based on statistical learning of behavioral regularities. However, whether the cerebellum is only involved in forming or also in using internal models for the prediction of forthcoming actions is still unclear. We used cerebellar transcranial Direct Current Stimulation (ctDCS) to modulate the performance of healthy adults in using previously learned expectations in an action prediction task. In a first learning phase of this task, participants were exposed to different levels of associations between specific actions and contextual elements, to induce the formation of either strongly or moderately informative expectations. In a following testing phase, which assessed the use of these expectations for predicting ambiguous (i.e. temporally occluded) actions, we delivered ctDCS. Results showed that anodic, compared to sham, ctDCS boosted the prediction of actions embedded in moderately, but not strongly, informative contexts. Since ctDCS was delivered during the testing phase, that is after expectations were established, our findings suggest that the cerebellum is causally involved in using internal models (and not just in generating them). This encourages the exploration of the clinical effects of ctDCS to compensate poor use of predictive internal models for social perception.

Reward network dysfunction is associated with cognitive impairment after stroke

Stroke survivors not only suffer from severe motor, speech and neurocognitive deficits, but in many cases also from a "lack of pleasure" and a reduced motivational level. Especially apathy and anhedonic symptoms can be linked to a dysfunction of the reward system. Rewards are considered as important co-factor for learning, so the question arises as to why and how this affects the rehabilitation of stroke patients. We investigated reward behaviour, learning ability and brain network connectivity in acute (3-7d) mild to moderate stroke patients (n = 28) and age-matched healthy controls (n = 26). Reward system activity was assessed using the Monetary Incentive Delay task (MID) during magnetoencephalography (MEG). Coherence analyses were used to demonstrate reward effects on brain functional network connectivity. The MID-task showed that stroke survivors had lower reward sensitivity and required greater monetary incentives to improve performance and showed deficits in learning improvement. MEG-analyses showed a reduced network connectivity in frontal and temporoparietal regions. All three effects (reduced reward sensitivity, reduced learning ability and altered cerebral connectivity) were found to be closely related and differed strongly from the healthy group. Our results reinforce the notion that acute stroke induces reward network dysfunction, leading to functional impairment of behavioural systems. These findings are representative of a general pattern in mild strokes and are independent of the specific lesion localisation. For stroke rehabilitation, these results represent an important point to identify the reduced learning capacity after stroke and to implement individualised recovery exercises accordingly.

The Effect of a Personalized Newsletter to Physical Therapists on Patient Recruitment: A Cluster Randomized Trial in Primary Physiotherapy Care

OBJECTIVE

To assess the effect of a personalized newsletter compared with a standard newsletter on patient recruitment in physiotherapy research.

METHODS

We performed a cluster-randomized trial including 120 physiotherapists who recruited patients for a prospective cohort and were randomly assigned to either receiving personalized feedback in a newsletter (intervention group) or a standard newsletter (control group). We calculated the difference in the number of patients included in the study corrected for inclusion time between both groups.

RESULTS

The physiotherapists in the control group (n = 59) included 110 patients (35.4% of the total number of patients included) compared with an inclusion of 200 patients (64.6% of the total number of patients included) by the physiotherapists in the intervention group (n = 61), a difference of 90 patients in favor of the intervention group. However, when corrected for inclusion time and a cluster effect, we found no statistically significant difference between both groups. In addition, therapists who did not include a single patient (inactive therapists) were evenly divided between the 2 groups (n = 29 [49%] in the control group; n = 30 [49%] in the intervention group).

CONCLUSIONS

A personalized newsletter does not significantly increase the number of recruited patients by physiotherapists. However, therapists receiving personalized feedback recruited nearly double the number of patients compared with the ones receiving standard feedback.

Resistance, vulnerability and resilience: A review of the cognitive cerebellum in aging and neurodegenerative diseases

In the context of neurodegeneration and aging, the cerebellum is an enigma. Genetic markers of cellular aging in cerebellum accumulate more slowly than in the rest of the brain, and it generates unknown factors that may slow or even reverse neurodegenerative pathology in animal models of Alzheimer's Disease (AD). Cerebellum shows increased activity in early AD and Parkinson's disease (PD), suggesting a compensatory function that may mitigate early symptoms of neurodegenerative pathophysiology. Perhaps most notably, different parts of the brain accumulate neuropathological markers of AD in a recognized progression and generally, cerebellum is the last brain region to do so. Taken together, these data suggest that cerebellum may be resistant to certain neurodegenerative mechanisms. On the other hand, in some contexts of accelerated neurodegeneration, such as that seen in chronic traumatic encephalopathy (CTE) following repeated traumatic brain injury (TBI), the cerebellum appears to be one of the most susceptible brain regions to injury and one of the first to exhibit signs of pathology. Cerebellar pathology in neurodegenerative disorders is strongly associated with cognitive dysfunction. In neurodegenerative or neurological disorders associated with cerebellar pathology, such as spinocerebellar ataxia, cerebellar cortical atrophy, and essential tremor, rates of cognitive dysfunction, dementia and neuropsychiatric symptoms increase. When the cerebellum shows AD pathology, such as in familial AD, it is associated with earlier onset and greater severity of disease. These data suggest that when neurodegenerative processes are active in the cerebellum, it may contribute to pathological behavioral outcomes. The cerebellum is well known for comparing internal representations of information with observed outcomes and providing real-time feedback to cortical regions, a critical function that is disturbed in neuropsychiatric disorders such as intellectual disability, schizophrenia, dementia, and autism, and required for cognitive domains such as working memory. While cerebellum has reciprocal connections with non-motor brain regions and likely plays a role in complex, goal-directed behaviors, it has proven difficult to establish what it does mechanistically to modulate these behaviors. Due to this lack of understanding, it's not surprising to see the cerebellum reflexively dismissed or even ignored in basic and translational neuropsychiatric literature. The overarching goals of this review are to answer the following questions from primary literature: When the cerebellum is affected by pathology, is it associated with decreased cognitive function? When it is intact, does it play a compensatory or protective role in maintaining cognitive function? Are there theoretical frameworks for understanding the role of cerebellum in cognition, and perhaps, illnesses characterized by cognitive dysfunction? Understanding the role of the cognitive cerebellum in neurodegenerative diseases has the potential to offer insight into origins of cognitive deficits in other neuropsychiatric disorders, which are often underappreciated, poorly understood, and not often treated.

The role of the cerebellum for feedback processing and behavioral switching in a reversal-learning task

Previous studies have reported cerebellar activations during error and reward processing. The present study investigated if the cerebellum differentially processes feedback depending on changes in response strategy during reversal learning, as is conceivable given its internal models for movement and thought. Negative relative to positive feedback in an fMRI-based reversal learning task was hypothesized to be associated with increased cerebellar activations. Moreover, increased activations were expected for negative feedback followed by a change in response strategy compared to negative feedback not followed by such a change, and for first positive feedback after compared to final negative feedback before a change, due to updating of internal models. As predicted, activation in lobules VI and VIIa/Crus I was increased for negative relative to positive feedback, and for final negative feedback before a change in response strategy relative to negative feedback not associated with a change. Moreover, activation was increased for first positive feedback after relative to final negative feedback before a change. These findings are consistent with updating of cerebellar internal models to accommodate new behavioral strategies. Recruitment of posterior regions in reversal learning is in line with the cerebellar functional topography, with posterior regions involved in complex motor and cognitive functions.

Learning stage-dependent effect of M1 disruption on value-based motor decisions

The present study aimed at characterizing the impact of M1 disruption on the implementation of implicit value information in motor decisions, at both early stages (during reinforcement learning) and late stages (after consolidation) of action value encoding. Fifty subjects performed, over three consecutive days, a task that required them to select between two finger responses according to the color (instruction) and to the shape (implicit, undisclosed rule) of an imperative signal: considering the implicit rule in addition to the instruction allowed subjects to earn more money. We investigated the functional contribution of M1 to the implementation of the implicit rule in subjects' motor decisions. Continuous theta burst stimulation (cTBS) was applied over M1 either on Day 1 or on Day 3, producing a temporary lesion either during reinforcement learning (cTBS_Learning group) or after consolidation of the implicit rule, during decision-making (cTBS_Decision group), respectively. Interestingly, disrupting M1 activity on Day 1 improved the reliance on the implicit rule, plausibly because M1 cTBS increased dopamine release in the putamen in an indirect way. This finding corroborates the view that cTBS may affect activity in unstimulated areas, such as the basal ganglia. Notably, this effect was short-lasting; it did not persist overnight, suggesting that the functional integrity of M1 during learning is a prerequisite for the consolidation of implicit value information to occur. Besides, cTBS over M1 did not impact the use of the implicit rule when applied on Day 3, although it did so when applied on Day 2 in a recent study where the reliance on the implicit rule declined following cTBS (Derosiere et al., 2017). Overall, these findings indicate that the human M1 is functionally involved in the consolidation and implementation of implicit value information underlying motor decisions. However, M1 contribution seems to vanish as subjects become more experienced in using the implicit value information to make their motor decisions.

Cerebellar tDCS Does Not Enhance Performance in an Implicit Categorization Learning Task

Background: Transcranial Direct Current Stimulation (tDCS) is a form of non-invasive electrical stimulation that changes neuronal excitability in a polarity and site-specific manner. In cognitive tasks related to prefrontal and cerebellar learning, cortical tDCS arguably facilitates learning, but the few studies investigating cerebellar tDCS, however, are inconsistent.

Objective: We investigate the effect of cerebellar tDCS on performance of an implicit categorization learning task.

Methods: Forty participants performed a computerized version of an implicit categorization learning task where squares had to be sorted into two categories, according to an unknown but fixed rule that integrated both the size and luminance of the square. Participants did one round of categorization to familiarize themselves with the task and to provide a baseline of performance. After that, 20 participants received anodal tDCS (20 min, 1.5 mA) over the right cerebellum, and 19 participants received sham stimulation and simultaneously started a second session of the categorization task using a new rule.

Results: As expected, subjects performed better in the second session than in the first, baseline session, showing increased accuracy scores and reduced reaction times. Over trials, participants learned the categorization rule, improving their accuracy and reaction times. However, we observed no effect of anodal tDCS stimulation on overall performance or on learning, compared to sham stimulation.

Conclusion: These results suggest that cerebellar tDCS does not modulate performance and learning on an implicit categorization task.

Cerebellar tDCS does not improve performance in probabilistic classification learning

In this study, the role of the cerebellum in a cognitive learning task using transcranial direct current stimulation (tDCS) was investigated. Using a weather prediction task, subjects had to learn the probabilistic associations between a stimulus (a combination of cards) and an outcome (sun or rain). This task is a variant of a probabilistic classification learning task, for which it has been reported that prefrontal tDCS enhances performance. Using a between-subject design, all 30 subjects learned to improve their performance with increasing accuracies and shortened response times over a series of 500 trials. Subjects also became more confident in their prediction during the experiment. However, no differences in performance and learning were observed between subjects receiving sham stimulation (n = 10) or anodal stimulation (2 mA for 20 min) over either the right cerebellum (n = 10) or the left prefrontal cortex (n = 10). This suggests that stimulating the brain with cerebellar tDCS does not readily influence probabilistic classification performances, probably due to the rather complex nature of this cognitive task.

Primary motor cortex contributes to the implementation of implicit value-based rules during motor decisions

In the present study, we investigated the functional contribution of the human primary motor cortex (M1) to motor decisions. Continuous theta burst stimulation (cTBS) was used to alter M1 activity while participants performed a decision-making task in which the reward associated with the subjects' responses (right hand finger movements) depended on explicit and implicit value-based rules. Subjects performed the task over two consecutive days and cTBS occurred in the middle of Day 2, once the subjects were just about to implement implicit rules, in addition to the explicit instructions, to choose their responses, as evident in the control group (cTBS over the right somatosensory cortex). Interestingly, cTBS over the left M1 prevented subjects from implementing the implicit value-based rule while its implementation was enhanced in the group receiving cTBS over the right M1. Hence, cTBS had opposite effects depending on whether it was applied on the contralateral or ipsilateral M1. The use of the explicit value-based rule was unaffected by cTBS in the three groups of subject. Overall, the present study provides evidence for a functional contribution of M1 to the implementation of freshly acquired implicit rules, possibly through its involvement in a cortico-subcortical network controlling value-based motor decisions.

The role of the human cerebellum in performance monitoring

While the cerebellum has traditionally been thought of as mainly involved in motor functions, evidence has been accumulating for cerebellar contributions also to non-motor, cognitive functions. The notion of a cerebellar internal model underlying prediction and processing of sensory events and coordination and fine-tuning of appropriate responses has put the cerebellum right at the interface of motor behavior and cognition. Along these lines, the cerebellum may critically contribute to performance monitoring, a set of cognitive and affective functions underlying adaptive behavior. This review presents and integrates evidence from recent neuroimaging and clinical studies for a cerebellar role in performance monitoring with focus on sensory prediction, error and conflict processing, response inhibition, and feedback learning. Together with evidence for involvement in articulatory monitoring during working memory, these findings suggest monitoring as the cerebellum's overarching function.

Roux-en-Y Gastric Bypass Alters Brain Activity in Regions that Underlie Reward and Taste Perception

BACKGROUND

Roux-en-Y gastric bypass (RYGB) surgery is a very effective bariatric procedure to achieve significant and sustained weight loss, yet little is known about the procedure's impact on the brain. This study examined the effects of RYGB on the brain's response to the anticipation of highly palatable versus regular food.

METHODS

High fat diet-induced obese rats underwent RYGB or sham operation and were then tested for conditioned place preference (CPP) for the bacon-paired chamber, relative to the chow-paired chamber. After CPP, animals were placed in either chamber without the food stimulus, and brain-glucose metabolism (BGluM) was measured using positron emission tomography (μPET).

RESULTS

Bacon CPP was only observed in RYGB rats that had stable weight loss following surgery. BGluM assessment revealed that RYGB selectively activated regions of the right and midline cerebellum (Lob 8) involved in subjective processes related to reward or expectation. Also, bacon anticipation led to significant activation in the medial parabrachial nuclei (important in gustatory processing) and dorsomedial tegmental area (key to reward, motivation, cognition and addiction) in RYGB rats; and activation in the retrosplenial cortex (default mode network), and the primary visual cortex in control rats.

CONCLUSIONS

RYGB alters brain activity in areas involved in reward expectation and sensory (taste) processing when anticipating a palatable fatty food. Thus, RYGB may lead to changes in brain activity in regions that process reward and taste-related behaviors. Specific cerebellar regions with altered metabolism following RYGB may help identify novel therapeutic targets for treatment of obesity.

Processing of action- but not stimulus-related prediction errors differs between active and observational feedback learning

Learning of stimulus-response-outcome associations is driven by outcome prediction errors (PEs). Previous studies have shown larger PE-dependent activity in the striatum for learning from own as compared to observed actions and the following outcomes despite comparable learning rates. We hypothesised that this finding relates primarily to a stronger integration of action and outcome information in active learners. Using functional magnetic resonance imaging, we investigated brain activations related to action-dependent PEs, reflecting the deviation between action values and obtained outcomes, and action-independent PEs, reflecting the deviation between subjective values of response-preceding cues and obtained outcomes. To this end, 16 active and 15 observational learners engaged in a probabilistic learning card-guessing paradigm. On each trial, active learners saw one out of five cues and pressed either a left or right response button to receive feedback (monetary win or loss). Each observational learner observed exactly those cues, responses and outcomes of one active learner. Learning performance was assessed in active test trials without feedback and did not differ between groups. For both types of PEs, activations were found in the globus pallidus, putamen, cerebellum, and insula in active learners. However, only for action-dependent PEs, activations in these structures and the anterior cingulate were increased in active relative to observational learners. Thus, PE-related activity in the reward system is not generally enhanced in active relative to observational learning but only for action-dependent PEs. For the cerebellum, additional activations were found across groups for cue-related uncertainty, thereby emphasising the cerebellum's role in stimulus-outcome learning.

The subthalamic nucleus modulates the early phase of probabilistic classification learning

Previous models proposed that the subthalamic nucleus (STN) is critical in the early phase of skill acquisition. We hypothesized that subthalamic deep brain stimulation modulates the learning curve in early classification learning. Thirteen idiopathic Parkinson's disease patients (iPD) with subthalamic deep brain stimulation (STN-DBS), 9 medically treated iPD, and 21 age-matched healthy controls were tested with a probabilistic classification task. STN-DBS patients were tested with stimulation OFF and ON, and medically treated patients with medication OFF and ON, respectively. Performance and reaction time were analyzed on the first 100 consecutive trials as early learning phase. Moreover, data were separated for low and high-probability patterns, and more differentiated strategy analyses were used. The major finding was a significant modulation of the learning curve in DBS patients with stimulation ON: although overall learning was similar to healthy controls, only the stimulation ON group showed a transient significant performance dip from trials '41-60' that rapidly recovered. Further analysis indicated that this might be paralleled by a modulation of the learning strategy, particularly on the high-probability patterns. The reaction time was unchanged during the dip. Our study supports that the STN serves as a relay in early classification learning and directs attention toward unacquainted content. The STN might play a role in balancing the short-term success against strategy optimization for improved long-term outcome.

Rewarding context accelerates implicit guidance in visual search

It is well known that observers can implicitly learn the spatial context of complex visual searches, such that future searches through repeated contexts are completed faster than those through novel contexts, even though observers remain at chance at discriminating repeated from new contexts. This contextual-cueing effect arises quickly (within less than five exposures) and asymptotes within 30 exposures to repeated contexts. In spite of being a robust effect (its magnitude is over 100 ms at the asymptotic level), the effect is implicit: Participants are usually at chance at discriminating old from new contexts at the end of an experiment, in spite of having seen each repeated context more than 30 times throughout a 50-min experiment. Here, we demonstrate that the speed at which the contextual-cueing effect arises can be modulated by external rewards associated with the search contexts (not with the performance itself). Following each visual search trial (and irrespective of a participant's search speed on the trial), we provided a reward, a penalty, or no feedback to the participant. Crucially, the type of feedback obtained was associated with the specific contexts, such that some repeated contexts were always associated with reward, and others were always associated with penalties. Implicit learning occurred fastest for contexts associated with positive feedback, though penalizing contexts also showed a learning benefit. Consistent feedback also produced faster learning than did variable feedback, though unexpected penalties produced the largest immediate effects on search performance.

Association between white matter fiber structure and reward-related reactivity of the ventral striatum

Individual responsiveness to rewards or rewarding stimuli may affect various domains of normal as well as pathological behavior. The ventral striatum/nucleus accumbens (NAcc) constitutes a key brain structure in the regulation of reward-appetitive behavior. It remains unclear, however, to which extent individual reward-related BOLD response in the NAcc is dependent on individual characteristics of connecting white matter fiber tracts. Using tract-based spatial statistics (TBSS) and statistical parametric mapping (SPM) this combined DTI - fMRI study investigated this question by correlating NAcc BOLD signal upon receipt of a monetary reward with different white matter characteristics (FA, axial diffusivity, radial diffusivity). The results show that increased integrity of white matter as assessed by FA in the cingulate and corpus callosum, the inferior fronto-occipital fasciculus, the anterior thalamic radiation and the anterior limb of the internal capsule was positively correlated with reward-related activation in the NAcc. There were no negative correlations as well as no significant results regarding axial and radial diffusivity. These findings indicate that microstructural properties of fiber tracts connecting, amongst others, the cortex with the striatum may influence intensity of reward-related responsiveness of the ventral striatum by constraining or increasing efficiency in information transfer within relevant circuitries involved in processing of reward.