When memory fails, intuition reigns: midazolam enhances implicit inference in humans

Cross-frequency and inter-regional phase synchronization in explicit transitive inference

Explicit logical reasoning, like transitive inference, is a hallmark of human intelligence. This study investigated cortical oscillations and their interactions in transitive inference with EEG. Participants viewed premises describing abstract relations among items. They accurately recalled the relationship between old pairs of items, effectively inferred the relationship between new pairs of items, and discriminated between true and false relationships for new pairs. First, theta (4-7 Hz) and alpha oscillations (8-15 Hz) had distinct functional roles. Frontal theta oscillations distinguished between new and old pairs, reflecting the inference of new information. Parietal alpha oscillations changed with serial position and symbolic distance of the pairs, representing the underlying relational structure. Frontal alpha oscillations distinguished between true and false pairs, linking the new information with the underlying relational structure. Second, theta and alpha oscillations interacted through cross-frequency and inter-regional phase synchronization. Frontal theta-alpha 1:2 phase locking appeared to coordinate spectrally diverse neural activity, enhanced for new versus old pairs and true versus false pairs. Alpha-band frontal-parietal phase coherence appeared to coordinate anatomically distributed neural activity, enhanced for new versus old pairs and false versus true pairs. It suggests that cross-frequency and inter-regional phase synchronization among theta and alpha oscillations supports human transitive inference.

Atypical Reinforcement Learning in Developmental Dyslexia

OBJECTIVES

According to the Procedural Deficit Hypothesis, abnormalities in corticostriatal pathways could account for the language-related deficits observed in developmental dyslexia. The same neural network has also been implicated in the ability to learn contingencies based on trial and error (i.e., reinforcement learning [RL]). On this basis, the present study tested the assumption that dyslexic individuals would be impaired in RL compared with neurotypicals in two different tasks.

METHODS

In a probabilistic selection task, participants were required to learn reinforcement contingencies based on probabilistic feedback. In an implicit transitive inference task, participants were also required to base their decisions on reinforcement histories, but feedback was deterministic and stimulus pairs were partially overlapping, such that participants were required to learn hierarchical relations.

RESULTS

Across tasks, results revealed that although the ability to learn from positive/negative feedback did not differ between the two groups, the learning of reinforcement contingencies was poorer in the dyslexia group compared with the neurotypicals group. Furthermore, in novel test pairs where previously learned information was presented in new combinations, dyslexic individuals performed similarly to neurotypicals.

CONCLUSIONS

Taken together, these results suggest that learning of reinforcement contingencies occurs less robustly in individuals with developmental dyslexia. Inferences for the neuro-cognitive mechanisms of developmental dyslexia are discussed.

A direct test of competitive versus cooperative episodic-procedural network dynamics in human memory

Classical lesion studies led to a consensus that episodic and procedural memory arises from segregated networks identified with the hippocampus and the caudate nucleus, respectively. Neuroimaging studies, however, show that competitive and cooperative interactions occur between networks during memory tasks. Furthermore, causal experiments to manipulate connectivity between these networks have not been performed in humans. Although nodes common to both networks, such as the precuneus and ventrolateral thalamus, may mediate their interaction, there is no experimental evidence for this. We tested how network-targeted noninvasive brain stimulation affects episodic-procedural network interactions and how these network manipulations affect episodic and procedural memory in healthy young adults. Compared to control (vertex) stimulation, hippocampal network-targeted stimulation increased within-network functional connectivity and hippocampal connectivity with the caudate. It also increased episodic, relative to procedural, memory, and this persisted one week later. The differential effect on episodic versus procedural memory was associated with increased functional connectivity between the caudate, precuneus, and ventrolateral thalamus. These findings provide direct evidence of episodic-procedural network competition, mediated by regions common to both networks. Enhanced hippocampal network connectivity may boost episodic, but decrease procedural, memory by co-opting resources shared between networks.

Unlocking adults' implicit statistical learning by cognitive depletion

Human learning is supported by multiple neural mechanisms that maturate at different rates and interact in mostly cooperative but also sometimes competitive ways. We tested the hypothesis that mature cognitive mechanisms constrain implicit statistical learning mechanisms that contribute to early language acquisition. Specifically, we tested the prediction that depleting cognitive control mechanisms in adults enhances their implicit, auditory word-segmentation abilities. Young adults were exposed to continuous streams of syllables that repeated into hidden novel words while watching a silent film. Afterward, learning was measured in a forced-choice test that contrasted hidden words with nonwords. The participants also had to indicate whether they explicitly recalled the word or not in order to dissociate explicit versus implicit knowledge. We additionally measured electroencephalography during exposure to measure neural entrainment to the repeating words. Engagement of the cognitive mechanisms was manipulated by using two methods. In experiment 1 (n = 36), inhibitory theta-burst stimulation (TBS) was applied to the left dorsolateral prefrontal cortex or to a control region. In experiment 2 (n = 60), participants performed a dual working-memory task that induced high or low levels of cognitive fatigue. In both experiments, cognitive depletion enhanced word recognition, especially when participants reported low confidence in remembering the words (i.e., when their knowledge was implicit). TBS additionally modulated neural entrainment to the words and syllables. These findings suggest that cognitive depletion improves the acquisition of linguistic knowledge in adults by unlocking implicit statistical learning mechanisms and support the hypothesis that adult language learning is antagonized by higher cognitive mechanisms.

Recovering Reliable Idiographic Biological Parameters from Noisy Behavioral Data: the Case of Basal Ganglia Indices in the Probabilistic Selection Task

Behavioral data, despite being a common index of cognitive activity, is under scrutiny for having poor reliability as a result of noise or lacking replications of reliable effects. Here, we argue that cognitive modeling can be used to enhance the test-retest reliability of the behavioral measures by recovering individual-level parameters from behavioral data. We tested this empirically with the Probabilistic Stimulus Selection (PSS) task, which is used to measure a participant’s sensitivity to positive or negative reinforcement. An analysis of 400,000 simulations from an Adaptive Control of Thought-Rational (ACT-R) model of this task showed that the poor reliability of the task is due to the instability of the end-estimates: because of the way the task works, the same participants might sometimes end up having apparently opposite scores. To recover the underlying interpretable parameters and enhance reliability, we used a Bayesian Maximum A Posteriori (MAP) procedure. We were able to obtain reliable parameters across sessions (intraclass correlation coefficient ≈ 0.5). A follow-up study on a modified version of the task also found the same pattern of results, with very poor test-retest reliability in behavior but moderate reliability in recovered parameters (intraclass correlation coefficient ≈ 0.4). Collectively, these results imply that this approach can further be used to provide superior measures in terms of reliability, and bring greater insights into individual differences.

Chained study and the discovery of relational structure

Prior knowledge of relational structure allows people to quickly make sense of and respond to new experiences. When awareness of such structure is not necessary to support learning, however, it is unclear when and why individuals "spontaneously discover" an underlying relational schema. The present study examines the determinants of such discovery in discrimination-based transitive inference (TI), whereby people learn about a hierarchy of interrelated premises and are tested on their ability to draw inferences that bridge studied relations. Experiencing "chained" sequences of overlapping premises during training was predicted to facilitate the discovery of relational structure. Among individuals without prior knowledge of the hierarchy, chaining improved relational learning and was most likely to result in explicit awareness of the underlying relations between items. Observation of chained training sequences was also more effective than the self-generation of training sequences. These findings add to growing evidence that the temporal dynamics of training, including successive presentation of overlapping associations, are key to understanding spontaneous relational discovery during learning.

Neuropsychopharmacological effects of midazolam on the human brain

As a commonly used anesthetic agent, midazolam has the properties of water-soluble, rapid onset, and short duration of action. With the rapid development in the field of neuroimaging, numerous studies have investigated how midazolam acts on the human brain to induce the alteration of consciousness. However, the neural bases of midazolam-induced sedation or anesthesia remain beginning to be understood in detail. In this review, we summarize findings from neuroimaging studies that have used midazolam to study altered consciousness at different levels and content. We also compare the results to those of neuroimaging studies using diverse anesthetic agents and describe the common neural correlates of anesthetic-induced alteration of consciousness.

Beyond dichotomies in reinforcement learning

Reinforcement learning (RL) is a framework of particular importance to psychology, neuroscience and machine learning. Interactions between these fields, as promoted through the common hub of RL, has facilitated paradigm shifts that relate multiple levels of analysis in a singular framework (for example, relating dopamine function to a computationally defined RL signal). Recently, more sophisticated RL algorithms have been proposed to better account for human learning, and in particular its oft-documented reliance on two separable systems: a model-based (MB) system and a model-free (MF) system. However, along with many benefits, this dichotomous lens can distort questions, and may contribute to an unnecessarily narrow perspective on learning and decision-making. Here, we outline some of the consequences that come from overconfidently mapping algorithms, such as MB versus MF RL, with putative cognitive processes. We argue that the field is well positioned to move beyond simplistic dichotomies, and we propose a means of refocusing research questions towards the rich and complex components that comprise learning and decision-making.

Learned Representation of Implied Serial Order in Posterior Parietal Cortex

Monkeys can learn the implied ranking of pairs of images drawn from an ordered set, despite never seeing all of the images simultaneously and without explicit spatial or temporal cues. We recorded the activity of posterior parietal cortex (including lateral intraparietal area LIP) neurons while monkeys learned 7-item transitive inference (TI) lists with 2 items presented on each trial. Behavior and neuronal activity were significantly influenced by the ordinal relationship of the stimulus pairs, specifically symbolic distance (the difference in rank) and joint rank (the sum of the ranks). Symbolic distance strongly predicted decision accuracy and learning rate. An effect of joint rank on performance was found nested within the symbolic distance effect. Across the population of neurons, there was significant modulation of firing correlated with the relative ranks of the two stimuli presented on each trial. Neurons exhibited selectivity for stimulus rank during learning, but not before or after. The observed behavior is poorly explained by associative or reward mechanisms, and appears more consistent with a mental workspace model in which implied serial order is mapped within a spatial framework. The neural data suggest that posterior parietal cortex supports serial learning by representing information about the ordinal relationship of the stimuli presented during a given trial.

Active transitive inference: When learner control facilitates integrative encoding

A growing body of research indicates that active control of learning improves episodic memory for material experienced during study. It is less clear how active learning impacts the integration of those experiences into flexible, generalizable knowledge. This study uses a novel active transitive inference task to investigate how people learn a relational hierarchy through active selection of premise pairs. Active control improved memory for studied premises as well as transitive inferences involving items that were never experienced together during study. Active learners also exhibited a systematic search preference, generating sequences of overlapping premises that may facilitate relational integration. Critically, however, advantages from active control were not universal: Only participants with higher working memory capacity benefited from the opportunity to select premise pairs during learning. These findings suggest that active control enhances integrative encoding of studied material, but only among individuals with sufficient cognitive resources.

In-Depth Relationships between Emotional Intelligence and Personality Traits in Meditation Practitioners

Objective

Meditation can elicit trait-like changes in psychological and social styles, as well as enhancement of emotional regulatory capacity. We investigated the relation between personality traits and emotional intelligence in meditation practitioners.

Methods

Seventy-two long-term practitioners of mind-body training (MBT) and 62 healthy comparative individuals participated in the study. The participants completed emotional intelligence questionnaires and the Myers-Briggs Type Indicator (MBTI).

Results

The MBT group revealed higher scores on all five emotional intelligence factors than did those in the control group, such as emotional awareness and expression, empathy, emotional thinking, emotional application, and emotional regulation (all p≤0.001). MBT practitioners also had higher scores on the intuition of perceiving function (t=−2.635, p=0.010) and on the feeling of the judging function (t=−3.340, p=0.001) of the MBTI compared with those in the control group. Only the MBT group showed a robust relationship with every factor of emotional intelligence and MBTI-defined intuitive styles, indicating that higher scores of emotional intelligence were related to higher scores for intuition.

Conclusion

Emotional intelligence of meditation practitioners showed notable relationships with some features of personality trait. In-depth associations between emotional intelligence and personality traits would help to foster psychological functions in meditation practitioners.

Language learning in the adult brain: disrupting the dorsolateral prefrontal cortex facilitates word-form learning

Adults do not learn languages as easily as children do. It has been hypothesized that the late-developing prefrontal cortex that supports executive functions competes with procedural learning mechanisms that are important for language learning. To address this hypothesis, we tested whether a temporary neural disruption of the left Dorsolateral Prefrontal Cortex (DLPFC) can improve implicit, procedural learning of word-forms in adults. Young adults were presented with repeating audio-visual sequences of syllables for immediate serial recall in a Hebb repetition learning task that simulates word-form learning. Inhibitory theta-burst Transcranial Magnetic Stimulation was applied to the left DLPFC or to the control site before the Hebb task. The DLPFC-disrupted group showed enhanced learning of the novel phonological sequences relative to the control group. Moreover, learning was negatively correlated with executive functions that rely on the DLPFC in the control group, but not in the DLPFC-disrupted group. The results support the hypothesis that a mature prefrontal cortex competes with implicit learning of word-forms. The findings provide new insight into the competition between brain mechanisms that contribute to language learning in the adult brain.

Mechanisms of automaticity and anticipatory control in fluid intelligence

The constructs of fluid (Gf) and crystalized (Gc) intelligence represent an early attempt to describe the mechanisms of problem solving in the vertebrate brain. Modern neuroscience demonstrates that problem solving involves interplay between the mechanisms of automaticity and anticipatory control, enabling nature's elegant solution to the challenges animals face in their environment. Studies of neural functioning are making clear the primary role of cortical-subcortical interactions in the manifestation of intelligent behavior in humans and other vertebrates. A tridimensional model of intelligent problem solving is explored, wherein the basal ganglia system (BGS) and cerebrocerebellar system (CCS) interact within large scale brain networks. The BGS and CCS work together to enable automaticity to occur. The BGS enables the organism to learn what to do through a powerful instrumental learning system. The BGS also regulates when behavior is released through an inhibitory system which is incredibly sensitive to context. The CCS enables the organism to learn how to perform adaptive behaviors. Internal cerebellar models enable gradual improvements in the quality of behavioral output. The BGS and CCS interact within large scale brain networks, including the dorsal attention network (DAN), ventral attention network (VAN), default mode network (DMN) and frontoparietal network (FPN). The interactions of these systems enable vertebrate organisms to develop a vast array of complex adaptive behaviors. The benefits and importance of developing clinical tests to measure the integrity of these systems is considered.

A Biologically Plausible Action Selection System for Cognitive Architectures: Implications of Basal Ganglia Anatomy for Learning and Decision-Making Models

Several attempts have been made previously to provide a biological grounding for cognitive architectures by relating their components to the computations of specific brain circuits. Often, the architecture's action selection system is identified with the basal ganglia. However, this identification overlooks one of the most important features of the basal ganglia-the existence of a direct and an indirect pathway that compete against each other. This characteristic has important consequences in decision-making tasks, which are brought to light by Parkinson's disease as well as genetic differences in dopamine receptors. This paper shows that a standard model of action selection in a cognitive architecture (ACT-R) cannot replicate any of these findings, details an alternative solution that reconciles action selection in the architecture with the physiology of the basal ganglia, and extends the domain of application of cognitive architectures. The implication of this solution for other architectures and existing models are discussed.

Intranasal midazolam administration enhances amnesic effect in rats

Intranasal (i.n.) administration of midazolam has been shown to be effective and safe for its sedative, anxiolytic, and anticonvulsant effects. However, there has been no investigation on the influence of i.n. administration on midazolam-induced anterograde amnesia. In addition, although the potential of direct drug delivery from the nose to the central nervous system (CNS) has recently become a topic of great interest, it remains unclear whether this pathway is also involved after i.n. midazolam. In this study, we examined the efficacy and the underlying mechanism of i.n. administration compared with intramuscular (i.m.) administration on midazolam-induced amnesia in rats. Equivalent doses of 0.6 mg/kg midazolam were administered via either the i.m or the i.n. route. Anterograde amnesia was assessed by a contextual/cued fear conditioning test. Each animal was conditioned 20 min after drug administration and then tested for a freezing response 24 h later. Midazolam administration by either route produced a similar level of light sedation (minimum spontaneous activity). However, i.n. administration of midazolam induced significantly less freezing behavior compared with i.m. midazolam. Furthermore, in rats with disrupted electrical input from the olfactory epithelium after an olfactotoxicant 3-methylindole administration, the i.n.-mediated enhanced amnesic effect of midazolam was not observed. Our findings indicate that i.n midazolam could probably generate olfactory signals to the brain via benzodiazepine receptors and, compared with i.m. administration, can produce a more significant amnesic effect without alteration in sedative levels. Further clinical studies are warranted.

Dopamine regulates stimulus generalization in the human hippocampus

The ability to generalize previously learned information to novel situations is fundamental for adaptive behavior. However, too wide or too narrow generalization is linked to neuropsychiatric disorders. Previous research suggests that interactions between the dopaminergic system and the hippocampus may play a role in generalization, but whether and how the degree of generalization can be modulated via these pathways is currently unknown. Here, we addressed this question in humans using pharmacology, functional magnetic resonance imaging, and computational modeling. Blocking dopamine D2-receptors (D2R) altered generalization behavior as revealed by an increased kurtosis of the generalization gradient, and a decreased width of model-derived generalization parameters. Moreover, D2R-blockade modulated similarity-based responses in the hippocampus and decreased midbrain-hippocampal connectivity, which in turn correlated with individual differences in generalization. These results suggest that dopaminergic activity in the hippocampus may relate to the degree of generalization and highlight a potential target for treatment.

Atypical Learning in Autism Spectrum Disorders: A Functional Magnetic Resonance Imaging Study of Transitive Inference

OBJECTIVE

To investigate the neural mechanisms underlying impairments in generalizing learning shown by adolescents with autism spectrum disorder (ASD).

METHOD

A total of 21 high-functioning individuals with ASD aged 12 to 18 years, and 23 gender-, IQ-, and age-matched adolescents with typical development (TYP), completed a transitive inference (TI) task implemented using rapid event-related functional magnetic resonance imaging (fMRI). Participants were trained on overlapping pairs in a stimulus hierarchy of colored ovals where A>B>C>D>E>F and then tested on generalizing this training to new stimulus pairings (AF, BD, BE) in a "Big Game." Whole-brain univariate, region of interest, and functional connectivity analyses were used.

RESULTS

During training, the TYP group exhibited increased recruitment of the prefrontal cortex (PFC), whereas the group with ASD showed greater functional connectivity between the PFC and the anterior cingulate cortex (ACC). Both groups recruited the hippocampus and caudate comparably; however, functional connectivity between these regions was positively associated with TI performance for only the group with ASD. During the Big Game, the TYP group showed greater recruitment of the PFC, parietal cortex, and the ACC. Recruitment of these regions increased with age in the group with ASD.

CONCLUSION

During TI, TYP individuals recruited cognitive control-related brain regions implicated in mature problem solving/reasoning including the PFC, parietal cortex, and ACC, whereas the group with ASD showed functional connectivity of the hippocampus and the caudate that was associated with task performance. Failure to reliably engage cognitive control-related brain regions may produce less integrated flexible learning in individuals with ASD unless they are provided with task support that, in essence, provides them with cognitive control; however, this pattern may normalize with age.

Cognitive correlates of psychosis in patients with Parkinson's disease

INTRODUCTION

Psychosis and hallucinations occur in 20-30% of patients with Parkinson's disease (PD). In the current study, we investigate cognitive functions in relation to the occurrence of psychosis in PD patients.

METHODS

We tested three groups of subjects - PD with psychosis, PD without psychosis and healthy controls - on working memory, learning and transitive inference tasks, which are known to assess prefrontal, basal ganglia and hippocampal functions.

RESULTS

In the working memory task, results show that patients with and without psychosis were more impaired than the healthy control group. In the transitive inference task, we did not find any difference among the groups in the learning phase performance. Importantly, PD patients with psychosis were more impaired than both PD patients without psychosis and controls at transitive inference. We also found that the severity of psychotic symptoms in PD patients [as measured by the Unified Parkinson Disease Rating Scale Thought Disorder (UPDRS TD) item] is directly associated with the severity of cognitive impairment [as measured by the mini-mental status exam (MMSE)], sleep disturbance [as measured by the Scales for Outcome in Parkinson Disease (SCOPA) sleep scale] and transitive inference (although the latter did not reach significance).

CONCLUSIONS

Although hypothetical, our data may suggest that the hippocampus is a neural substrate underlying the occurrence of psychosis, sleep disturbance and cognitive impairment in PD patients.

How cognitive theory guides neuroscience

The field of cognitive science studies latent, unobservable cognitive processes that generate observable behaviors. Similarly, cognitive neuroscience attempts to link latent cognitive processes with the neural mechanisms that generate them. Although neural processes are partially observable (with imaging and electrophysiology), it would be a mistake to 'skip' the cognitive level and pursue a purely neuroscientific enterprise to studying behavior. In fact, virtually all of the major advances in understanding the neural basis of behavior over the last century have relied fundamentally on principles of cognition for guiding the appropriate measurements, manipulations, tasks, and interpretations. We provide several examples from the domains of episodic memory, working memory and cognitive control, and decision making in which cognitive theorizing and prior experimentation has been essential in guiding neuroscientific investigations and discoveries.

Different strategies underlying uncertain decision making: higher executive performance is associated with enhanced feedback-related negativity

The aim of the present study was to investigate the role of executive functions (EFs) in different strategies underlying risky decision making. Adult participants from a nonclinical sample were assigned to low or high EF groups based on their performance on EF tasks measuring shifting, updating, and inhibition. ERPs were recorded while participants performed the Balloon Analogue Risk Task (BART). In this task, each balloon pump was associated with either a reward or a balloon pop with unknown probability. The BART behavioral measures did not show between-group differences. However, the feedback-related negativity (FRN) associated with undesirable outcomes was larger in the high EF group than in the low EF group. Since the FRN represents salience prediction error, our results suggest that the high EF group formed internal models that were violated by the outcomes. Thus, we provided ERP evidence for EFs influencing risky decision-making processes.

Age-dependent and coordinated shift in performance between implicit and explicit skill learning

It has been reported recently that while general sequence learning across ages conforms to the typical inverted-U shape pattern, with best performance in early adulthood, surprisingly, the basic ability of picking up in an implicit manner triplets that occur with high vs. low probability in the sequence is best before 12 years of age and it significantly weakens afterwards. Based on these findings, it has been hypothesized that the cognitively controlled processes coming online at around 12 are useful for more targeted explicit learning at the cost of becoming relatively less sensitive to raw probabilities of events. To test this hypothesis, we collected data in a sequence learning task using probabilistic sequences in five age groups from 11 to 39 years of age (N = 288), replicating the original implicit learning paradigm in an explicit task setting where subjects were guided to find repeating sequences. We found that in contrast to the implicit results, performance with the high- vs. low-probability triplets was at the same level in all age groups when subjects sought patterns in the sequence explicitly. Importantly, measurements of explicit knowledge about the identity of the sequences revealed a significant increase in ability to explicitly access the true sequences exactly around the age where the earlier study found the significant drop in ability to learn implicitly raw probabilities. These findings support the conjecture that the gradually increasing involvement of more complex internal models optimizes our skill learning abilities by compensating for the performance loss due to down-weighting the raw probabilities of the sensory input, while expanding our ability to acquire more sophisticated skills.

Implicit sequence learning and working memory: correlated or complicated?

The relationship between implicit/incidental sequence learning and working memory motivated a series of research because it is plausible that higher working memory capacity opens a "larger window" to a sequence, allowing thereby the sequence learning process to be easier. Although the majority of studies found no relationship between implicit sequence learning and working memory capacity, in the past few years several studies have tried to demonstrate the shared or partly shared brain networks underlying these two systems. In order to help the interpretation of these and future results, in this mini-review we suggest the following factors to be taken into consideration before testing the relationship between sequence learning and working memory: 1) the explicitness of the sequence; 2) the method of measuring working memory capacity; 3) online and offline stages of sequence learning; and 4) general skill- and sequence-specific learning.

How glitter relates to gold: similarity-dependent reward prediction errors in the human striatum

Optimal choices benefit from previous learning. However, it is not clear how previously learned stimuli influence behavior to novel but similar stimuli. One possibility is to generalize based on the similarity between learned and current stimuli. Here, we use neuroscientific methods and a novel computational model to inform the question of how stimulus generalization is implemented in the human brain. Behavioral responses during an intradimensional discrimination task showed similarity-dependent generalization. Moreover, a peak shift occurred, i.e., the peak of the behavioral generalization gradient was displaced from the rewarded conditioned stimulus in the direction away from the unrewarded conditioned stimulus. To account for the behavioral responses, we designed a similarity-based reinforcement learning model wherein prediction errors generalize across similar stimuli and update their value. We show that this model predicts a similarity-dependent neural generalization gradient in the striatum as well as changes in responding during extinction. Moreover, across subjects, the width of generalization was negatively correlated with functional connectivity between the striatum and the hippocampus. This result suggests that hippocampus-striatal connections contribute to stimulus-specific value updating by controlling the width of generalization. In summary, our results shed light onto the neurobiology of a fundamental, similarity-dependent learning principle that allows learning the value of stimuli that have never been encountered.

Reinforcement learning in young adults with developmental language impairment

The aim of the study was to examine reinforcement learning (RL) in young adults with developmental language impairment (DLI) within the context of a neurocomputational model of the basal ganglia-dopamine system (Frank, Seeberger, & O'Reilly, 2004). Two groups of young adults, one with DLI and the other without, were recruited. A probabilistic selection task was used to assess how participants implicitly extracted reinforcement history from the environment based on probabilistic positive/negative feedback. The findings showed impaired RL in individuals with DLI, indicating an altered gating function of the striatum in testing. However, they exploited similar learning strategies as comparison participants at the beginning of training, reflecting relatively intact functions of the prefrontal cortex to rapidly update reinforcement information. Within the context of Frank's model, these results can be interpreted as evidence for alterations in the basal ganglia of individuals with DLI.

The relationship between associative learning, transfer generalization, and homocysteine levels in mild cognitive impairment

Previous studies have shown that high total homocysteine levels are associated with Alzheimer's disease (AD) and mild cognitive impairment (MCI). In this study, we test the relationship between cognitive function and total homocysteine levels in healthy subjects (Global Dementia Rating, CDR = 0) and individuals with MCI (CDR = 0.5). We have used a cognitive task that tests learning and generalization of rules, processes that have been previously shown to rely on the integrity of the striatal and hippocampal regions, respectively. We found that total homocysteine levels are higher in MCI individuals than in healthy controls. Unlike what we expected, we found no difference between MCI subjects and healthy controls in learning and generalization. We conducted further analysis after diving MCI subjects in two groups, depending on their Global Deterioration Scale (GDS) scores: individuals with very mild cognitive decline (vMCD, GDS = 2) and mild cognitive decline (MCD, GDS = 3). There was no difference among the two MCI and healthy control groups in learning performance. However, we found that individuals with MCD make more generalization errors than healthy controls and individuals with vMCD. We found no difference in the number of generalization errors between healthy controls and MCI individuals with vMCD. In addition, interestingly, we found that total homocysteine levels correlate positively with generalization errors, but not with learning errors. Our results are in agreement with prior results showing a link between hippocampal function, generalization performance, and total homocysteine levels. Importantly, our study is perhaps among the first to test the relationship between learning (and generalization) of rules and homocysteine levels in healthy controls and individuals with MCI.

Stress modulates reinforcement learning in younger and older adults

Animal research and human neuroimaging studies indicate that stress increases dopamine levels in brain regions involved in reward processing, and stress also appears to increase the attractiveness of addictive drugs. The current study tested the hypothesis that stress increases reward salience, leading to more effective learning about positive than negative outcomes in a probabilistic selection task. Changes to dopamine pathways with age raise the question of whether stress effects on incentive-based learning differ by age. Thus, the present study also examined whether effects of stress on reinforcement learning differed for younger (age 18-34) and older participants (age 65-85). Cold pressor stress was administered to half of the participants in each age group, and salivary cortisol levels were used to confirm biophysiological response to cold stress. After the manipulation, participants completed a probabilistic learning task involving positive and negative feedback. In both younger and older adults, stress enhanced learning about cues that predicted positive outcomes. In addition, during the initial learning phase, stress diminished sensitivity to recent feedback across age groups. These results indicate that stress affects reinforcement learning in both younger and older adults and suggests that stress exerts different effects on specific components of reinforcement learning depending on their neural underpinnings.

High reward makes items easier to remember, but harder to bind to a new temporal context

Learning through reward is central to adaptive behavior. Indeed, items are remembered better if they are experienced while participants expect a reward, and people can deliberately prioritize memory for high- over low-valued items. Do memory advantages for high-valued items only emerge after deliberate prioritization in encoding? Or, do reward-based memory enhancements also apply to unrewarded memory tests and to implicit memory? First, we tested for a high-value memory advantage in unrewarded implicit- and explicit-tests (Experiment 1). Participants first learned high or low-reward values of 36 words, followed by unrewarded lexical decision and free-recall tests. High-value words were judged faster in lexical decision, and more often recalled in free recall. These two memory advantages for high-value words were negatively correlated suggesting at least two mechanisms by which reward value can influence later item-memorability. The ease with which the values were originally acquired explained the negative correlation: people who learned values earlier showed reward effects in implicit memory whereas people who learned values later showed reward effects in explicit memory. We then asked whether a high-value advantage would persist if trained items were linked to a new context (Experiments 2a and 2b). Following the same value training as in Experiment 1, participants learned lists composed of previously trained words mixed with new words, each followed by free recall. Thus, participants had to retrieve words only from the most recent list, irrespective of their values. High- and low-value words were recalled equally, but low-value words were recalled earlier than high-value words and high-value words were more often intruded (proactive interference). Thus, the high-value advantage holds for implicit- and explicit-memory, but comes with a side effect: High-value items are more difficult to relearn in a new context. Similar to emotional arousal, reward value can both enhance and impair memory.

Unconscious relational inference recruits the hippocampus

Relational inference denotes the capacity to encode, flexibly retrieve, and integrate multiple memories to combine past experiences to update knowledge and improve decision-making in new situations. Although relational inference is thought to depend on the hippocampus and consciousness, we now show in young, healthy men that it may occur outside consciousness but still recruits the hippocampus. In temporally distinct and unique subliminal episodes, we presented word pairs that either overlapped ("winter-red", "red-computer") or not. Effects of unconscious relational inference emerged in reaction times recorded during unconscious encoding and in the outcome of decisions made 1 min later at test, when participants judged the semantic relatedness of two supraliminal words. These words were either episodically related through a common word ("winter-computer" related through "red") or unrelated. Hippocampal activity increased during the unconscious encoding of overlapping versus nonoverlapping word pairs and during the unconscious retrieval of episodically related versus unrelated words. Furthermore, hippocampal activity during unconscious encoding predicted the outcome of decisions made at test. Hence, unconscious inference may influence decision-making in new situations.

Learning from experience: event-related potential correlates of reward processing, neural adaptation, and behavioral choice

To behave adaptively, we must learn from the consequences of our actions. Studies using event-related potentials (ERPs) have been informative with respect to the question of how such learning occurs. These studies have revealed a frontocentral negativity termed the feedback-related negativity (FRN) that appears after negative feedback. According to one prominent theory, the FRN tracks the difference between the values of actual and expected outcomes, or reward prediction errors. As such, the FRN provides a tool for studying reward valuation and decision making. We begin this review by examining the neural significance of the FRN. We then examine its functional significance. To understand the cognitive processes that occur when the FRN is generated, we explore variables that influence its appearance and amplitude. Specifically, we evaluate four hypotheses: (1) the FRN encodes a quantitative reward prediction error; (2) the FRN is evoked by outcomes and by stimuli that predict outcomes; (3) the FRN and behavior change with experience; and (4) the system that produces the FRN is maximally engaged by volitional actions.

The human ventromedial prefrontal cortex is critical for transitive inference

We hypothesized that the ventromedial pFC (vmPFC) is critical for making transitive inferences (e.g., the logical operation that if A > B and B > C, then A > C). To test this, participants with focal vmPFC damage, brain-damaged comparison participants, and neurologically normal participants completed a transitive inference task consisting an ordered set of arbitrary patterns. Participants first learned through trial-and-error the relationships of the patterns (e.g., Pattern A > Pattern B, Pattern B > Pattern C). After initial learning, participants were presented with novel pairings, some of which required transitive inference (e.g., Pattern A > Pattern C from the relationship above). We observed that vmPFC damage led to a specific deficit in transitive inference, suggesting that an intact vmPFC is necessary for making normal transitive inferences. Given the usefulness of transitivity in inferring social relationships, this deficit may be one of the basic features of social conduct problems associated with vmPFC damage.

Working-memory load and temporal myopia in dynamic decision making

We examined the role of working memory (WM) in dynamic decision making by having participants perform decision-making tasks under single-task or dual-task conditions. In 2 experiments participants performed dynamic decision-making tasks in which they chose 1 of 2 options on each trial. The decreasing option always gave a larger immediate reward but caused future rewards for both options to decrease. The increasing option always gave a smaller immediate reward but caused future rewards for both options to increase. In each experiment we manipulated the reward structure such that the decreasing option was the optimal choice in 1 condition and the increasing option was the optimal choice in the other condition. Behavioral results indicated that dual-task participants selected the immediately rewarding decreasing option more often, and single-task participants selected the increasing option more often, regardless of which option was optimal. Thus, dual-task participants performed worse on 1 type of task but better on the other type. Modeling results showed that single-task participants' data were most often best fit by a win-stay, lose-shift (WSLS) rule-based model that tracked differences across trials, and dual-task participants' data were most often best fit by a Softmax reinforcement learning model that tracked recency-weighted average rewards for each option. This suggests that manipulating WM load affects the degree to which participants focus on the immediate versus delayed consequences of their actions and whether they employ a rule-based WSLS strategy, but it does not necessarily affect how well people weigh the immediate versus delayed benefits when determining the long-term utility of each option.

The best time to acquire new skills: age-related differences in implicit sequence learning across the human lifespan

Implicit skill learning underlies obtaining not only motor, but also cognitive and social skills through the life of an individual. Yet, the ontogenetic changes in humans' implicit learning abilities have not yet been characterized, and, thus, their role in acquiring new knowledge efficiently during development is unknown. We investigated such learning across the lifespan, between 4 and 85 years of age with an implicit probabilistic sequence learning task, and we found that the difference in implicitly learning high- vs. low-probability events--measured by raw reaction time (RT)--exhibited a rapid decrement around age of 12. Accuracy and z-transformed data showed partially different developmental curves, suggesting a re-evaluation of analysis methods in developmental research. The decrement in raw RT differences supports an extension of the traditional two-stage lifespan skill acquisition model: in addition to a decline above the age 60 reported in earlier studies, sensitivity to raw probabilities and, therefore, acquiring new skills is significantly more effective until early adolescence than later in life. These results suggest that due to developmental changes in early adolescence, implicit skill learning processes undergo a marked shift in weighting raw probabilities vs. more complex interpretations of events, which, with appropriate timing, prove to be an optimal strategy for human skill learning.

Comparison of the performance of DBA/2 and C57BL/6 mice in transitive inference and foreground and background contextual fear conditioning

DBA/2 mice have altered hippocampal structure and perform poorly in several hippocampus-dependent contextual/spatial learning tasks. The performance of this strain in higher cognitive tasks is less studied. Transitive inference is a hippocampus-dependent task that requires an abstraction to be made from prior rules to form a new decision matrix; performance of DBA/2 mice in this task is unknown, whereas contextual fear conditioning is a hippocampus-dependent task in which DBA/2 mice have deficits. The present study compared DBA/2J and C57BL/6J inbred mice in two different contextual fear conditioning paradigms and transitive inference to test whether similar deficits are seen across these hippocampus-dependent tasks. For background fear conditioning, mice were trained with two paired presentations of an auditory conditioned stimulus (CS, 30 seconds, 85 dB white noise) paired with an unconditioned stimulus (US, 2 seconds, 0.57 mA footshock), the context was a continuous background CS. Mice were tested for contextual learning 24 hours later. Foreground fear conditioning differed in that no auditory CS was presented. For transitive inference, separate mice were trained to acquire a series of overlapping odor discrimination problems and tested with novel odor pairings that either did or did not require the use of transitive inference. DBA/2 mice performed significantly worse than the C57BL/6 in both foreground and background fear conditioning and transitive inference. These results demonstrate that the DBA/2 mice have deficits in higher-cognitive processes and suggest that similar substrates may underlie deficits in contextual learning and transitive inference.

Age-based differences in strategy use in choice tasks

We incorporated behavioral and computational modeling techniques to examine age-based differences in strategy use in two four-choice decision-making tasks. Healthy older (aged 60-82 years) and younger adults (aged 18-23 years) performed one of two decision-making tasks that differed in the degree to which rewards for each option depended on the choices made on previous trials. In the choice-independent task rewards for each choice were not affected by the sequence of previous choices that had been made. In contrast, in the choice-dependent task rewards for each option were based on how often each option had been chosen in the past. We compared the fits of a model that assumes the use of a win-stay-lose-shift (WSLS) heuristic to make decisions, to the fits of a reinforcement-learning (RL) model that compared expected reward values for each option to make decisions. Younger adults were best fit by the RL model, while older adults showed significantly more evidence of being best fit by the WSLS heuristic model. This led older adults to perform worse than younger adults in the choice-independent task, but better in the choice-dependent task. These results coincide with previous work in our labs that also found better performance for older adults in choice-dependent tasks (Worthy et al., 2011), and the present results suggest that qualitative age-based differences in the strategies used in choice tasks may underlie older adults' advantage in choice-dependent tasks. We discuss possible factors behind these differences such as neurobiological changes associated with aging, and increased use of heuristics by older adults.

Transitive inference in adults with autism spectrum disorders

Individuals with autism spectrum disorders (ASDs) exhibit intact rote learning with impaired generalization. A transitive inference paradigm, involving training on four sequentially presented stimulus pairs containing overlapping items, with subsequent testing on two novel pairs, was used to investigate this pattern of learning in 27 young adults with ASDs and 31 matched neurotypical individuals (TYPs). On the basis of findings about memory and neuropathology, we hypothesized that individuals with ASDs would use a relational flexibility/conjunctive strategy reliant on an intact hippocampus, versus an associative strength/value transfer strategy requiring intact interactions between the prefrontal cortex and the striatum. Hypotheses were largely confirmed. ASDs demonstrated reduced interference from intervening pairs in early training; only TYPs formed a serial position curve by test; and ASDs exhibited impairments on the novel test pair consisting of end items with intact performance on the inner test pair. However, comparable serial position curves formed for both groups by the end of the first block.

Modulation of the feedback-related negativity by instruction and experience

A great deal of research focuses on how humans and animals learn from trial-and-error interactions with the environment. This research has established the viability of reinforcement learning as a model of behavioral adaptation and neural reward valuation. Error-driven learning is inefficient and dangerous, however. Fortunately, humans learn from nonexperiential sources of information as well. In the present study, we focused on one such form of information, instruction. We recorded event-related potentials as participants performed a probabilistic learning task. In one experiment condition, participants received feedback only about whether their responses were rewarded. In the other condition, they also received instruction about reward probabilities before performing the task. We found that instruction eliminated participants' reliance on feedback as evidenced by their immediate asymptotic performance in the instruction condition. In striking contrast, the feedback-related negativity, an event-related potential component thought to reflect neural reward prediction error, continued to adapt with experience in both conditions. These results show that, whereas instruction may immediately control behavior, certain neural responses must be learned from experience.

Dopaminergic genes predict individual differences in susceptibility to confirmation bias

The striatum is critical for the incremental learning of values associated with behavioral actions. The prefrontal cortex (PFC) represents abstract rules and explicit contingencies to support rapid behavioral adaptation in the absence of cumulative experience. Here we test two alternative models of the interaction between these systems, and individual differences thereof, when human subjects are instructed with prior information about reward contingencies that may or may not be accurate. Behaviorally, subjects are overly influenced by prior instructions, at the expense of learning true reinforcement statistics. Computational analysis found that this pattern of data is best accounted for by a confirmation bias mechanism in which prior beliefs--putatively represented in PFC--influence the learning that occurs in the striatum such that reinforcement statistics are distorted. We assessed genetic variants affecting prefrontal and striatal dopaminergic neurotransmission. A polymorphism in the COMT gene (rs4680), associated with prefrontal dopaminergic function, was predictive of the degree to which participants persisted in responding in accordance with prior instructions even as evidence against their veracity accumulated. Polymorphisms in genes associated with striatal dopamine function (DARPP-32, rs907094, and DRD2, rs6277) were predictive of learning from positive and negative outcomes. Notably, these same variants were predictive of the degree to which such learning was overly inflated or neglected when outcomes are consistent or inconsistent with prior instructions. These findings indicate dissociable neurocomputational and genetic mechanisms by which initial biases are strengthened by experience.

Probabilistic reinforcement learning in adults with autism spectrum disorders

BACKGROUND

Autism spectrum disorders (ASDs) can be conceptualized as disorders of learning, however there have been few experimental studies taking this perspective.

METHODS

We examined the probabilistic reinforcement learning performance of 28 adults with ASDs and 30 typically developing adults on a task requiring learning relationships between three stimulus pairs consisting of Japanese characters with feedback that was valid with different probabilities (80%, 70%, and 60%). Both univariate and Bayesian state-space data analytic methods were employed. Hypotheses were based on the extant literature as well as on neurobiological and computational models of reinforcement learning.

RESULTS

Both groups learned the task after training. However, there were group differences in early learning in the first task block where individuals with ASDs acquired the most frequently accurately reinforced stimulus pair (80%) comparably to typically developing individuals; exhibited poorer acquisition of the less frequently reinforced 70% pair as assessed by state-space learning curves; and outperformed typically developing individuals on the near chance (60%) pair. Individuals with ASDs also demonstrated deficits in using positive feedback to exploit rewarded choices.

CONCLUSIONS

Results support the contention that individuals with ASDs are slower learners. Based on neurobiology and on the results of computational modeling, one interpretation of this pattern of findings is that impairments are related to deficits in flexible updating of reinforcement history as mediated by the orbito-frontal cortex, with spared functioning of the basal ganglia. This hypothesis about the pathophysiology of learning in ASDs can be tested using functional magnetic resonance imaging.

Category learning in the brain

The ability to group items and events into functional categories is a fundamental characteristic of sophisticated thought. It is subserved by plasticity in many neural systems, including neocortical regions (sensory, prefrontal, parietal, and motor cortex), the medial temporal lobe, the basal ganglia, and midbrain dopaminergic systems. These systems interact during category learning. Corticostriatal loops may mediate recursive, bootstrapping interactions between fast reward-gated plasticity in the basal ganglia and slow reward-shaded plasticity in the cortex. This can provide a balance between acquisition of details of experiences and generalization across them. Interactions between the corticostriatal loops can integrate perceptual, response, and feedback-related aspects of the task and mediate the shift from novice to skilled performance. The basal ganglia and medial temporal lobe interact competitively or cooperatively, depending on the demands of the learning task.

Time to go our separate ways: opposite effects of study duration on priming and recognition reveal distinct neural substrates

Amnesic patients have difficulties recognizing when stimuli are repeated, even though their responses to stimuli can change as a function of repetition in indirect tests of memory – a pattern known as priming without recognition. Likewise, experimental manipulations can impair recognition in healthy individuals while leaving priming relatively unaffected, and priming and recognition have been associated with distinct neural correlates in these circumstances. Does this evidence necessarily indicate that priming and recognition rely on distinct brain systems? An alternative explanation is that recognition is merely more sensitive to amnestic insults and experimental manipulations than is priming, and that both priming and recognition are produced by a single brain system. If so, then experimental manipulations would tend to drive priming and recognition in the same direction, albeit to a greater extent for one versus the other in some circumstances. We found evidence to the contrary – that manipulating study duration has opposite effects on priming versus recognition. Studying objects for one-quarter second led to worse recognition than studying objects for 2 s, whereas the opposite was true for priming (greater for one-quarter-second study than two-second study). Furthermore, distinct electrophysiological repetition effects were associated with priming versus recognition. We therefore conclude that study duration had opposite effects on priming and recognition, and on the engagement of implicit versus explicit memory systems. These findings call into question single-process accounts of priming and recognition, and substantiate previous behavioral, neuropsychological, and neuroimaging dissociations between implicit and explicit memory.