Activation of the noradrenergic system facilitates an attentional shift in the rat

Toward a computational role for locus coeruleus/norepinephrine arousal systems

Brain and behavior undergo measurable changes in their underlying state and neuromodulators are thought to contribute to these fluctuations. Why do we undergo such changes, and what function could the underlying neuromodulatory systems perform? Here we examine theoretical answers to these questions with respect to the locus coeruleus/norepinephrine system focusing on peripheral markers for arousal, such as pupil diameter, that are thought to provide a window into brain wide noradrenergic signaling. We explore a computational role for arousal systems in facilitating internal state transitions that facilitate credit assignment and promote accurate perceptions in non-stationary environments. We summarize recent work that supports this idea and highlight open questions as well as alternative views of how arousal affects cognition.

Prediction error determines how memories are organized in the brain

How is new information organized in memory? According to latent state theories, this is determined by the level of surprise, or prediction error, generated by the new information: a small prediction error leads to the updating of existing memory, large prediction error leads to encoding of a new memory. We tested this idea using a protocol in which rats were first conditioned to fear a stimulus paired with shock. The stimulus was then gradually extinguished by progressively reducing the shock intensity until the stimulus was presented alone. Consistent with latent state theories, this gradual extinction protocol (small prediction errors) was better than standard extinction (large prediction errors) in producing long-term suppression of fear responses, and the benefit of gradual extinction was due to updating of the conditioning memory with information about extinction. Thus, prediction error determines how new information is organized in memory, and latent state theories adequately describe the ways in which this occurs.

Prefrontal dynamics and encoding of flexible rule switching

Behavioral flexibility, the ability to adjust behavioral strategies in response to changing environmental contingencies and internal demands, is fundamental to cognitive functions. Despite a large body of pharmacology and lesion studies, the underlying neurophysiological correlates and mechanisms that support flexible rule switching remain elusive. To address this question, we trained mice to distinguish complex sensory cues comprising different perceptual dimensions (set shifting). Endoscopic calcium imaging revealed that medial prefrontal cortex (mPFC) neurons represented multiple task-related events and exhibited pronounced dynamic changes during rule switching. Notably, prominent encoding capacity in the mPFC was associated with switching across, but not within perceptual dimensions. We then showed the involvement of the ascending modulatory input from the locus coeruleus (LC), as inhibiting the LC impaired rule switching behavior and impeded mPFC dynamic processes and encoding. Our results highlight the pivotal role of the mPFC in set shifting processes and demonstrate the profound impact of ascending neuromodulation on shaping prefrontal neural dynamics and behavioral flexibility.

Brainstem fMRI signaling of surprise across different types of deviant stimuli

Detection of deviant stimuli is crucial to orient and adapt our behavior. Previous work shows that deviant stimuli elicit phasic activation of the locus coeruleus (LC), which releases noradrenaline and controls central arousal. However, it is unclear whether the detection of behaviorally relevant deviant stimuli selectively triggers LC responses or other neuromodulatory systems (dopamine, serotonin, and acetylcholine). We combine human functional MRI (fMRI) recordings optimized for brainstem imaging with pupillometry to perform a mapping of deviant-related responses in subcortical structures. Participants have to detect deviant items in a "local-global" paradigm that distinguishes between deviance based on the stimulus probability and the sequence structure. fMRI responses to deviant stimuli are distributed in many cortical areas. Both types of deviance elicit responses in the pupil, LC, and other neuromodulatory systems. Our results reveal that the detection of task-relevant deviant items recruits the same multiple subcortical systems across computationally different types of deviance.

Adaptive learning from outcome contingencies in eating-disorder risk groups

Eating disorders are characterised by altered eating patterns alongside overvaluation of body weight or shape, and have relatively low rates of successful treatment and recovery. Notably, cognitive inflexibility has been implicated in both the development and maintenance of eating disorders, and understanding the reasons for this inflexibility might indicate avenues for treatment development. We therefore investigate one potential cause of this inflexibility: an inability to adjust learning when outcome contingencies change. We recruited (n = 82) three groups of participants: those who had recovered from anorexia nervosa (RA), those who had high levels of eating disorder symptoms but no formal diagnosis (EA), and control participants (HC). They performed a reinforcement learning task (alongside eye-tracking) in which the volatility of wins and losses was independently manipulated. We predicted that both the RA and EA groups would adjust their learning rates less than the control participants. Unexpectedly, the RA group showed elevated adjustment of learning rates for both win and loss outcomes compared to control participants. The RA group also showed increased pupil dilation to stable wins and reduced pupil dilation to stable losses. Their learning rate adjustment was associated with the difference between their pupil dilation to volatile vs. stable wins. In conclusion, we find evidence that learning rate adjustment is unexpectedly higher in those who have recovered from anorexia nervosa, indicating that the relationship between eating disorders and cognitive inflexibility may be complex. Given our findings, investigation of noradrenergic agents may be valuable in the field of eating disorders.

Insight and the selection of ideas

Perhaps it is no accident that insight moments accompany some of humanity's most important discoveries in science, medicine, and art. Here we propose that feelings of insight play a central role in (heuristically) selecting an idea from the stream of consciousness by capturing attention and eliciting a sense of intuitive confidence permitting fast action under uncertainty. The mechanisms underlying this Eureka heuristic are explained within an active inference framework. First, implicit restructuring via Bayesian reduction leads to a higher-order prediction error (i.e., the content of insight). Second, dopaminergic precision-weighting of the prediction error accounts for the intuitive confidence, pleasure, and attentional capture (i.e., the feeling of insight). This insight as precision account is consistent with the phenomenology, accuracy, and neural unfolding of insight, as well as its effects on belief and decision-making. We conclude by reflecting on dangers of the Eureka Heuristic, including the arising and entrenchment of false beliefs and the vulnerability of insights under psychoactive substances and misinformation.

Aging, uncertainty, and decision making-A review

There is a great deal of uncertainty in the world. One common source of uncertainty results from incomplete or missing information about probabilistic outcomes (i.e., outcomes that may occur), which influences how people make decisions. The impact of this type of uncertainty may particularly pronounced for older adults, who, as the primary leaders around the world, make highly impactful decisions with lasting outcomes. This review examines the ways in which uncertainty about probabilistic outcomes is perceived, handled, and represented in the aging brain, with an emphasis on how uncertainty may specifically affect decision making in later life. We describe the role of uncertainty in decision making and aging from four perspectives, including 1) theoretical, 2) self-report, 3) behavioral, and 4) neuroscientific. We report evidence of any age-related differences in uncertainty among these contexts and describe how these changes may affect decision making. We then integrate the findings across the distinct perspectives, followed by a discussion of important future directions for research on aging and uncertainty, including prospection, domain-specificity in risk-taking behaviors, and choice overload.

Theory of visual attention (TVA) applied to rats performing the 5-choice serial reaction time task: differential effects of dopaminergic and noradrenergic manipulations

RATIONALE

Attention is compromised in many psychiatric disorders, including attention-deficit/hyperactivity disorder (ADHD). While dopamine and noradrenaline systems have been implicated in ADHD, their exact role in attentional processing is yet unknown.

OBJECTIVES

We applied the theory of visual attention (TVA) model, adapted from human research, to the rat 5-choice serial reaction time task (5CSRTT) to investigate catecholaminergic modulation of visual attentional processing in healthy subjects of high- and low-attention phenotypes.

METHODS

Rats trained on the standard 5CSRTT and tested with variable stimulus durations were treated systemically with noradrenergic and/or dopaminergic agents (atomoxetine, methylphenidate, amphetamine, phenylephrine and atipamezole). TVA modelling was applied to estimate visual processing speed for correct and incorrect visual perceptual categorisations, independent of motor reaction times, as measures of attentional capacity.

RESULTS

Atomoxetine and phenylephrine decreased response frequencies, including premature responses, increased omissions and slowed responding. In contrast, methylphenidate, amphetamine and atipamezole sped up responding and increased premature responses. Visual processing speed was also affected differentially. Atomoxetine and phenylephrine slowed, whereas methylphenidate and atipamezole sped up, visual processing, both for correct and incorrect categorisations. Amphetamine selectively improved visual processing for correct, though not incorrect, responses in high-attention rats only, possibly reflecting improved attention.

CONCLUSIONS

These data indicate that the application of TVA to the 5CSRTT provides an enhanced sensitivity to capturing attentional effects. Unexpectedly, we found overall slowing effects, including impaired visual processing, following drugs either increasing extracellular noradrenaline (atomoxetine) or activating the α1-adrenoceptor (phenylephrine), while also ameliorating premature responses (impulsivity). In contrast, amphetamine had potential pro-attentional effects by enhancing visual processing, probably due to central dopamine upregulation.

The locus coeruleus mediates behavioral flexibility

Behavioral flexibility is the ability to adjust behavioral strategies in response to changing environmental contingencies. A major hypothesis in the field posits that the activity of neurons in the locus coeruleus (LC) plays an important role in mediating behavioral flexibility. To test this hypothesis, we developed a tactile-based rule-shift detection task in which mice responded to left and right whisker deflections in a context-dependent manner and exhibited varying degrees of switching behavior. Recording spiking activity from optogenetically tagged neurons in the LC at millisecond precision during task performance revealed a prominent graded correlation between baseline LC activity and behavioral flexibility, where higher baseline activity following a rule change was associated with faster behavioral switching to the new rule. Increasing baseline LC activity with optogenetic activation accelerated task switching and improved task performance. Overall, our study provides important evidence to reveal the link between LC activity and behavioral flexibility.

Neuroprotection in late life attention-deficit/hyperactivity disorder: A review of pharmacotherapy and phenotype across the lifespan

For decades, psychostimulants have been the gold standard pharmaceutical treatment for attention-deficit/hyperactivity disorder (ADHD). In the United States, an astounding 9% of all boys and 4% of girls will be prescribed stimulant drugs at some point during their childhood. Recent meta-analyses have revealed that individuals with ADHD have reduced brain volume loss later in life (>60 y.o.) compared to the normal aging brain, which suggests that either ADHD or its treatment may be neuroprotective. Crucially, these neuroprotective effects were significant in brain regions (e.g., hippocampus, amygdala) where severe volume loss is linked to cognitive impairment and Alzheimer's disease. Historically, the ADHD diagnosis and its pharmacotherapy came about nearly simultaneously, making it difficult to evaluate their effects in isolation. Certain evidence suggests that psychostimulants may normalize structural brain changes typically observed in the ADHD brain. If ADHD itself is neuroprotective, perhaps exercising the brain, then psychostimulants may not be recommended across the lifespan. Alternatively, if stimulant drugs are neuroprotective, then this class of medications may warrant further investigation for their therapeutic effects. Here, we take a bottom-up holistic approach to review the psychopharmacology of ADHD in the context of recent models of attention. We suggest that future studies are greatly needed to better appreciate the interactions amongst an ADHD diagnosis, stimulant treatment across the lifespan, and structure-function alterations in the aging brain.

Noradrenaline in Alzheimer's Disease: A New Potential Therapeutic Target

A growing body of evidence demonstrates the important role of the noradrenergic system in the pathogenesis of many neurodegenerative processes, especially Alzheimer’s disease, due to its ability to control glial activation and chemokine production resulting in anti-inflammatory and neuroprotective effects. Noradrenaline involvement in this disease was first proposed after finding deficits of noradrenergic neurons in the locus coeruleus from Alzheimer’s disease patients. Based on this, it has been hypothesized that the early loss of noradrenergic projections and the subsequent reduction of noradrenaline brain levels contribute to cognitive dysfunctions and the progression of neurodegeneration. Several studies have focused on analyzing the role of noradrenaline in the development and progression of Alzheimer’s disease. In this review we summarize some of the most relevant data describing the alterations of the noradrenergic system normally occurring in Alzheimer’s disease as well as experimental studies in which noradrenaline concentration was modified in order to further analyze how these alterations affect the behavior and viability of different nervous cells. The combination of the different studies here presented suggests that the maintenance of adequate noradrenaline levels in the central nervous system constitutes a key factor of the endogenous defense systems that help prevent or delay the development of Alzheimer’s disease. For this reason, the use of noradrenaline modulating drugs is proposed as an interesting alternative therapeutic option for Alzheimer’s disease.

The Hole-Board Test in Mutant Mice

First described by Boissier and Simon in (Ther Recreat J 17:1225-1232, 1962), the hole-board has become a recognized test of anxiety and spatial memory. Benzodiazepines acting at the GABA_A-BZD site increase hole-pokes in rats and mice, indicating a loss in behavioral inhibition concordant with the behavior of mutant mice deficient in the GABA transporter. Hole-poking also depends on arousal mechanisms dependent on dopaminergic transmission, as indicated by drug and null mutant studies. In addition, the behavior is modified in natural and null mutants affecting the cerebellum as well as null mutants affecting neuropeptides, growth factors, cell adhesion, and inflammation. Further research is required to determine convergences between genetic and pharmacological effects.

Context-dependent relationships between locus coeruleus firing patterns and coordinated neural activity in the anterior cingulate cortex

Ascending neuromodulatory projections from the locus coeruleus (LC) affect cortical neural networks via the release of norepinephrine (NE). However, the exact nature of these neuromodulatory effects on neural activity patterns in vivo is not well understood. Here, we show that in awake monkeys, LC activation is associated with changes in coordinated activity patterns in the anterior cingulate cortex (ACC). These relationships, which are largely independent of changes in firing rates of individual ACC neurons, depend on the type of LC activation: ACC pairwise correlations tend to be reduced when ongoing (baseline) LC activity increases but enhanced when external events evoke transient LC responses. Both relationships covary with pupil changes that reflect LC activation and arousal. These results suggest that modulations of information processing that reflect changes in coordinated activity patterns in cortical networks can result partly from ongoing, context-dependent, arousal-related changes in activation of the LC-NE system.

The role of noradrenaline in cognition and cognitive disorders

Many aspects of cognition and behaviour are regulated by noradrenergic projections to the forebrain originating from the locus coeruleus, acting through alpha and beta adrenoreceptors. Loss of these projections is common in neurodegenerative diseases and contributes to their cognitive and behavioural deficits. We review the evidence for a noradrenergic modulation of cognition in its contribution to Alzheimer's disease, Parkinson's disease and other cognitive disorders. We discuss the advances in human imaging and computational methods that quantify the locus coeruleus and its function in humans, and highlight the potential for new noradrenergic treatment strategies.

Noise Correlations for Faster and More Robust Learning

Distributed population codes are ubiquitous in the brain and pose a challenge to downstream neurons that must learn an appropriate readout. Here we explore the possibility that this learning problem is simplified through inductive biases implemented by stimulus-independent noise correlations that constrain learning to task-relevant dimensions. We test this idea in a set of neural networks that learn to perform a perceptual discrimination task. Correlations among similarly tuned units were manipulated independently of an overall population signal-to-noise ratio to test how the format of stored information affects learning. Higher noise correlations among similarly tuned units led to faster and more robust learning, favoring homogenous weights assigned to neurons within a functionally similar pool, and could emerge through Hebbian learning. When multiple discriminations were learned simultaneously, noise correlations across relevant feature dimensions sped learning, whereas those across irrelevant feature dimensions slowed it. Our results complement the existing theory on noise correlations by demonstrating that when such correlations are produced without significant degradation of the signal-to-noise ratio, they can improve the speed of readout learning by constraining it to appropriate dimensions.SIGNIFICANCE STATEMENT Positive noise correlations between similarly tuned neurons theoretically reduce the representational capacity of the brain, yet they are commonly observed, emerge dynamically in complex tasks, and persist even in well-trained animals. Here we show that such correlations, when embedded in a neural population with a fixed signal-to-noise ratio, can improve the speed and robustness with which an appropriate readout is learned. In a simple discrimination task such correlations can emerge naturally through Hebbian learning. In more complex tasks that require multiple discriminations, correlations between neurons that similarly encode the task-relevant feature improve learning by constraining it to the appropriate task dimension.

Locus Coeruleus Norepinephrine in Learned Behavior: Anatomical Modularity and Spatiotemporal Integration in Targets

The locus coeruleus (LC), a small brainstem nucleus, is the primary source of the neuromodulator norepinephrine (NE) in the brain. The LC receives input from widespread brain regions, and projects throughout the forebrain, brainstem, cerebellum, and spinal cord. LC neurons release NE to control arousal, but also in the context of a variety of sensory-motor and behavioral functions. Despite its brain-wide effects, much about the role of LC-NE in behavior and the circuits controlling LC activity is unknown. New evidence suggests that the modular input-output organization of the LC could enable transient, task-specific modulation of distinct brain regions. Future work must further assess whether this spatial modularity coincides with functional differences in LC-NE subpopulations acting at specific times, and how such spatiotemporal specificity might influence learned behaviors. Here, we summarize the state of the field and present new ideas on the role of LC-NE in learned behaviors.

Entorhinal mismatch: A model of self-supervised learning in the hippocampus

The hippocampal formation displays a wide range of physiological responses to different spatial manipulations of the environment. However, very few attempts have been made to identify core computational principles underlying those hippocampal responses. Here, we capitalize on the observation that the entorhinal-hippocampal complex (EHC) forms a closed loop and projects inhibitory signals "countercurrent" to the trisynaptic pathway to build a self-supervised model that learns to reconstruct its own inputs by error backpropagation. The EHC is then abstracted as an autoencoder, with the hidden layers acting as an information bottleneck. With the inputs mimicking the firing activity of lateral and medial entorhinal cells, our model is shown to generate place cells and to respond to environmental manipulations as observed in rodent experiments. Altogether, we propose that the hippocampus builds conjunctive compressed representations of the environment by learning to reconstruct its own entorhinal inputs via gradient descent.

The Locus Coeruleus- Norepinephrine System in Stress and Arousal: Unraveling Historical, Current, and Future Perspectives

Arousal may be understood on a spectrum, with excessive sleepiness, cognitive dysfunction, and inattention on one side, a wakeful state in the middle, and hypervigilance, panic, and psychosis on the other side. However, historically, the concepts of arousal and stress have been challenging to define as measurable experimental variables. Divergent efforts to study these subjects have given rise to several disciplines, including neurobiology, neuroendocrinology, and cognitive neuroscience. We discuss technological advancements that chronologically led to our current understanding of the arousal system, focusing on the multifaceted nucleus locus coeruleus. We share our contemporary perspective and the hypotheses of others in the context of our current technological capabilities and future developments that will be required to move forward in this area of research.

Dynamic decision making and value computations in medial frontal cortex

Dynamic decision making requires an intact medial frontal cortex. Recent work has combined theory and single-neuron measurements in frontal cortex to advance models of decision making. We review behavioral tasks that have been used to study dynamic decision making and algorithmic models of these tasks using reinforcement learning theory. We discuss studies linking neurophysiology and quantitative decision variables. We conclude with hypotheses about the role of other cortical and subcortical structures in dynamic decision making, including ascending neuromodulatory systems.

Transcription Factors Phox2a/2b Upregulate Expression of Noradrenergic and Dopaminergic Phenotypes in Aged Rat Brains

The present study investigated the effects of forced overexpression of Phox2a/2b, two transcription factors, in the locus coeruleus (LC) of aged rats on noradrenergic and dopaminergic phenotypes in brains. Results showed that a significant increase in Phox2a/2b mRNA levels in the LC region was paralleled by marked enhancement in expression of DBH and TH per se. Furthermore, similar increases in TH protein levels were observed in the substantial nigra and striatum, as well as in the hippocampus and frontal cortex. Overexpression of Phox2 genes also significantly increased BrdU-positive cells in the hippocampal dentate gyrus and NE levels in the striatum. Moreover, this manipulation significantly improved the cognition behavior. The in vitro experiments revealed that norepinephrine treatments may increase the transcription of TH gene through the epigenetic action on the TH promoter. The results indicate that Phox2 genes may play an important role in improving the function of the noradrenergic and dopaminergic neurons in aged animals, and regulation of Phox2 gene expression may have therapeutic utility in aging or disorders involving degeneration of noradrenergic neurons.

Oculometric signature of switch into awareness? Pupil size predicts sudden insight whereas microsaccades predict problem-solving via analysis

According to the Gestalt theorists, restructuring is an essential component of insight problem-solving, contributes to the "Aha!" experience, and is similar to the perceptual switch experienced when reinterpreting ambiguous figures. Previous research has demonstrated that pupil diameter increases during the perceptual switch of ambiguous figures, and indexes norepeinephrine functioning mediated by the locus coeruleus. In this study, we investigated if pupil diameter similarly predicts the switch into awareness people experience when solving a problem via insight. Additionally, we explored eye movement dynamics during the same task to investigate if the problem-solving strategies used are linked to specific oculomotor behaviors. In 38 participants, pupil diameter increased about 500 msec prior to solution only in trials for which subjects report having an insight. In contrast, participants increased their microsaccade rate only prior to non-insight solutions. Pupil dilation and microsaccades were not reliably related, but both appear to be robust markers of how people solve problems (with or without insight). The pupil size change seen when people have an "Aha!" moment represents an indicator of the switch into awareness of unconscious processes humans depend upon for insight, and suggests important involvement of norepinephrine, via the locus coeruleus, in sudden insight.

Brain dynamics for confidence-weighted learning

Learning in a changing, uncertain environment is a difficult problem. A popular solution is to predict future observations and then use surprising outcomes to update those predictions. However, humans also have a sense of confidence that characterizes the precision of their predictions. Bayesian models use a confidence-weighting principle to regulate learning: for a given surprise, the update is smaller when the confidence about the prediction was higher. Prior behavioral evidence indicates that human learning adheres to this confidence-weighting principle. Here, we explored the human brain dynamics sub-tending the confidence-weighting of learning using magneto-encephalography (MEG). During our volatile probability learning task, subjects’ confidence reports conformed with Bayesian inference. MEG revealed several stimulus-evoked brain responses whose amplitude reflected surprise, and some of them were further shaped by confidence: surprise amplified the stimulus-evoked response whereas confidence dampened it. Confidence about predictions also modulated several aspects of the brain state: pupil-linked arousal and beta-range (15–30 Hz) oscillations. The brain state in turn modulated specific stimulus-evoked surprise responses following the confidence-weighting principle. Our results thus indicate that there exist, in the human brain, signals reflecting surprise that are dampened by confidence in a way that is appropriate for learning according to Bayesian inference. They also suggest a mechanism for confidence-weighted learning: confidence about predictions would modulate intrinsic properties of the brain state to amplify or dampen surprise responses evoked by discrepant observations.

Learning in a changing and uncertain world is difficult. In this context, facing a discrepancy between my current belief and new observations may reflect random fluctuations (e.g. my commute train is unexpectedly late, but it happens sometimes), if so, I should ignore this discrepancy and not change erratically my belief. However, this discrepancy could also denote a profound change (e.g. the train company changed and is less reliable), in this case, I should promptly revise my current belief. Human learning is adaptive: we change how much we learn from new observations, in particular, we promote flexibility when facing profound changes. A mathematical analysis of the problem shows that we should increase flexibility when the confidence about our current belief is low, which occurs when a change is suspected. Here, I show that human learners entertain rational confidence levels during the learning of changing probabilities. This confidence modulates intrinsic properties of the brain state (oscillatory activity and neuromodulation) which in turn amplifies or reduces, depending on whether confidence is low or high, the neural responses to discrepant observations. This confidence-weighting mechanism could underpin adaptive learning.

Noradrenergic But Not Dopaminergic Neurons Signal Task State Changes and Predict Reengagement After a Failure

The two catecholamines, noradrenaline and dopamine, have been shown to play comparable roles in behavior. Both noradrenergic and dopaminergic neurons respond to cues predicting reward availability and novelty. However, even though both are thought to be involved in motivating actions, their roles in motivation have seldom been directly compared. We therefore examined the activity of putative noradrenergic neurons in the locus coeruleus and putative midbrain dopaminergic neurons in monkeys cued to perform effortful actions for rewards. The activity in both regions correlated with engagement with a presented option. By contrast, only noradrenaline neurons were also (i) predictive of engagement in a subsequent trial following a failure to engage and (ii) more strongly activated in nonrepeated trials, when cues indicated a new task condition. This suggests that while both catecholaminergic neurons are involved in promoting action, noradrenergic neurons are sensitive to task state changes, and their influence on behavior extends beyond the immediately rewarded action.

Catecholaminergic modulation of meta-learning

The remarkable expedience of human learning is thought to be underpinned by meta-learning, whereby slow accumulative learning processes are rapidly adjusted to the current learning environment. To date, the neurobiological implementation of meta-learning remains unclear. A burgeoning literature argues for an important role for the catecholamines dopamine and noradrenaline in meta-learning. Here, we tested the hypothesis that enhancing catecholamine function modulates the ability to optimise a meta-learning parameter (learning rate) as a function of environmental volatility. 102 participants completed a task which required learning in stable phases, where the probability of reinforcement was constant, and volatile phases, where probabilities changed every 10-30 trials. The catecholamine transporter blocker methylphenidate enhanced participants' ability to adapt learning rate: Under methylphenidate, compared with placebo, participants exhibited higher learning rates in volatile relative to stable phases. Furthermore, this effect was significant only with respect to direct learning based on the participants' own experience, there was no significant effect on inferred-value learning where stimulus values had to be inferred. These data demonstrate a causal link between catecholaminergic modulation and the adjustment of the meta-learning parameter learning rate.

Effects of precondition α2-adrenoceptor agents on memory- and anxiety-related processes in the transient cerebral ischemic rats

Neurological evidence for the neuroprotective function of α₂-adrenoceptors in the cerebral ischemia is inconsistent. It is not examined how pretreatment with a single dose of α₂-adrenoceptor agents can affect motor function and anxiety- and memory-related responses in the cerebral ischemic animals. The transient forebrain ischemia model was provided, using a bilateral common carotid arterial occlusion (two-vessel occlusion, 2VO) in male Wistar rats. The 2VO rats impaired motor functions in the Rota-rod and wire grip tests and also decreased the step-through latency and the percentage of time spent on the open arms (%OAT), the percentage of entries into the open arms (%OAE) as well as locomotion in the elevated plus maze (EPM), indicating a memory deficit and anxiety-like behavior. Intraperitoneal single administration of yohimbine (0, 0.001, 0.01, and 0.1 mg/kg) before the 2VO did not alter these parameters while the higher and middle doses of clonidine (0.01 and 0.1 mg/kg) prevented the memory deficit and hypo-locomotion and its middle dose abrogated Rota-rod dysfunction and anxiety-like response. Meanwhile, both drugs did not influence on the measured behaviors in the sham groups by themselves. Moreover, yohimbine (0.001 mg/kg) abolished the beneficial effects of clonidine (0.01 and 0.1 mg/kg) on motor function in the Rota-rod and memory retention and also at its middle dose on the %OAT and locomotion in the 2VO rats. Our findings show a neuroprotective role for clonidine in motor function and memory- and anxiety-related behaviors of 2VO rats and the importance of α₂-adrenoceptors in these processes.

Pupil-linked phasic arousal evoked by violation but not emergence of regularity within rapid sound sequences

The ability to track the statistics of our surroundings is a key computational challenge. A prominent theory proposes that the brain monitors for unexpected uncertainty - events which deviate substantially from model predictions, indicating model failure. Norepinephrine is thought to play a key role in this process by serving as an interrupt signal, initiating model-resetting. However, evidence is from paradigms where participants actively monitored stimulus statistics. To determine whether Norepinephrine routinely reports the statistical structure of our surroundings, even when not behaviourally relevant, we used rapid tone-pip sequences that contained salient pattern-changes associated with abrupt structural violations vs. emergence of regular structure. Phasic pupil dilations (PDR) were monitored to assess Norepinephrine. We reveal a remarkable specificity: When not behaviourally relevant, only abrupt structural violations evoke a PDR. The results demonstrate that Norepinephrine tracks unexpected uncertainty on rapid time scales relevant to sensory signals.

Individual differences in resting-state pupil size: Evidence for association between working memory capacity and pupil size variability

Dynamic non-luminance-mediated changes in pupil diameter have frequently been shown to be a reliable index for the level of arousal, mental effort, and activity in the locus coeruleus, the brainstem's noradrenergic arousal center. While pupillometry has most commonly been used to assess the level of arousal in particular psychological states or the level of engagement in cognitive tasks, some recent studies have found a relationship between average resting-state (i.e. baseline) pupil sizes and individuals' working memory capacity (WMC), indicating that individuals with higher WMC on average have larger pupils than individuals with relatively lower WMC. In the present study, we measured pupil size continuously in 212 participants during rest (i.e. while fixating) and estimated WMC in all participants by administering the Letter-Number Sequencing (LNS) task from WAIS-III. We were unable to replicate the relation between average pupil size and WMC. However, the novel finding was that higher WMC was associated with higher variability in resting-state pupil size. The present results are relevant for the current debate on the role of noradrenergic activity on working memory capacity.

Behavioral correlates of activity of optogenetically identified locus coeruleus noradrenergic neurons in rats performing T-maze tasks

The nucleusLocus Coeruleus (LC) is the major source of forebrain norepinephrine. LC is implicated in arousal, response to novelty, and cognitive functions, including decision-making and behavioral flexibility. One hypothesis is that LC activation promotes rapid shifts in cortical attentional networks following changes in environmental contingencies. Recent recordings further suggest LC is critical for mobilizing resources to deal with challenging situations. In the present study optogenetically identified LC neuronal activity was recorded in rats in a self-paced T-maze. Rats were trained on visual discrimination; then place-reward contingencies were instated. In the session where the animal shifted tasks the first time, the LC firing rate after visual cue onset increased significantly, even as the animal adhered to the previous rule. Firing rate also increased prior to crossing photodetectors that controlled stimulus onset and offset, and this was positively correlated with accelerations, consistent with a role in mobilizing effort. The results contribute to the growing evidence that the noradrenergic LC is essential for behavioral adaptation by promoting cognitive flexibility and mobilizing effort in face of changing environmental contingencies.

Noradrenergic and Cholinergic Modulation of Belief Updating

To make optimal predictions in a dynamic environment, the impact of new observations on existing beliefs-that is, the learning rate-should be guided by ongoing estimates of change and uncertainty. Theoretical work has proposed specific computational roles for various neuromodulatory systems in the control of learning rate, but empirical evidence is still sparse. The aim of the current research was to examine the role of the noradrenergic and cholinergic systems in learning rate regulation. First, we replicated our recent findings that the centroparietal P3 component of the EEG-an index of phasic catecholamine release in the cortex-predicts trial-to-trial variability in learning rate and mediates the effects of surprise and belief uncertainty on learning rate (Study 1, n = 17). Second, we found that pharmacological suppression of either norepinephrine or acetylcholine activity produced baseline-dependent effects on learning rate following nonobvious changes in an outcome-generating process (Study 1). Third, we identified two genes, coding for α2A receptor sensitivity (ADRA2A) and norepinephrine reuptake (NET), as promising targets for future research on the genetic basis of individual differences in learning rate (Study 2, n = 137). Our findings suggest a role for the noradrenergic and cholinergic systems in belief updating and underline the importance of studying interactions between different neuromodulatory systems.

Locus coeruleus activation accelerates perceptual learning

Neural representations of the external world are constructed and updated in a manner that depends on behavioral context. For neocortical networks, this contextual information is relayed by a diverse range of neuromodulatory systems, which govern attention and signal the value of internal state variables such as arousal, motivation, and stress. Neuromodulators enable cortical circuits to differentially process specific stimuli and modify synaptic strengths in order to maintain short- or long-term memory traces of significant perceptual events and behavioral episodes. One of the most important subcortical neuromodulatory systems for attention and arousal is the noradrenergic locus coeruleus. Here we report that the noradrenergic system can enhance behavior in rats performing a self-initiated auditory recognition task, and optogenetic stimulation of noradrenergic locus coeruleus neurons accelerated the rate at which trained rats began correctly responding to a change in reward contingency. Animals successively progressed through distinct behavioral epochs, including periods of perseverance and exploration that occurred much more rapidly when animals received locus coeruleus stimulation. In parallel, we made recordings from primary auditory cortex and found that pairing tones with locus coeruleus stimulation led to a similar set of changes to cortical tuning profiles. Thus both behavioral and neural responses go through phases of adjustment for exploring and exploiting environmental reward contingencies. Furthermore, behavioral engagement does not necessarily recruit optimal locus coeruleus activity.

Curiosity in old age: A possible key to achieving adaptive aging

Curiosity is a fundamental part of human motivation that supports a variety of human intellectual behaviors ranging from early learning in children to scientific discovery. However, there has been little attention paid to the role of curiosity in aging populations. By bringing together broad but sparse neuroscientific and psychological literature on curiosity and related concepts (e.g., novelty seeking in older adults), we propose that curiosity, although it declines with age, plays an important role in maintaining cognitive function, mental health, and physical health in older adults. We identify the dopaminergic reward system and the noradrenergic system as the key brain systems implicated in curiosity processing and discuss how these brain systems contribute to the relationship between curiosity and adaptive aging.

Interrupting behaviour: Minimizing decision costs via temporal commitment and low-level interrupts

Ideal decision-makers should constantly assess all sources of information about opportunities and threats, and be able to redetermine their choices promptly in the face of change. However, perpetual monitoring and reassessment impose inordinate sensing and computational costs, making them impractical for animals and machines alike. The obvious alternative of committing for extended periods of time to limited sensory strategies associated with particular courses of action can be dangerous and wasteful. Here, we explore the intermediate possibility of making provisional temporal commitments whilst admitting interruption based on limited broader observation. We simulate foraging under threat of predation to elucidate the benefits of such a scheme. We relate our results to diseases of distractibility and roving attention, and consider mechanistic substrates such as noradrenergic neuromodulation.

Animals should ideally be able to monitor all relevant aspects of their environment constantly and be ever prepared to alter their course of action in the face of unexpected change. However, the impractically high costs of continual monitoring and deliberation mean that a more realistic strategy is required. Here, we explore a solution in which an animal makes provisional commitments to a temporally-extended action while maintaining the ability to interrupt this behaviour prematurely on the basis of more limited sensing. We demonstrate the benefits of such a scheme through the example of foraging under predation risk, and propose a simple mechanism for implementing interruption. We suggest possible relationships between these results and neural substrates, particularly norepinephrine, and also highlight potential relevance to diseases of distractibility.

Dual contributions of noradrenaline to behavioural flexibility and motivation

INTRODUCTION

While several theories have highlighted the importance of the noradrenergic system for behavioral flexibility, a number of recent studies have also shown a role for noradrenaline in motivation, particularly in effort processing. Here, we designed a novel sequential cost/benefit decision task to test the causal influence of noradrenaline on these two functions in rhesus monkeys.

METHODS

We manipulated noradrenaline using clonidine, an alpha-2 noradrenergic receptor agonist, which reduces central noradrenaline levels and examined how this manipulation influenced performance on the task.

RESULTS

Clonidine had two specific and distinct effects: first, it decreased choice variability, without affecting the cost/benefit trade-off; and second, it reduced force production, without modulating the willingness to work.

CONCLUSIONS

Together, these results support an overarching role for noradrenaline in facing challenging situations in two complementary ways: by modulating behavioral volatility, which would facilitate adaptation depending on the lability of the environment, and by modulating the mobilization of resources to face immediate challenges.

The role of NMDA receptor-dependent activity of noradrenergic neurons in attention, impulsivity and exploratory behaviors

Activity of the brain's noradrenergic (NA) neurons plays a major role in cognitive processes, including the ability to adapt behavior to changing environmental circumstances. Here, we used the NR1^DbhCre transgenic mouse strain to test how NMDA receptor-dependent activity of NA neurons influenced performance in tasks requiring sustained attention, attentional shifting and a trade-off between exploration and exploitation. We found that the loss of NMDA receptors caused irregularity in activity of NA cells in the locus coeruleus and increased the number of neurons with spontaneous burst firing. On a behavioral level, this was associated with increased impulsivity in the go/no-go task and facilitated attention shifts in the attentional set-shifting task. Mutation effects were also observed in the two-armed bandit task, in which mutant mice were generally more likely to employ an exploitative rather than exploratory decision-making strategy. At the same time, the mutation had no appreciable effects on locomotor activity or anxiety-like behavior in the open field. Taken together, these data show that NMDA receptor-dependent activity of brain's NA neurons influences behavioral flexibility.

A computational psychiatry approach identifies how alpha-2A noradrenergic agonist Guanfacine affects feature-based reinforcement learning in the macaque

Noradrenaline is believed to support cognitive flexibility through the alpha 2A noradrenergic receptor (a2A-NAR) acting in prefrontal cortex. Enhanced flexibility has been inferred from improved working memory with the a2A-NA agonist Guanfacine. But it has been unclear whether Guanfacine improves specific attention and learning mechanisms beyond working memory, and whether the drug effects can be formalized computationally to allow single subject predictions. We tested and confirmed these suggestions in a case study with a healthy nonhuman primate performing a feature-based reversal learning task evaluating performance using Bayesian and Reinforcement learning models. In an initial dose-testing phase we found a Guanfacine dose that increased performance accuracy, decreased distractibility and improved learning. In a second experimental phase using only that dose we examined the faster feature-based reversal learning with Guanfacine with single-subject computational modeling. Parameter estimation suggested that improved learning is not accounted for by varying a single reinforcement learning mechanism, but by changing the set of parameter values to higher learning rates and stronger suppression of non-chosen over chosen feature information. These findings provide an important starting point for developing nonhuman primate models to discern the synaptic mechanisms of attention and learning functions within the context of a computational neuropsychiatry framework.

Noradrenaline Improves Behavioral Contrast Sensitivity via the β-Adrenergic Receptor

Noradrenaline (NA) is released from the locus coeruleus in the brainstem to almost the whole brain depending on the physiological state or behavioral context. NA modulates various brain functions including vision, but many questions about the functional role of its effects and mechanisms remain unclear. To explore these matters, we focused on three questions, 1) whether NA improves detectability of a behavior-relevant visual stimulus, 2) which receptor subtypes contribute to the NA effects, and 3) whether the NA effects are specific for visual features such as spatial frequency (SF). We measured contrast sensitivity in rats by a two-alternative forced choice visual detection task and tested the effects of NA receptor blockers in three SF conditions. Propranolol, a β-adrenergic receptor inhibitor, significantly decreased contrast sensitivity, but neither prazosin nor idazoxan, α1- and α2-adrenergic receptor inhibitors, respectively, had an effect. This β blocker effect was observed only at optimal SF. These results indicate that endogenous NA enhances visual detectability depending on stimulus spatial properties via mainly β-adrenergic receptors.

Pharmacological Fingerprints of Contextual Uncertainty

Successful interaction with the environment requires flexible updating of our beliefs about the world. By estimating the likelihood of future events, it is possible to prepare appropriate actions in advance and execute fast, accurate motor responses. According to theoretical proposals, agents track the variability arising from changing environments by computing various forms of uncertainty. Several neuromodulators have been linked to uncertainty signalling, but comprehensive empirical characterisation of their relative contributions to perceptual belief updating, and to the selection of motor responses, is lacking. Here we assess the roles of noradrenaline, acetylcholine, and dopamine within a single, unified computational framework of uncertainty. Using pharmacological interventions in a sample of 128 healthy human volunteers and a hierarchical Bayesian learning model, we characterise the influences of noradrenergic, cholinergic, and dopaminergic receptor antagonism on individual computations of uncertainty during a probabilistic serial reaction time task. We propose that noradrenaline influences learning of uncertain events arising from unexpected changes in the environment. In contrast, acetylcholine balances attribution of uncertainty to chance fluctuations within an environmental context, defined by a stable set of probabilistic associations, or to gross environmental violations following a contextual switch. Dopamine supports the use of uncertainty representations to engender fast, adaptive responses.

Pharmacological interventions and hierarchical Bayesian modelling pinpoint the roles of noradrenaline, acetylcholine, and dopamine in computing different forms of uncertainty and in sensitizing actions to our beliefs about uncertainty.

Interacting with dynamic and ever-changing environments requires frequent updating of our beliefs about the world. By learning the relationships that link events in the current environmental context, it is possible to prepare and execute fast, accurate responses to those events that are predictable. However, the world’s complex dynamics give rise to uncertainty about the relationships that exist between events and uncertainty about how these relationships might change over time. Several neuromodulators have been proposed to signal these different forms of uncertainty, but their relative contributions to updating beliefs and modulating responses have remained elusive. Here we combine a probabilistic reaction time task, pharmacological interventions, and a hierarchical Bayesian learning model to identify the roles of noradrenaline, acetylcholine, and dopamine in individual computations of uncertainty. We propose that noradrenaline modulates learning about the instability of the relationships that link environmental events. Acetylcholine balances the attribution of uncertainty to unexpected events occurring within an environmental context or to gross violations of our expectations following a context change. In contrast, dopamine sensitises our actions to our beliefs about uncertainty.

Catecholaminergic Regulation of Learning Rate in a Dynamic Environment

Adaptive behavior in a changing world requires flexibly adapting one's rate of learning to the rate of environmental change. Recent studies have examined the computational mechanisms by which various environmental factors determine the impact of new outcomes on existing beliefs (i.e., the 'learning rate'). However, the brain mechanisms, and in particular the neuromodulators, involved in this process are still largely unknown. The brain-wide neurophysiological effects of the catecholamines norepinephrine and dopamine on stimulus-evoked cortical responses suggest that the catecholamine systems are well positioned to regulate learning about environmental change, but more direct evidence for a role of this system is scant. Here, we report evidence from a study employing pharmacology, scalp electrophysiology and computational modeling (N = 32) that suggests an important role for catecholamines in learning rate regulation. We found that the P3 component of the EEG-an electrophysiological index of outcome-evoked phasic catecholamine release in the cortex-predicted learning rate, and formally mediated the effect of prediction-error magnitude on learning rate. P3 amplitude also mediated the effects of two computational variables-capturing the unexpectedness of an outcome and the uncertainty of a preexisting belief-on learning rate. Furthermore, a pharmacological manipulation of catecholamine activity affected learning rate following unanticipated task changes, in a way that depended on participants' baseline learning rate. Our findings provide converging evidence for a causal role of the human catecholamine systems in learning-rate regulation as a function of environmental change.

Investigating the genetic basis of attention to facial expressions: the role of the norepinephrine transporter gene

OBJECTIVE

Levels of norepinephrine (NE) in the brain are related to attention ability in animals and risk of attention-deficit hyperactivity disorder in humans. Given the modulation of the norepinephrine transporter (NET) on NE levels in the brain and the link between NE and attention impairment of attention-deficit hyperactivity disorder, it was possible that the NET gene underpinned individual differences in attention processes in healthy populations.

METHODS

To investigate to what extent NET could modulate one's attention orientation to facial expressions, we categorized individuals according to the genotypes of the -182 T/C (rs2242446) polymorphism and measured individuals' attention orientation with the spatial cueing task.

RESULTS

Our results indicated that the -182 T/C polymorphism significantly modulated attention orientation to facial expressions, of which the CC genotype facilitated attention reorientation to the locations where cued faces were previously presented. However, this polymorphism showed no significant effects on the regulations of emotional cues on attention orientation.

CONCLUSION

Our findings suggest that the NET gene modulates the individual difference in attention to facial expressions, which provides new insights into the roles of NE in social interactions.

Differential involvement of brainstem noradrenergic and midbrain dopaminergic nuclei in cognitive control

Several lines of evidence suggest that the lateral prefrontal cortex (PFC), the dorsal anterior cingulate cortex (dACC), the parietal cortex, and the thalamus are central cortical nodes in a network underlying cognitive control. However, the role of catecholamine producing midbrain and brainstem structures has rarely been addressed by functional magnetic resonance imaging (fMRI). We hypothesized differential activation patterns in the ventral tegmental area (VTA)/substantia nigra (SN) and locus coeruleus (LC) with respect to the degree of cognitive control during a Stroop task in healthy subjects. Forty-five healthy subjects were investigated by the manual version of the Stroop task in an event-related fMRI design. We observed significant BOLD activation of both the SN/VTA and LC during the Stroop interference condition (incongruent vs. congruent condition). LC, but not SN/VTA activation significantly correlated with the Stroop interference. Interestingly, a significant linear decrease in BOLD activation during the incongruent condition during the experiment was mainly observed in the fronto-cingulo-striatal network, but not in SN/VTA and LC. Using psychophysiological (PPI) analyses, a significant functional connectivity during cognitive control was observed between SN/VTA and the nigrostriatal/mesolimbic dopaminergic system. For the LC, distinct functional connectivity pattern was observed mainly to the dorsolateral and ventrolateral PFC. Both regions revealed significant functional connectivity to the dACC, parietal and occipital regions. Thus, we demonstrate for the first time that functional activation patterns in the SN/VTA and the LC are modulated by different demands of cognitive control. In addition, these nuclei exhibit distinguishable functional connectivity patterns to cortical brain networks. Hum Brain Mapp 37:2305-2318, 2016. © 2016 Wiley Periodicals, Inc.

For better or worse, or for a change?

The noradrenergic system is intimately related to the autonomic system and is thought to play a key role at the interface between arousal and cognition. The GANE (glutamate amplifies noradrenergic effects) theory proposes a complete account of that role, with an emphasis on the quantitative effect of noradrenaline on stimulus processing. This is in marked contrast to network reset theory, which emphasizes the qualitative effect of noradrenaline of updating the representation of the environment.

Optogenetic silencing of locus coeruleus activity in mice impairs cognitive flexibility in an attentional set-shifting task

The locus coeruleus (LC) is the sole source of noradrenergic projections to the cortex and essential for attention-dependent cognitive processes. In this study we used unilateral optogenetic silencing of the LC in an attentional set-shifting task (ASST) to evaluate the influence of the LC on prefrontal cortex-dependent functions in mice. We expressed the halorhodopsin eNpHR 3.0 to reversibly silence LC activity during task performance, and found that silencing selectively impaired learning of those parts of the ASST that most strongly rely on cognitive flexibility. In particular, extra-dimensional set-shifting (EDS) and reversal learning was impaired, suggesting an involvement of the medial prefrontal cortex (mPFC) and the orbitofrontal cortex. In contrast, those parts of the task that are less dependent on cognitive flexibility, i.e., compound discrimination (CD) and the intra-dimensional shifts (IDS) were not affected. Furthermore, attentional set formation was unaffected by LC silencing. Our results therefore suggest a modulatory influence of the LC on cognitive flexibility, mediated by different frontal networks.

Chronic stress and brain plasticity: Mechanisms underlying adaptive and maladaptive changes and implications for stress-related CNS disorders

Stress responses entail neuroendocrine, autonomic, and behavioral changes to promote effective coping with real or perceived threats to one's safety. While these responses are critical for the survival of the individual, adverse effects of repeated exposure to stress are widely known to have deleterious effects on health. Thus, a considerable effort in the search for treatments to stress-related CNS disorders necessitates unraveling the brain mechanisms responsible for adaptation under acute conditions and their perturbations following chronic stress exposure. This paper is based upon a symposium from the 2014 International Behavioral Neuroscience Meeting, summarizing some recent advances in understanding the effects of stress on adaptive and maladaptive responses subserved by limbic forebrain networks. An important theme highlighted in this review is that the same networks mediating neuroendocrine, autonomic, and behavioral processes during adaptive coping also comprise targets of the effects of repeated stress exposure in the development of maladaptive states. Where possible, reference is made to the similarity of neurobiological substrates and effects observed following repeated exposure to stress in laboratory animals and the clinical features of stress-related disorders in humans.

Phasic activation of individual neurons in the locus ceruleus/subceruleus complex of monkeys reflects rewarded decisions to go but not stop

Neurons in the brainstem nucleus locus ceruleus (LC) often exhibit phasic activation in the context of simple sensory-motor tasks. The functional role of this activation, which leads to the release of norepinephrine throughout the brain, is not yet understood in part because the conditions under which it occurs remain in question. Early studies focused on the relationship of LC phasic activation to salient sensory events, whereas more recent work has emphasized its timing relative to goal-directed behavioral responses, possibly representing the end of a sensory-motor decision process. To better understand the relationship between LC phasic activation and sensory, motor, and decision processing, we recorded spiking activity of neurons in the LC+ (LC and the adjacent, norepinephrine-containing subceruleus nucleus) of monkeys performing a countermanding task. The task required the monkeys to occasionally withhold planned, saccadic eye movements to a visual target. We found that many well isolated LC+ units responded to both the onset of the visual cue instructing the monkey to initiate the saccade and again after saccade onset, even when it was initiated erroneously in the presence of a stop signal. Many of these neurons did not respond to saccades made outside of the task context. In contrast, neither the appearance of the stop signal nor the successful withholding of the saccade elicited an LC+ response. Therefore, LC+ phasic activation encodes sensory and motor events related to decisions to execute, but not withhold, movements, implying a functional role in goal-directed actions, but not necessarily more covert forms of processing.