Simplified dynamics in a model of noradrenergic modulation of cognitive performance

Spotting rare items makes the brain "blink" harder: Evidence from pupillometry

In many visual search tasks (e.g., cancer screening, airport baggage inspections), the most serious search targets occur infrequently. As an ironic side effect, when observers finally encounter important objects (e.g., a weapon in baggage), they often fail to notice them, a phenomenon known as the low-prevalence effect (LPE). Although many studies have investigated LPE search errors, we investigated the attentional consequences of successful rare target detection. Using an attentional blink paradigm, we manipulated how often observers encountered the first serial target (T1), then measured its effects on their ability to detect a following target (T2). Across two experiments, we show that the LPE is more than just an inflated miss rate: When observers successfully detected rare targets, they were less likely to spot subsequent targets. Using pupillometry to index locus-coeruleus (LC) mediated attentional engagement, Experiment 2 confirmed that an LC refractory period mediates the attentional blink (`Nieuwenhuis, Gilzenrat, Holmes, & Cohen, 2005, Journal of Experimental Psychology: General, 134[3], 291-307), and that these effects emerge relatively quickly following T1 onset. Moreover, in both behavioral and pupil analyses, we found that detecting low-prevalence targets exacerbates the LC refractory period. Consequences for theories of the LPE are discussed.

Could LC-NE-Dependent Adjustment of Neural Gain Drive Functional Brain Network Reorganization?

The locus coeruleus-norepinephrine (LC-NE) system is thought to act at synaptic, cellular, microcircuit, and network levels to facilitate cognitive functions through at least two different processes, not mutually exclusive. Accordingly, as a reset signal, the LC-NE system could trigger brain network reorganizations in response to salient information in the environment and/or adjust the neural gain within its target regions to optimize behavioral responses. Here, we provide evidence of the co-occurrence of these two mechanisms at the whole-brain level, in resting-state conditions following a pharmacological stimulation of the LC-NE system. We propose that these two mechanisms are interdependent such that the LC-NE-dependent adjustment of the neural gain inferred from the clustering coefficient could drive functional brain network reorganizations through coherence in the gamma rhythm. Via the temporal dynamic of gamma-range band-limited power, the release of NE could adjust the neural gain, promoting interactions only within the neuronal populations whose amplitude envelopes are correlated, thus making it possible to reorganize neuronal ensembles, functional networks, and ultimately, behavioral responses. Thus, our proposal offers a unified framework integrating the putative influence of the LC-NE system on both local- and long-range adjustments of brain dynamics underlying behavioral flexibility.

The effect of atomoxetine on random and directed exploration in humans

The adaptive regulation of the trade-off between pursuing a known reward (exploitation) and sampling lesser-known options in search of something better (exploration) is critical for optimal performance. Theory and recent empirical work suggest that humans use at least two strategies for solving this dilemma: a directed strategy in which choices are explicitly biased toward information seeking, and a random strategy in which decision noise leads to exploration by chance. Here we examined the hypothesis that random exploration is governed by the neuromodulatory locus coeruleus-norepinephrine system. We administered atomoxetine, a norepinephrine transporter blocker that increases extracellular levels of norepinephrine throughout the cortex, to 22 healthy human participants in a double-blind crossover design. We examined the effect of treatment on performance in a gambling task designed to produce distinct measures of directed exploration and random exploration. In line with our hypothesis we found an effect of atomoxetine on random, but not directed exploration. However, contrary to expectation, atomoxetine reduced rather than increased random exploration. We offer three potential explanations of our findings, involving the non-linear relationship between tonic NE and cognitive performance, the interaction of atomoxetine with other neuromodulators, and the possibility that atomoxetine affected phasic norepinephrine activity more so than tonic norepinephrine activity.

The norepinephrine system shows information-content specific properties during cognitive control - Evidence from EEG and pupillary responses

The ability to exert cognitive control is a major function of the prefrontal cortex, the efficiency of which depends on the phasic release of norepinephrine (NE) at particular time points. However, different aspects of information are simultaneously processed at any given moment. This raises the question of whether the norepinephrine system is also capable of specifically modulating selected aspects of all ongoing information processing, especially when several of those processes are carried out by the same functional neuroanatomical structure at the same time. We examine this question in humans using a flanker paradigm by integrating neurophysiological (EEG) and pupil diameter data using novel signal processing techniques including Residue Iteration Decomposition (RIDE) and source localization. We show that during conflict monitoring, motor response-related processes are more strongly modulated by the NE system than stimulus-related processes or central decision processes between stimulus and response. This was the case even though these processes occurred at the same time point and were mediated by overlapping medial frontal cortical structures. The results indicate that the NE system exerts specific modulatory effects for different informational contents that are simultaneously processed in the medial frontal cortex.

Cognitive control, dynamic salience, and the imperative toward computational accounts of neuromodulatory function

We draw attention to studies indicating that phasic arousal increases interference effects in tasks necessitating the recruitment of cognitive control. We suggest that arousal-biased competition models such as GANE (glutamate amplifies noradrenergic effects) may be able to explain these findings by taking into account dynamic, within-trial changes in the relative salience of task-relevant and task-irrelevant features. However, testing this hypothesis requires a computational model.

Tourette Syndrome: Complementary Insights from Measures of Cognitive Control, Eyeblink Rate, and Pupil Diameter

Some individuals with Tourette syndrome (TS) have severe motoric and vocal tics that interfere with all aspects of their lives, while others have mild tics that pose few problems. We hypothesize that observed tic severity reflects a combination of factors, including the degree to which dopaminergic (DA) and/or noradrenergic (NE) neurotransmitter systems have been affected by the disorder, and the degree to which the child can exert cognitive control to suppress unwanted tics. To explore these hypotheses, we collected behavioral and eyetracking data from 26 patients with TS and 26 controls between ages 7 and 14, both at rest and while they performed a test of cognitive control. To our knowledge, this is the first study to use eyetracking measures in patients with TS. We measured spontaneous eyeblink rate as well as pupil diameter, which have been linked, respectively, to DA and NE levels in the central nervous system. Here, we report a number of key findings that held when we restricted analyses to unmedicated patients. First, patients' accuracy on our test of cognitive control accounted for fully 50% of the variance in parentally reported tic severity. Second, patients exhibited elevated spontaneous eyeblink rates compared to controls, both during task performance and at rest, consistent with heightened DA transmission. Third, although neither task-evoked pupil dilation nor resting pupil diameter differed between TS patients and controls, pupil diameter was positively related to parentally reported anxiety levels in patients, suggesting heightened NE transmission in patients with comorbid anxiety. Thus, with the behavioral and eyetracking data gathered from a single task, we can gather objective data that are related both to tic severity and anxiety levels in pediatric patients with TS, and that likely reflect patients' underlying neurochemical disturbances.

Adaptive gain control during human perceptual choice

Neural systems adapt to background levels of stimulation. Adaptive gain control has been extensively studied in sensory systems but overlooked in decision-theoretic models. Here, we describe evidence for adaptive gain control during the serial integration of decision-relevant information. Human observers judged the average information provided by a rapid stream of visual events (samples). The impact that each sample wielded over choices depended on its consistency with the previous sample, with more consistent or expected samples wielding the greatest influence over choice. This bias was also visible in the encoding of decision information in pupillometric signals and in cortical responses measured with functional neuroimaging. These data can be accounted for with a serial sampling model in which the gain of information processing adapts rapidly to reflect the average of the available evidence.

The Impact of Deliberative Strategy Dissociates ERP Components Related to Conflict Processing vs. Reinforcement Learning

We applied the event-related brain potential (ERP) technique to investigate the involvement of two neuromodulatory systems in learning and decision making: The locus coeruleus-norepinephrine system (NE system) and the mesencephalic dopamine system (DA system). We have previously presented evidence that the N2, a negative deflection in the ERP elicited by task-relevant events that begins approximately 200 ms after onset of the eliciting stimulus and that is sensitive to low-probability events, is a manifestation of cortex-wide noradrenergic modulation recruited to facilitate the processing of unexpected stimuli. Further, we hold that the impact of DA reinforcement learning signals on the anterior cingulate cortex (ACC) produces a component of the ERP called the feedback-related negativity (FRN). The N2 and the FRN share a similar time range, a similar topography, and similar antecedent conditions. We varied factors related to the degree of cognitive deliberation across a series of experiments to dissociate these two ERP components. Across four experiments we varied the demand for a deliberative strategy, from passively watching feedback, to more complex/challenging decision tasks. Consistent with our predictions, the FRN was largest in the experiment involving active learning and smallest in the experiment involving passive learning whereas the N2 exhibited the opposite effect. Within each experiment, when subjects attended to color, the N2 was maximal at frontal-central sites, and when they attended to gender it was maximal over lateral-occipital areas, whereas the topology of the FRN was frontal-central in both task conditions. We conclude that both the DA system and the NE system act in concert when learning from rewards that vary in expectedness, but that the DA system is relatively more exercised when subjects are relatively more engaged by the learning task.

Dimension Reduction and Dynamics of a Spiking Neural Network Model for Decision Making under Neuromodulation()

Previous models of neuromodulation in cortical circuits have used either physiologically based networks of spiking neurons or simplified gain adjustments in low-dimensional connectionist models. Here we reduce a high-dimensional spiking neuronal network model, first to a four-population mean-field model and then to a two-population model. This provides a realistic implementation of neuromodulation in low-dimensional decision-making models, speeds up simulations by three orders of magnitude, and allows bifurcation and phase-plane analyses of the reduced models that illuminate neuromodulatory mechanisms. As modulation of excitation-inhibition varies, the network can move from unaroused states, through optimal performance to impulsive states, and eventually lose inhibition-driven winner-take-all behavior: all are clear outcomes of the bifurcation structure. We illustrate the value of reduced models by a study of the speed-accuracy tradeoff in decision making. The ability of such models to recreate neuromodulatory dynamics of the spiking network will accelerate the pace of future experiments linking behavioral data to cellular neurophysiology.

Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function

An important dimension of cognitive control is the adaptive regulation of the balance between exploitation (pursuing known sources of reward) and exploration (seeking new ones) in response to changes in task utility. Recent studies have suggested that the locus coeruleus-norepinephrine system may play an important role in this function and that pupil diameter can be used to index locus coeruleus activity. On the basis of this, we reasoned that pupil diameter may correlate closely with control state and associated changes in behavior. Specifically, we predicted that increases in baseline pupil diameter would be associated with decreases in task utility and disengagement from the task (exploration), whereas reduced baseline diameter (but increases in task-evoked dilations) would be associated with task engagement (exploitation). Findings in three experiments were consistent with these predictions, suggesting that pupillometry may be useful as an index of both control state and, indirectly, locus coeruleus function.

Optimization of decision making in multilayer networks: the role of locus coeruleus

Previous theoretical work has shown that a single-layer neural network can implement the optimal decision process for simple, two-alternative forced-choice (2AFC) tasks. However, it is likely that the mammalian brain comprises multilayer networks, raising the question of whether and how optimal performance can be approximated in such an architecture. Here, we present theoretical work suggesting that the noradrenergic nucleus locus coeruleus (LC) may help optimize 2AFC decision making in the brain. This is based on the observations that neurons of the LC selectively fire following the presentation of salient stimuli in decision tasks and that the corresponding release of norepinephrine can transiently increase the responsivity, or gain, of cortical processing units. We describe computational simulations that investigate the role of such gain changes in optimizing performance of 2AFC decision making. In the tasks we model, no prior cueing or knowledge of stimulus onset time is assumed. Performance is assessed in terms of the rate of correct responses over time (the reward rate). We first present the results of a single-layer model that accumulates (integrates) sensory input and implements the decision process as a threshold crossing. Gain transients, representing the modulatory effect of the LC, are driven by separate threshold crossings in this layer. We optimize over all free parameters to determine the maximum reward rate achievable by this model and compare it to the maximum reward rate when gain is held fixed. We find that the dynamic gain mechanism yields no improvement in reward for this single-layer model. We then examine a two-layer model, in which competing sensory accumulators in the first layer (capable of implementing the task relevant decision) pass activity to response accumulators in a second layer. Again, we compare a version in which threshold crossing in the first (decision) layer elicits an LC response (and a concomitant increase in gain) with a fixed-gain version of the model. Here, we find that gain transients modeling the LC phasic response yield an improvement in reward rate of 12% to 24%. Furthermore, we show that the timing characteristics of these gain transients agree with observations concerning LC firing patterns reported in recent experimental studies. This provides converging evidence for the hypothesis that the LC optimizes processes underlying 2AFC decision making in multilayer networks.

A reciprocal relationship between bottom-up trace strength and the attentional blink bottleneck: relating the LC-NE and ST(2) models

There is considerable current interest in neural modeling of the attentional blink phenomenon. Two prominent models of this task are the Simultaneous Type Serial Token (ST(2)) model and the Locus Coeruleus-Norepinephrine (LC-NE) model. The former of these generates a broad spectrum of behavioral data, while the latter provides a neurophysiologically detailed account. This paper explores the relationship between these two approaches. Specifically, we consider the spectrum of empirical phenomena that the two models generate, particularly emphasizing the need to generate a reciprocal relationship between bottom-up trace strength and the blink bottleneck. Then we discuss the implications of using ST(2) token mechanisms in the LC-NE setting.

The role of the locus coeruleus in mediating the attentional blink: a neurocomputational theory

The attentional blink refers to the transient impairment in perceiving the 2nd of 2 targets presented in close temporal proximity. In this article, the authors propose a neurobiological mechanism for this effect. The authors extend a recently developed computational model of the potentiating influence of the locus coeruleus-norepinephrine system on information processing and hypothesize that a refractoriness in the function of this system may account for the attentional blink. The model accurately simulates the time course of the attentional blink, including Lag 1 sparing. The theory also offers an account of the close relationship of the attentional blink to the electrophysiological P3 component. The authors report results from two behavioral experiments that support a critical prediction of their theory regarding the time course of Lag 1 sparing. Finally, the relationship between the authors' neurocomputational theory and existing cognitive theories of the attentional blink is discussed.

Decision making, the P3, and the locus coeruleus-norepinephrine system

Psychologists and neuroscientists have had a long-standing interest in the P3, a prominent component of the event-related brain potential. This review aims to integrate knowledge regarding the neural basis of the P3 and to elucidate its functional role in information processing. The authors review evidence suggesting that the P3 reflects phasic activity of the neuromodulatory locus coeruleus-norepinephrine (LC-NE) system. They discuss the P3 literature in the light of empirical findings and a recent theory regarding the information-processing function of the LC-NE phasic response. The theoretical framework emerging from this research synthesis suggests that the P3 reflects the response of the LC-NE system to the outcome of internal decision-making processes and the consequent effects of noradrenergic potentiation of information processing.

The influence of spike rate and stimulus duration on noradrenergic neurons

We model spiking neurons in locus coeruleus (LC), a brain nucleus involved in modulating cognitive performance, and compare with recent experimental data. Extracellular recordings from LC of monkeys performing target detection and selective attention tasks show varying responses dependent on stimuli and performance accuracy. From membrane voltage and ion channel equations, we derive a phase oscillator model for LC neurons. Average spiking probabilities of a pool of cells over many trials are then computed via a probability density formulation. These show that: (1) Post-stimulus response is elevated in populations with lower spike rates; (2) Responses decay exponentially due to noise and variable pre-stimulus spike rates; and (3) Shorter stimuli preferentially cause depressed post-activation spiking. These results allow us to propose mechanisms for the different LC responses observed across behavioral and task conditions, and to make explicit the role of baseline firing rates and the duration of task-related inputs in determining LC response.

Metalearning and neuromodulation

This paper presents a computational theory on the roles of the ascending neuromodulatory systems from the viewpoint that they mediate the global signals that regulate the distributed learning mechanisms in the brain. Based on the review of experimental data and theoretical models, it is proposed that dopamine signals the error in reward prediction, serotonin controls the time scale of reward prediction, noradrenaline controls the randomness in action selection, and acetylcholine controls the speed of memory update. The possible interactions between those neuromodulators and the environment are predicted on the basis of computational theory of metalearning.