Pharmacological Fingerprints of Contextual Uncertainty

Understanding Music and Aging through the lens of Bayesian Inference

Bayesian inference has recently gained momentum in explaining music perception and aging. A fundamental mechanism underlying Bayesian inference is the notion of prediction. This framework could explain how predictions pertaining to musical (melodic, rhythmic, harmonic) structures engender action, emotion, and learning, expanding related concepts of music research, such as musical expectancies, groove, pleasure, and tension. Moreover, a Bayesian perspective of music perception may shed new insights on the beneficial effects of music in aging. Aging could be framed as an optimization process of Bayesian inference. As predictive inferences refine over time, the reliance on consolidated priors increases, while the updating of prior models through Bayesian inference attenuates. This may affect the ability of older adults to estimate uncertainties in their environment, limiting their cognitive and behavioral repertoire. With Bayesian inference as an overarching framework, this review synthesizes the literature on predictive inferences in music and aging, and details how music could be a promising tool in preventive and rehabilitative interventions for older adults through the lens of Bayesian inference.

A computational account of the development and evolution of psychotic symptoms

The mechanisms of psychotic symptoms like hallucinations and delusions are often investigated in fully-formed illness, well after symptoms emerge. These investigations have yielded key insights, but are not well-positioned to reveal the dynamic forces underlying symptom formation itself. Understanding symptom development over time would allow us to identify steps in the pathophysiological process leading to psychosis, shifting the focus of psychiatric intervention from symptom alleviation to prevention. We propose a model for understanding the emergence of psychotic symptoms within the context of an adaptive, developing neural system. We will make the case for a pathophysiological process that begins with cortical hyperexcitability and bottom-up noise transmission, which engenders inappropriate belief formation via aberrant prediction error signaling. We will argue that this bottom-up noise drives learning about the (im)precision of new incoming sensory information because of diminished signal-to-noise ratio, causing an adaptive relative over-reliance on prior beliefs. This over-reliance on priors predisposes to hallucinations and covaries with hallucination severity. An over-reliance on priors may also lead to increased conviction in the beliefs generated by bottom-up noise and drive movement toward conversion to psychosis. We will identify predictions of our model at each stage, examine evidence to support or refute those predictions, and propose experiments that could falsify or help select between alternative elements of the overall model. Nesting computational abnormalities within longitudinal development allows us to account for hidden dynamics among the mechanisms driving symptom formation and to view established symptomatology as a point of equilibrium among competing biological forces.

Acetylcholine modulates the precision of prediction error in the auditory cortex

A fundamental property of sensory systems is their ability to detect novel stimuli in the ambient environment. The auditory brain contains neurons that decrease their response to repetitive sounds but increase their firing rate to novel or deviant stimuli; the difference between both responses is known as stimulus-specific adaptation or neuronal mismatch (nMM). Here, we tested the effect of microiontophoretic applications of ACh on the neuronal responses in the auditory cortex (AC) of anesthetized rats during an auditory oddball paradigm, including cascade controls. Results indicate that ACh modulates the nMM, affecting prediction error responses but not repetition suppression, and this effect is manifested predominantly in infragranular cortical layers. The differential effect of ACh on responses to standards, relative to deviants (in terms of averages and variances), was consistent with the representational sharpening that accompanies an increase in the precision of prediction errors. These findings suggest that ACh plays an important role in modulating prediction error signaling in the AC and gating the access of these signals to higher cognitive levels.

Control dilemma: Evidence of the stability-flexibility trade-off

Cognitive control can be applied flexibly when task goals or environments change (i.e., cognitive flexibility), or stably to pursue a goal in the face of distraction (i.e., cognitive stability). Whether these seemingly contradictory characteristics have an inverse relationship has been controversial, as some studies have suggested a trade-off mechanism between cognitive flexibility and cognitive stability, while others have not found such reciprocal associations. This study investigated the possible antagonistic correlation between cognitive flexibility and stability using a novel version of the flexibility-stability paradigm and the classic cued task switching paradigm. In Experiment 1, we showed that cognitive flexibility was inversely correlated with cognitive stability, as increased distractor proportions were associated with decreased cognitive flexibility and greater cognitive stability. Moreover, cognitive flexibility and stability were regulated by a single control system instead of two independent control mechanisms, as the model selection results indicated that the reciprocally regulated model with one integration parameter outperformed all other models, and the model parameter was inversely linked to cognitive flexibility and stability. We found similar results using the classic cued task switching paradigm in Experiment 2. Therefore, a trade-off between cognitive flexibility and stability was observed from the paradigms used in this study.

Blocking D2/D3 dopamine receptors in male participants increases volatility of beliefs when learning to trust others

The ability to learn about other people is crucial for human social functioning. Dopamine has been proposed to regulate the precision of beliefs, but direct behavioural evidence of this is lacking. In this study, we investigate how a high dose of the D2/D3 dopamine receptor antagonist sulpiride impacts learning about other people's prosocial attitudes in a repeated Trust game. Using a Bayesian model of belief updating, we show that in a sample of 76 male participants sulpiride increases the volatility of beliefs, which leads to higher precision weights on prediction errors. This effect is driven by participants with genetically conferred higher dopamine availability (Taq1a polymorphism) and remains even after controlling for working memory performance. Higher precision weights are reflected in higher reciprocal behaviour in the repeated Trust game but not in single-round Trust games. Our data provide evidence that the D2 receptors are pivotal in regulating prediction error-driven belief updating in a social context.

Aggressive and psychopathic traits are linked to the acquisition of stable but imprecise hostile expectations

Individuals with hostile expectations (HEX) anticipate harm from seemingly neutral or ambiguous stimuli. However, it is unclear how HEX are acquired, and whether specific components of HEX learning can predict antisocial thought, conduct, and personality. In an online sample of healthy young individuals (n = 256, 69% women), we administered a virtual shooting task and applied computational modelling of behaviour to investigate HEX learning and its constellation of correlates. HEX acquisition was best explained by a hierarchical reinforcement learning mechanism. Crucially, we found that individuals with relatively higher self-reported aggressiveness and psychopathy developed stronger and less accurate hostile beliefs as well as larger prediction errors. Moreover, aggressive and psychopathic traits were associated with more temporally stable hostility representations. Our study thus shows that aggressiveness and psychopathy are linked with the acquisition of robust yet imprecise hostile beliefs through reinforcement learning.

Expecting the unexpected: a review of learning under uncertainty across development

Many of our decisions take place under uncertainty. To successfully navigate the environment, individuals need to estimate the degree of uncertainty and adapt their behaviors accordingly by learning from experiences. However, uncertainty is a broad construct and distinct types of uncertainty may differentially influence our learning. We provide a semi-systematic review to illustrate cognitive and neurobiological processes involved in learning under two types of uncertainty: learning in environments with stochastic outcomes, and with volatile outcomes. We specifically reviewed studies (N = 26 studies) that included an adolescent population, because adolescence is a period in life characterized by heightened exploration and learning, as well as heightened uncertainty due to experiencing many new, often social, environments. Until now, reviews have not comprehensively compared learning under distinct types of uncertainties in this age range. Our main findings show that although the overall developmental patterns were mixed, most studies indicate that learning from stochastic outcomes, as indicated by increased accuracy in performance, improved with age. We also found that adolescents tended to have an advantage compared with adults and children when learning from volatile outcomes. We discuss potential mechanisms explaining these age-related differences and conclude by outlining future research directions.

Understanding visual hallucinations: a new synthesis

Despite decades of research, we do not definitively know how people sometimes see things that are not there. Eight models of complex visual hallucinations have been published since 2000, including Deafferentation, Reality Monitoring, Perception and Attention Deficit, Activation, Input, and Modulation, Hodological, Attentional Networks, Active inference, and Thalamocortical Dysrhythmia Default Mode Network Decoupling. Each was derived from different understandings of brain organisation. To reduce this variability, representatives from each research group agreed an integrated Visual Hallucination Framework that is consistent with current theories of veridical and hallucinatory vision. The Framework delineates cognitive systems relevant to hallucinations. It allows a systematic, consistent, investigation of relationships between the phenomenology of visual hallucinations and changes in underpinning cognitive structures. The episodic nature of hallucinations highlights separate factors associated with the onset, persistence, and end of specific hallucinations suggesting a complex relationship between state and trait markers of hallucination risk. In addition to a harmonised interpretation of existing evidence, the Framework highlights new avenues of research, and potentially, new approaches to treating distressing hallucinations.

M1-selective muscarinic allosteric modulation enhances cognitive flexibility and effective salience in nonhuman primates

Acetylcholine (ACh) in cortical neural circuits mediates how selective attention is sustained in the presence of distractors and how flexible cognition adjusts to changing task demands. The cognitive domains of attention and cognitive flexibility might be differentially supported by the M1 muscarinic acetylcholine receptor (mAChR) subtype. Understanding how M1 mAChR mechanisms support these cognitive subdomains is of highest importance for advancing novel drug treatments for conditions with altered attention and reduced cognitive control including Alzheimer's disease or schizophrenia. Here, we tested this question by assessing how the subtype-selective M1 mAChR positive allosteric modulator (PAM) VU0453595 affects visual search and flexible reward learning in nonhuman primates. We found that allosteric potentiation of M1 mAChRs enhanced flexible learning performance by improving extradimensional set shifting, reducing latent inhibition from previously experienced distractors and reducing response perseveration in the absence of adverse side effects. These procognitive effects occurred in the absence of apparent changes of attentional performance during visual search. In contrast, nonselective ACh modulation using the acetylcholinesterase inhibitor (AChEI) donepezil improved attention during visual search at doses that did not alter cognitive flexibility and that already triggered gastrointestinal cholinergic side effects. These findings illustrate that M1 mAChR positive allosteric modulation enhances cognitive flexibility without affecting attentional filtering of distraction, consistent with M1 activity boosting the effective salience of relevant over irrelevant objects specifically during learning. These results suggest that M1 PAMs are versatile compounds for enhancing cognitive flexibility in disorders spanning schizophrenia and Alzheimer's diseases.

Transdiagnostic computations of uncertainty: towards a new lens on intolerance of uncertainty

People radically differ in how they cope with uncertainty. Clinical researchers describe a dispositional characteristic known as "intolerance of uncertainty", a tendency to find uncertainty aversive, reported to be elevated across psychiatric and neurodevelopmental conditions. Concurrently, recent research in computational psychiatry has leveraged theoretical work to characterise individual differences in uncertainty processing. Under this framework, differences in how people estimate different forms of uncertainty can contribute to mental health difficulties. In this review, we briefly outline the concept of intolerance of uncertainty within its clinical context, and we argue that the mechanisms underlying this construct may be further elucidated through modelling how individuals make inferences about uncertainty. We will review the evidence linking psychopathology to different computationally specified forms of uncertainty and consider how these findings might suggest distinct mechanistic routes towards intolerance of uncertainty. We also discuss the implications of this computational approach for behavioural and pharmacological interventions, as well as the importance of different cognitive domains and subjective experiences in studying uncertainty processing.

Modulation of Motor Vigor by Expectation of Reward Probability Trial-by-Trial Is Preserved in Healthy Ageing and Parkinson's Disease Patients

Motor improvements, such as faster movement times or increased velocity, have been associated with reward magnitude in deterministic contexts. Yet whether individual inferences on reward probability influence motor vigor dynamically remains undetermined. We investigated how dynamically inferring volatile action-reward contingencies modulated motor performance trial-by-trial. We conducted three studies that coupled a reversal learning paradigm with a motor sequence task and used a validated hierarchical Bayesian model to fit trial-by-trial data. In Study 1, we tested healthy younger [HYA; 37 (24 females)] and older adults [HOA; 37 (17 females)], and medicated Parkinson's disease (PD) patients [20 (7 females)]. We showed that stronger predictions about the tendency of the action-reward contingency led to faster performance tempo, commensurate with movement time, on a trial-by-trial basis without robustly modulating reaction time (RT). Using Bayesian linear mixed models, we demonstrated a similar invigoration effect on performance tempo in HYA, HOA, and PD, despite HOA and PD being slower than HYA. In Study 2 [HYA, 39 (29 females)], we additionally showed that retrospective subjective inference about credit assignment did not contribute to differences in motor vigor effects. Last, Study 3 [HYA, 33 (27 females)] revealed that explicit beliefs about the reward tendency (confidence ratings) modulated performance tempo trial-by-trial. Our study is the first to reveal that the dynamic updating of beliefs about volatile action-reward contingencies positively biases motor performance through faster tempo. We also provide robust evidence for a preserved sensitivity of motor vigor to inferences about the action-reward mapping in aging and medicated PD.SIGNIFICANCE STATEMENT Navigating a world rich in uncertainty relies on updating beliefs about the probability that our actions lead to reward. Here, we investigated how inferring the action-reward contingencies in a volatile environment modulated motor vigor trial-by-trial in healthy younger and older adults, and in Parkinson's disease (PD) patients on medication. We found an association between trial-by-trial predictions about the tendency of the action-reward contingency and performance tempo, with stronger expectations speeding the movement. We additionally provided evidence for a similar sensitivity of performance tempo to the strength of these predictions in all groups. Thus, dynamic beliefs about the changing relationship between actions and their outcome enhanced motor vigor. This positive bias was not compromised by age or Parkinson's disease.

Brain connectivity modulation by Bayesian surprise in relation to control demand drives cognitive flexibility via control engagement

Human control is characterized by its flexibility and adaptability in response to the conditional probability in the environment. Previous studies have revealed that efficient conflict control could be attained by predicting and adapting to the changing control demand. However, it is unclear whether cognitive flexibility could also be gained by predicting and adapting to the changing control demand. The present study aimed to explore this issue by combining the model-based analyses of behavioral and neuroimaging data with a probabilistic cued task switching paradigm. We demonstrated that the Bayesian surprise (i.e. unsigned precision-weighted prediction error [PE]) negatively modulated the connections among stimulus processing brain regions and control regions/networks. The effect of Bayesian surprise modulation on these connections guided control engagement as reflected by the control PE effect on behavior, which in turn facilitated cognitive flexibility. These results bridge a gap in the literature by illustrating the neural and behavioral effect of control demand prediction (or PE) on cognitive flexibility and offer novel insights into the source of switch cost and the mechanism of cognitive flexibility.

Reaction Time "Mismatch Costs" Change with the Likelihood of Stimulus-Response Compatibility

Dyadic interactions require dynamic correspondence between one's own movements and those of the other agent. This mapping is largely viewed as imitative, with the behavioural hallmark being a reaction-time cost for mismatched actions. Yet the complex motor patterns humans enact together extend beyond direct-matching, varying adaptively between imitation, complementary movements, and counter-imitation. Optimal behaviour requires an agent to predict not only what is likely to be observed but also how that observed action will relate to their own motor planning. In 28 healthy adults, we examined imitation and counter-imitation in a task that varied the likelihood of stimulus-response congruence from highly predictable, to moderately predictable, to unpredictable. To gain mechanistic insights into the statistical learning of stimulus-response compatibility, we compared two computational models of behaviour: (1) a classic fixed learning-rate model (Rescorla-Wagner reinforcement [RW]) and (2) a hierarchical model of perceptual-behavioural processes in which the learning rate adapts to the inferred environmental volatility (hierarchical Gaussian filter [HGF]). Though more complex and hence penalized by model selection, the HGF provided a more likely model of the participants' behaviour. Matching motor responses were only primed (faster) in the most experimentally volatile context. This bias was reversed so that mismatched actions were primed when beliefs about volatility were lower. Inferential statistics indicated that matching responses were only primed in unpredictable contexts when stimuli-response congruence was at 50:50 chance. Outside of these unpredictable blocks the classic stimulus-response compatibility effect was reversed: Incongruent responses were faster than congruent ones. We show that hierarchical Bayesian learning of environmental statistics may underlie response priming during dyadic interactions.

Atypical prediction error learning is associated with prodromal symptoms in individuals at clinical high risk for psychosis

Reductions in the auditory mismatch negativity (MMN) have been well-demonstrated in schizophrenia rendering it a promising biomarker for understanding the emergence of psychosis. According to the predictive coding theory of psychosis, MMN impairments may reflect disturbances in hierarchical information processing driven by maladaptive precision-weighted prediction errors (pwPEs) and enhanced belief updating. We applied a hierarchical Bayesian model of learning to single-trial EEG data from an auditory oddball paradigm in 31 help-seeking antipsychotic-naive high-risk individuals and 23 healthy controls to understand the computational mechanisms underlying the auditory MMN. We found that low-level sensory and high-level volatility pwPE expression correlated with EEG amplitudes, coinciding with the timing of the MMN. Furthermore, we found that prodromal positive symptom severity was associated with increased expression of sensory pwPEs and higher-level belief uncertainty. Our findings provide support for the role of pwPEs in auditory MMN generation, and suggest that increased sensory pwPEs driven by changes in belief uncertainty may render the environment seemingly unpredictable. This may predispose high-risk individuals to delusion-like ideation to explain this experience. These results highlight the value of computational models for understanding the pathophysiological mechanisms of psychosis.

Conditioned Hallucinations and Prior Overweighting Are State-Sensitive Markers of Hallucination Susceptibility

BACKGROUND

Recent advances in computational psychiatry have identified latent cognitive and perceptual states that predispose to psychotic symptoms. Behavioral data fit to Bayesian models have demonstrated an overreliance on priors (i.e., prior overweighting) during perception in select samples of individuals with hallucinations, corresponding to increased precision of prior expectations over incoming sensory evidence. However, the clinical utility of this observation depends on the extent to which it reflects static symptom risk or current symptom state.

METHODS

To determine whether task performance and estimated prior weighting relate to specific elements of symptom expression, a large, heterogeneous, and deeply phenotyped sample of hallucinators (n = 249) and nonhallucinators (n = 209) performed the conditioned hallucination (CH) task.

RESULTS

We found that CH rates predicted stable measures of hallucination status (i.e., peak frequency). However, CH rates were more sensitive to hallucination state (i.e., recent frequency), significantly correlating with recent hallucination severity and driven by heightened reliance on past experiences (priors). To further test the sensitivity of CH rate and prior weighting to symptom severity, a subset of participants with hallucinations (n = 40) performed a repeated-measures version of the CH task. Changes in both CH frequency and prior weighting varied with changes in auditory hallucination frequency on follow-up.

CONCLUSIONS

These results indicate that CH rate and prior overweighting are state markers of hallucination status, potentially useful in tracking disease development and treatment response.

Pupil-Linked Arousal Response Reveals Aberrant Attention Regulation among Children with Autism Spectrum Disorder

Autism spectrum disorder (ASD) is a developmental disorder that is characterized by difficulties with social interaction and interpersonal communication. It has been argued that abnormal attentional function to exogenous stimuli precedes and contributes to the core ASD symptoms. Notably, the locus ceruleus (LC) and its noradrenergic projections throughout the brain modulate attentional function, but the extent to which this locus ceruleus-norepinephrine (LC-NE) system influences attention in individuals with ASD, who frequently exhibit dysregulated alerting and attention orienting, is unknown. We examined dynamic attention control in girls and boys with ASD at rest using the pupil dilation response (PDR) as a noninvasive measure of LC-NE activity. When gender- and age-matched neurotypical participants were passively exposed to an auditory stream, their PDR decreased for recurrent stimuli but remained sensitive to surprising deviant stimuli. In contrast, children with ASD showed less habituation to recurrent stimuli as well as a diminished phasic response to deviants, particularly those containing social information. Their tonic habituation impairment predicts their phasic orienting impairment, and both impairments correlated with the severity of ASD symptom. Because of the fact that these pupil-linked responses are observed when individuals passively listen without any task engagement, our findings imply that the intricate and dynamic attention allocation mechanism, mediated by the subcortical LC-NE system, is impaired in ASD.SIGNIFICANCE STATEMENT Autistic individuals show attentional abnormalities to even simple sensory inputs, which emerge even before formal diagnosis. One possible mechanism behind these abnormalities is a malfunctioning pacemaker of their attention system, the locus ceruleus-norepinephrine pathway. Here we found, according to the pupillary response (a noradrenergic activity proxy), autistic children are hypersensitive to repeated sounds but hyposensitive to surprising deviant sounds when compared with age-matched controls. Importantly, hypersensitivity to repetitions predicts hyposensitivity to deviant sounds, and both abnormalities positively correlate to the severity of autistic symptoms. This provides strong evidence that autistic children have faulty noradrenergic regulation, which might underly the attentional atypicalities previously evidenced in various cortical responses in autistic individuals.

Perceptual pathways to hallucinogenesis

Recent advances in computational psychiatry have provided unique insights into the neural and cognitive underpinnings of psychotic symptoms. In particular, a host of new data has demonstrated the utility of computational frameworks for understanding how hallucinations might arise from alterations in typical perceptual processing. Of particular promise are models based in Bayesian inference that link hallucinatory perceptual experiences to latent states that may drive them. In this piece, we move beyond these findings to ask: how and why do these latent states arise, and how might we take advantage of heterogeneity in that process to develop precision approaches to the treatment of hallucinations? We leverage specific models of Bayesian inference to discuss components that might lead to the development of hallucinations. Using the unifying power of our model, we attempt to place disparate findings in the study of psychotic symptoms within a common framework. Finally, we suggest directions for future elaboration of these models in the service of a more refined psychiatric nosology based on predictable, testable, and ultimately treatable information processing derangements.

Outcome-Locked Cholinergic Signaling Suppresses Prefrontal Encoding of Stimulus Associations

Acetylcholine (ACh) is thought to control arousal, attention, and learning by slowly modulating cortical excitability and plasticity. Recent studies, however, discovered that cholinergic neurons emit precisely timed signals about the aversive outcome at millisecond precision. To investigate the functional relevance of such phasic cholinergic signaling, we manipulated and monitored cholinergic terminals in the mPFC while male mice associated a neutral conditioned stimulus (CS) with mildly aversive eyelid shock (US) over a short temporal gap. Optogenetic inhibition of cholinergic terminals during the US promoted the formation of the CS-US association. On the contrary, optogenetic excitation of cholinergic terminals during the US blocked the association formation. The bidirectional behavioral effects paralleled the corresponding change in the expression of an activity-regulated gene, c-Fos in the mPFC. In contrast, optogenetic inhibition of cholinergic terminals during the CS impaired associative learning, whereas their excitation had marginal effects. In parallel, photometric recording from cholinergic terminals in the mPFC revealed strong innate phasic responses to the US. With subsequent CS-US pairings, cholinergic terminals weakened the responses to the US while developing strong responses to the CS. The across-session changes in the CS- and US-evoked terminal responses were correlated with associative memory strength. These findings suggest that phasic cholinergic signaling in the mPFC exerts opposite effects on aversive associative learning depending on whether it is emitted by the outcome or the cue.SIGNIFICANCE STATEMENT Drugs compensating for the decline of acetylcholine (ACh) are used for cognitive impairment, such as Alzheimer's disease. However, their beneficial effects are limited, demanding new strategies based on better understandings of how ACh modulates cognition. Here, we report that by manipulating ACh signals in the mPFC, we can control the strength of aversive associative learning in mice. Specifically, the suppression of ACh signals during an aversive outcome facilitated its association with a preceding cue. In contrast, the suppression of ACh signals during the cue impaired learning. Considering that this paradigm depends on the brain regions affected in Alzheimer's disease, our findings indicate that precisely timed control of ACh signals is essential to refine ACh-based strategies for cognitive enhancement.

Intact predictive motor sequence learning in autism spectrum disorder

Atypical motor learning has been suggested to underpin the development of motoric challenges (e.g., handwriting difficulties) in autism. Bayesian accounts of autistic cognition propose a mechanistic explanation for differences in the learning process in autism. Specifically, that autistic individuals overweight incoming, at the expense of prior, information and are thus less likely to (a) build stable expectations of upcoming events and (b) react to statistically surprising events. Although Bayesian accounts have been suggested to explain differences in learning across a range of domains, to date, such accounts have not been extended to motor learning. 28 autistic and 35 non-autistic controls (IQ > 70) completed a computerised task in which they learned sequences of actions. On occasional "surprising" trials, an expected action had to be replaced with an unexpected action. Sequence learning was indexed as the reaction time difference between blocks which featured a predictable sequence and those that did not. Surprise-related slowing was indexed as the reaction time difference between surprising and unsurprising trials. No differences in sequence-learning or surprise-related slowing were observed between the groups. Bayesian statistics provided anecdotal to moderate evidence to support the conclusion that sequence learning and surprise-related slowing were comparable between the two groups. We conclude that individuals with autism do not show atypicalities in response to surprising events in the context of motor sequence-learning. These data demand careful consideration of the way in which Bayesian accounts of autism can (and cannot) be extended to the domain of motor learning.

Neurophysiological contributors to advantageous risk-taking: an experimental psychopharmacological investigation

The ability to learn from experience is critical for determining when to take risks and when to play it safe. However, we know little about how within-person state changes, such as an individual's degree of neurophysiological arousal, may impact the ability to learn which risks are most likely to fail vs succeed. To test this, we used a randomized, double-blind, placebo-controlled design to pharmacologically manipulate neurophysiological arousal and assess its causal impact on risk-related learning and performance. Eighty-seven adults (45% female, Mage = 20.1 ± 1.46 years) took either propranolol (n = 42), a beta-adrenergic receptor blocker that attenuates sympathetic nervous system-related signaling, or a placebo (n = 45). Participants then completed the Balloon Emotional Learning Task, a risk-taking task wherein experiential learning is necessary for task success. We found that individuals on propranolol, relative to placebo, earned fewer points on the task, suggesting that they were less effective risk-takers. This effect was mediated by the fact that those on propranolol made less optimal decisions in the final phase of the task on trials with the greatest opportunity for advantageous risk-taking. These findings highlight that neurophysiological arousal supports risk-related learning and, in turn, more advantageous decision-making and optimal behavior under conditions of risk.

The computational neurology of movement under active inference

We propose a computational neurology of movement based on the convergence of theoretical neurobiology and clinical neurology. A significant development in the former is the idea that we can frame brain function as a process of (active) inference, in which the nervous system makes predictions about its sensory data. These predictions depend upon an implicit predictive (generative) model used by the brain. This means neural dynamics can be framed as generating actions to ensure sensations are consistent with these predictions-and adjusting predictions when they are not. We illustrate the significance of this formulation for clinical neurology by simulating a clinical examination of the motor system using an upper limb coordination task. Specifically, we show how tendon reflexes emerge naturally under the right kind of generative model. Through simulated perturbations, pertaining to prior probabilities of this model's variables, we illustrate the emergence of hyperreflexia and pendular reflexes, reminiscent of neurological lesions in the corticospinal tract and cerebellum. We then turn to the computational lesions causing hypokinesia and deficits of coordination. This in silico lesion-deficit analysis provides an opportunity to revisit classic neurological dichotomies (e.g. pyramidal versus extrapyramidal systems) from the perspective of modern approaches to theoretical neurobiology-and our understanding of the neurocomputational architecture of movement control based on first principles.

Sustained Pupil Responses Are Modulated by Predictability of Auditory Sequences

The brain is highly sensitive to auditory regularities and exploits the predictable order of sounds in many situations, from parsing complex auditory scenes, to the acquisition of language. To understand the impact of stimulus predictability on perception, it is important to determine how the detection of predictable structure influences processing and attention. Here, we use pupillometry to gain insight into the effect of sensory regularity on arousal. Pupillometry is a commonly used measure of salience and processing effort, with more perceptually salient or perceptually demanding stimuli consistently associated with larger pupil diameters. In two experiments we tracked human listeners' pupil dynamics while they listened to sequences of 50-ms tone pips of different frequencies. The order of the tone pips was either random, contained deterministic (fully predictable) regularities (experiment 1, n = 18, 11 female) or had a probabilistic regularity structure (experiment 2, n = 20, 17 female). The sequences were rapid, preventing conscious tracking of sequence structure thus allowing us to focus on the automatic extraction of different types of regularities. We hypothesized that if regularity facilitates processing by reducing processing demands, a smaller pupil diameter would be seen in response to regular relative to random patterns. Conversely, if regularity is associated with heightened arousal and attention (i.e., engages processing resources) the opposite pattern would be expected. In both experiments we observed a smaller sustained (tonic) pupil diameter for regular compared with random sequences, consistent with the former hypothesis and confirming that predictability facilitates sequence processing.SIGNIFICANCE STATEMENT The brain is highly sensitive to auditory regularities. To appreciate the impact that the presence of predictability has on perception, we need to better understand how a predictable structure influences processing and attention. We recorded listeners' pupil responses to sequences of tones that followed either a predictable or unpredictable pattern, as the pupil can be used to implicitly tap into these different cognitive processes. We found that the pupil showed a smaller sustained diameter to predictable sequences, indicating that predictability eased processing rather than boosted attention. The findings suggest that the pupil response can be used to study the automatic extraction of regularities, and that the effects are most consistent with predictability helping the listener to efficiently process upcoming sounds.

TAPAS: An Open-Source Software Package for Translational Neuromodeling and Computational Psychiatry

Psychiatry faces fundamental challenges with regard to mechanistically guided differential diagnosis, as well as prediction of clinical trajectories and treatment response of individual patients. This has motivated the genesis of two closely intertwined fields: (i) Translational Neuromodeling (TN), which develops "computational assays" for inferring patient-specific disease processes from neuroimaging, electrophysiological, and behavioral data; and (ii) Computational Psychiatry (CP), with the goal of incorporating computational assays into clinical decision making in everyday practice. In order to serve as objective and reliable tools for clinical routine, computational assays require end-to-end pipelines from raw data (input) to clinically useful information (output). While these are yet to be established in clinical practice, individual components of this general end-to-end pipeline are being developed and made openly available for community use. In this paper, we present the Translational Algorithms for Psychiatry-Advancing Science (TAPAS) software package, an open-source collection of building blocks for computational assays in psychiatry. Collectively, the tools in TAPAS presently cover several important aspects of the desired end-to-end pipeline, including: (i) tailored experimental designs and optimization of measurement strategy prior to data acquisition, (ii) quality control during data acquisition, and (iii) artifact correction, statistical inference, and clinical application after data acquisition. Here, we review the different tools within TAPAS and illustrate how these may help provide a deeper understanding of neural and cognitive mechanisms of disease, with the ultimate goal of establishing automatized pipelines for predictions about individual patients. We hope that the openly available tools in TAPAS will contribute to the further development of TN/CP and facilitate the translation of advances in computational neuroscience into clinically relevant computational assays.

Excitatory-inhibitory tone shapes decision strategies in a hierarchical neural network model of multi-attribute choice

We are constantly faced with decisions between alternatives defined by multiple attributes, necessitating an evaluation and integration of different information sources. Time-varying signals in multiple brain areas are implicated in decision-making; but we lack a rigorous biophysical description of how basic circuit properties, such as excitatory-inhibitory (E/I) tone and cascading nonlinearities, shape attribute processing and choice behavior. Furthermore, how such properties govern choice performance under varying levels of environmental uncertainty is unknown. We investigated two-attribute, two-alternative decision-making in a dynamical, cascading nonlinear neural network with three layers: an input layer encoding choice alternative attribute values; an intermediate layer of modules processing separate attributes; and a final layer producing the decision. Depending on intermediate layer E/I tone, the network displays distinct regimes characterized by linear (I), convex (II) or concave (III) choice indifference curves. In regimes I and II, each option's attribute information is additively integrated. In regime III, time-varying nonlinear operations amplify the separation between offer distributions by selectively attending to the attribute with the larger differences in input values. At low environmental uncertainty, a linear combination most consistently selects higher valued alternatives. However, at high environmental uncertainty, regime III is more likely than a linear operation to select alternatives with higher value. Furthermore, there are conditions where readout from the intermediate layer could be experimentally indistinguishable from the final layer. Finally, these principles are used to examine multi-attribute decisions in systems with reduced inhibitory tone, leading to predictions of different choice patterns and overall performance between those with restrictions on inhibitory tone and neurotypicals.

The Computational, Pharmacological, and Physiological Determinants of Sensory Learning under Uncertainty

The ability to represent and respond to uncertainty is fundamental to human cognition and decision-making. Noradrenaline (NA) is hypothesized to play a key role in coordinating the sensory, learning, and physiological states necessary to adapt to a changing world, but direct evidence for this is lacking in humans. Here, we tested the effects of attenuating noradrenergic neurotransmission on learning under uncertainty. We probed the effects of the β-adrenergic receptor antagonist propranolol (40 mg) using a between-subjects, double-blind, placebo-controlled design. Participants performed a probabilistic associative learning task, and we employed a hierarchical learning model to formally quantify prediction errors about cue-outcome contingencies and changes in these associations over time (volatility). Both unexpectedness and noise slowed down reaction times, but propranolol augmented the interaction between these main effects such that behavior was influenced more by prior expectations when uncertainty was high. Computationally, this was driven by a reduction in learning rates, with people slower to update their beliefs in the face of new information. Attenuating the global effects of NA also eliminated the phasic effects of prediction error and volatility on pupil size, consistent with slower belief updating. Finally, estimates of environmental volatility were predicted by baseline cardiac measures in all participants. Our results demonstrate that NA underpins behavioral and computational responses to uncertainty. These findings have important implications for understanding the impact of uncertainty on human biology and cognition.

Between Action and Emotional Survival During the COVID-19 era: Sensorimotor Pathways as Control Systems of Transdiagnostic Anxiety-Related Intolerance to Uncertainty

Objectives: The COVID-19 pandemic and aligned social and physical distancing regulations increase the sense of uncertainty, intensifying the risk for psychopathology globally. Anxiety disorders are associated with intolerance to uncertainty. In this review we describe brain circuits and sensorimotor pathways involved in human reactions to uncertainty. We present the healthy mode of coping with uncertainty and discuss deviations from this mode. Methods: Literature search of PubMed and Google Scholar. Results: As manifestation of anxiety disorders includes peripheral reactions and negative cognitions, we suggest an integrative model of threat cognitions modulated by sensorimotor regions: "The Sensorimotor-Cognitive-Integration-Circuit." The model emphasizes autonomic nervous system coupling with the cortex, addressing peripheral anxious reactions to uncertainty, pathways connecting cortical regions and cost-reward evaluation circuits to sensorimotor regions, filtered by the amygdala and basal ganglia. Of special interest are the ascending and descending tracts for sensory-motor crosstalk in healthy and pathological conditions. We include arguments regarding uncertainty in anxiety reactions to the pandemic and derive from our model treatment suggestions which are supported by scientific evidence. Our model is based on systematic control theories and emphasizes the role of goal conflict regulation in health and pathology. We also address anxiety reactions as a spectrum ranging from healthy to pathological coping with uncertainty, and present this spectrum as a transdiagnostic entity in accordance with recent claims and models. Conclusions: The human need for controllability and predictability suggests that anxiety disorders reactive to the pandemic's uncertainties reflect pathological disorganization of top-down bottom-up signaling and neural noise resulting from non-pathological human needs for coherence in life.

Cholinergic and dopaminergic effects on prediction error and uncertainty responses during sensory associative learning

Navigating the physical world requires learning probabilistic associations between sensory events and their change in time (volatility). Bayesian accounts of this learning process rest on hierarchical prediction errors (PEs) that are weighted by estimates of uncertainty (or its inverse, precision). In a previous fMRI study we found that low-level precision-weighted PEs about visual outcomes (that update beliefs about associations) activated the putative dopaminergic midbrain; by contrast, precision-weighted PEs about cue-outcome associations (that update beliefs about volatility) activated the cholinergic basal forebrain. These findings suggested selective dopaminergic and cholinergic influences on precision-weighted PEs at different hierarchical levels. Here, we tested this hypothesis, repeating our fMRI study under pharmacological manipulations in healthy participants. Specifically, we performed two pharmacological fMRI studies with a between-subject double-blind placebo-controlled design: study 1 used antagonists of dopaminergic (amisulpride) and muscarinic (biperiden) receptors, study 2 used enhancing drugs of dopaminergic (levodopa) and cholinergic (galantamine) modulation. Pooled across all pharmacological conditions of study 1 and study 2, respectively, we found that low-level precision-weighted PEs activated the midbrain and high-level precision-weighted PEs the basal forebrain as in our previous study. However, we found pharmacological effects on brain activity associated with these computational quantities only when splitting the precision-weighted PEs into their PE and precision components: in a brainstem region putatively containing cholinergic (pedunculopontine and laterodorsal tegmental) nuclei, biperiden (compared to placebo) enhanced low-level PE responses and attenuated high-level PE activity, while amisulpride reduced high-level PE responses. Additionally, in the putative dopaminergic midbrain, galantamine compared to placebo enhanced low-level PE responses (in a body-weight dependent manner) and amisulpride enhanced high-level precision activity. Task behaviour was not affected by any of the drugs. These results do not support our hypothesis of a clear-cut dichotomy between different hierarchical inference levels and neurotransmitter systems, but suggest a more complex interaction between these neuromodulatory systems and hierarchical Bayesian quantities. However, our present results may have been affected by confounds inherent to pharmacological fMRI. We discuss these confounds and outline improved experimental tests for the future.

Glutamatergic Contribution to Probabilistic Reasoning and Jumping to Conclusions in Schizophrenia: A Double-Blind, Randomized Experimental Trial

BACKGROUND

Impaired probabilistic reasoning and the jumping-to-conclusions reasoning bias are hallmark features of schizophrenia (SCZ), yet the neuropharmacological basis of these deficits remains unclear. Here we tested the hypothesis that glutamatergic neurotransmission specifically contributes to jumping to conclusions and impaired probabilistic reasoning in SCZ.

METHODS

A total of 192 healthy participants received either NMDA receptor agonists/antagonists (D-cycloserine/dextromethorphan), dopamine type 2 receptor agonists/antagonists (bromocriptine/haloperidol), or placebo in a randomized, double-blind, between-subjects design. In addition, we tested 32 healthy control participants matched to 32 psychotic inpatients with SCZ-a state associated with compromised probabilistic reasoning due to reduced glutamatergic neurotransmission. All experiments employed two versions of a probabilistic reasoning (beads) task, which required participants to either sample individual amounts of sensory information to infer correct decisions or provide explicit probability estimates for presented sensory information. Our task instantiations assessed both information sampling and explicit probability estimates in different probabilistic contexts (easy vs. difficult conditions) and changing sensory information through random transitions among easy, difficult, and ambiguous trial types.

RESULTS

Following administration of D-cycloserine, haloperidol, and bromocriptine, healthy participants displayed data-gathering behavior that was normal compared with placebo and was adequate in the context of all employed task conditions and trial level difficulties. However, healthy participants receiving dextromethorphan displayed a jumping-to-conclusions bias, abnormally increased probability estimates, and overweighting of sensory information. These effects were mirrored in patients with SCZ performing the same versions of the beads task.

CONCLUSIONS

Our findings provide novel neuropharmacological evidence linking reduced glutamatergic neurotransmission to impaired information sampling and to disrupted probabilistic reasoning, namely to overweighting of sensory evidence, in patients with SCZ.

State anxiety biases estimates of uncertainty and impairs reward learning in volatile environments

Clinical and subclinical (trait) anxiety impairs decision making and interferes with learning. Less understood are the effects of temporary anxious states on learning and decision making in healthy populations, and whether these can serve as a model for clinical anxiety. Here we test whether anxious states in healthy individuals elicit a pattern of aberrant behavioural, neural, and physiological responses comparable with those found in anxiety disorders-particularly when processing uncertainty in unstable environments. In our study, both a state anxious and a control group learned probabilistic stimulus-outcome mappings in a volatile task environment while we recorded their electrophysiological (EEG) signals. By using a hierarchical Bayesian model of inference and learning, we assessed the effect of state anxiety on Bayesian belief updating with a focus on uncertainty estimates. State anxiety was associated with an underestimation of environmental uncertainty, and informational uncertainty about the reward tendency. Anxious individuals' beliefs about reward contingencies were more precise (had smaller uncertainty) and thus more resistant to updating, ultimately leading to impaired reward-based learning. State anxiety was also associated with greater uncertainty about volatility. We interpret this pattern as evidence that state anxious individuals are less tolerant to informational uncertainty about the contingencies governing their environment and more willing to be uncertain about the level of stability of the world itself. Further, we tracked the neural representation of belief update signals in the trial-by-trial EEG amplitudes. In control participants, lower-level precision-weighted prediction errors (pwPEs) about reward tendencies were represented in the ERP signals across central and parietal electrodes peaking at 496 ms, overlapping with the late P300 in classical ERP analysis. The state anxiety group did not exhibit a significant representation of low-level pwPEs, and there were no significant differences between the groups. Smaller variance in low-level pwPE about reward tendencies in state anxiety could partially account for the null results. Expanding previous computational work on trait anxiety, our findings establish that temporary anxious states in healthy individuals impair reward-based learning in volatile environments, primarily through changes in uncertainty estimates, which play a central role in current Bayesian accounts of perceptual inference and learning.

Neural arbitration between social and individual learning systems

Decision making requires integrating knowledge gathered from personal experiences with advice from others. The neural underpinnings of the process of arbitrating between information sources has not been fully elucidated. In this study, we formalized arbitration as the relative precision of predictions, afforded by each learning system, using hierarchical Bayesian modeling. In a probabilistic learning task, participants predicted the outcome of a lottery using recommendations from a more informed advisor and/or self-sampled outcomes. Decision confidence, as measured by the number of points participants wagered on their predictions, varied with our definition of arbitration as a ratio of precisions. Functional neuroimaging demonstrated that arbitration signals were independent of decision confidence and involved modality-specific brain regions. Arbitrating in favor of self-gathered information activated the dorsolateral prefrontal cortex and the midbrain, whereas arbitrating in favor of social information engaged the ventromedial prefrontal cortex and the amygdala. These findings indicate that relative precision captures arbitration between social and individual learning systems at both behavioral and neural levels.

Enhancing Psychosis Risk Prediction Through Computational Cognitive Neuroscience

Research suggests that early identification and intervention with individuals at clinical high risk (CHR) for psychosis may be able to improve the course of illness. The first generation of studies suggested that the identification of CHR through the use of specialized interviews evaluating attenuated psychosis symptoms is a promising strategy for exploring mechanisms associated with illness progression, etiology, and identifying new treatment targets. The next generation of research on psychosis risk must address two major limitations: (1) interview methods have limited specificity, as recent estimates indicate that only 15%-30% of individuals identified as CHR convert to psychosis and (2) the expertise needed to make CHR diagnosis is only accessible in a handful of academic centers. Here, we introduce a new approach to CHR assessment that has the potential to increase accessibility and positive predictive value. Recent advances in clinical and computational cognitive neuroscience have generated new behavioral measures that assay the cognitive mechanisms and neural systems that underlie the positive, negative, and disorganization symptoms that are characteristic of psychotic disorders. We hypothesize that measures tied to symptom generation will lead to enhanced sensitivity and specificity relative to interview methods and the cognitive intermediate phenotype measures that have been studied to date that are typically indicators of trait vulnerability and, therefore, have a high false positive rate for conversion to psychosis. These new behavioral measures have the potential to be implemented on the internet and at minimal expense, thereby increasing accessibility of assessments.

Dopamine modulates subcortical responses to surprising sounds

Dopamine guides behavior and learning through pleasure, according to classic understanding. Dopaminergic neurons are traditionally thought to signal positive or negative prediction errors (PEs) when reward expectations are, respectively, exceeded or not matched. These signed PEs are quite different from the unsigned PEs, which report surprise during sensory processing. But mounting theoretical accounts from the predictive processing framework postulate that dopamine, as a neuromodulator, could potentially regulate the postsynaptic gain of sensory neurons, thereby scaling unsigned PEs according to their expected precision or confidence. Despite ample modeling work, the physiological effects of dopamine on the processing of surprising sensory information are yet to be addressed experimentally. In this study, we tested how dopamine modulates midbrain processing of unexpected tones. We recorded extracellular responses from the rat inferior colliculus to oddball and cascade sequences, before, during, and after the microiontophoretic application of dopamine or eticlopride (a D₂-like receptor antagonist). Results demonstrate that dopamine reduces the net neuronal responsiveness exclusively to unexpected sensory input without significantly altering the processing of expected input. We conclude that dopaminergic projections from the thalamic subparafascicular nucleus to the inferior colliculus could encode the expected precision of unsigned PEs, attenuating via D₂-like receptors the postsynaptic gain of sensory inputs forwarded by the auditory midbrain neurons. This direct dopaminergic modulation of sensory PE signaling has profound implications for both the predictive coding framework and the understanding of dopamine function.

Information about unexpected stimuli is encoded in the form of prediction error signals. The earliest prediction error signals identified in the auditory brain emerge subcortically in the inferior colliculus. This study reveals the essential role of dopamine in encoding the precision of prediction errors at the auditory midbrain.

Brain dynamics for confidence-weighted learning

Learning in a changing, uncertain environment is a difficult problem. A popular solution is to predict future observations and then use surprising outcomes to update those predictions. However, humans also have a sense of confidence that characterizes the precision of their predictions. Bayesian models use a confidence-weighting principle to regulate learning: for a given surprise, the update is smaller when the confidence about the prediction was higher. Prior behavioral evidence indicates that human learning adheres to this confidence-weighting principle. Here, we explored the human brain dynamics sub-tending the confidence-weighting of learning using magneto-encephalography (MEG). During our volatile probability learning task, subjects’ confidence reports conformed with Bayesian inference. MEG revealed several stimulus-evoked brain responses whose amplitude reflected surprise, and some of them were further shaped by confidence: surprise amplified the stimulus-evoked response whereas confidence dampened it. Confidence about predictions also modulated several aspects of the brain state: pupil-linked arousal and beta-range (15–30 Hz) oscillations. The brain state in turn modulated specific stimulus-evoked surprise responses following the confidence-weighting principle. Our results thus indicate that there exist, in the human brain, signals reflecting surprise that are dampened by confidence in a way that is appropriate for learning according to Bayesian inference. They also suggest a mechanism for confidence-weighted learning: confidence about predictions would modulate intrinsic properties of the brain state to amplify or dampen surprise responses evoked by discrepant observations.

Learning in a changing and uncertain world is difficult. In this context, facing a discrepancy between my current belief and new observations may reflect random fluctuations (e.g. my commute train is unexpectedly late, but it happens sometimes), if so, I should ignore this discrepancy and not change erratically my belief. However, this discrepancy could also denote a profound change (e.g. the train company changed and is less reliable), in this case, I should promptly revise my current belief. Human learning is adaptive: we change how much we learn from new observations, in particular, we promote flexibility when facing profound changes. A mathematical analysis of the problem shows that we should increase flexibility when the confidence about our current belief is low, which occurs when a change is suspected. Here, I show that human learners entertain rational confidence levels during the learning of changing probabilities. This confidence modulates intrinsic properties of the brain state (oscillatory activity and neuromodulation) which in turn amplifies or reduces, depending on whether confidence is low or high, the neural responses to discrepant observations. This confidence-weighting mechanism could underpin adaptive learning.

Paranoia as a deficit in non-social belief updating

Paranoia is the belief that harm is intended by others. It may arise from selective pressures to infer and avoid social threats, particularly in ambiguous or changing circumstances. We propose that uncertainty may be sufficient to elicit learning differences in paranoid individuals, without social threat. We used reversal learning behavior and computational modeling to estimate belief updating across individuals with and without mental illness, online participants, and rats chronically exposed to methamphetamine, an elicitor of paranoia in humans. Paranoia is associated with a stronger prior on volatility, accompanied by elevated sensitivity to perceived changes in the task environment. Methamphetamine exposure in rats recapitulates this impaired uncertainty-driven belief updating and rigid anticipation of a volatile environment. Our work provides evidence of fundamental, domain-general learning differences in paranoid individuals. This paradigm enables further assessment of the interplay between uncertainty and belief-updating across individuals and species.

Everyone has had fleeting concerns that others might be against them at some point in their lives. Sometimes these concerns can escalate into paranoia and become debilitating. Paranoia is a common symptom in serious mental illnesses like schizophrenia. It can cause extreme distress and is linked with an increased risk of violence towards oneself or others. Understanding what happens in the brains of people experiencing paranoia might lead to better ways to treat or manage it.

Some experts argue that paranoia is caused by errors in the way people assess social situations. An alternative idea is that paranoia stems from the way the brain forms and updates beliefs about the world. Now, Reed et al. show that both people with paranoia and rats exposed to a paranoia-inducing substance expect the world will change frequently, change their minds often, and have a harder time learning in response to changing circumstances.

In the experiments, human volunteers with and without psychiatric disorders played a game where the best choices change. Then, the participants completed a survey to assess their level of paranoia. People with higher levels of paranoia predicted more changes would occur and made less predictable choices. In a second set of experiments, rats were put in a cage with three holes where they sometimes received sugar rewards. Some of the rats received methamphetamine, a drug that causes paranoia in humans. Rats given the drug also expected the location of the sugar reward would change often. The drugged animals had harder time learning and adapting to changing circumstances.

The experiments suggest that brain processes found in both rats, which are less social than humans, and humans contribute to paranoia. This suggests paranoia may make it harder to update beliefs. This may help scientists understand what causes paranoia and develop therapies or drugs that can reduce paranoia. This information may also help scientists understand why during societal crises like wars or natural disasters humans are prone to believing conspiracies. This is particularly important now as the world grapples with climate change and a global pandemic. Reed et al. note paranoia may impede the coordination of collaborative solutions to these challenging situations.

Alterations in the amplitude and burst rate of beta oscillations impair reward-dependent motor learning in anxiety

Anxiety results in sub-optimal motor learning, but the precise mechanisms through which this effect occurs remain unknown. Using a motor sequence learning paradigm with separate phases for initial exploration and reward-based learning, we show that anxiety states in humans impair learning by attenuating the update of reward estimates. Further, when such estimates are perceived as unstable over time (volatility), anxiety constrains adaptive behavioral changes. Neurally, anxiety during initial exploration increased the amplitude and the rate of long bursts of sensorimotor and prefrontal beta oscillations (13-30 Hz). These changes extended to the subsequent learning phase, where phasic increases in beta power and burst rate following reward feedback were linked to smaller updates in reward estimates, with a higher anxiety-related increase explaining the attenuated belief updating. These data suggest that state anxiety alters the dynamics of beta oscillations during reward processing, thereby impairing proper updating of motor predictions when learning in unstable environments.

Impaired context-sensitive adjustment of behaviour in Parkinson's disease patients tested on and off medication: An fMRI study

The brain's sensitivity to and accentuation of unpredicted over predicted sensory signals plays a fundamental role in learning. According to recent theoretical models of the predictive coding framework, dopamine is responsible for balancing the interplay between bottom-up input and top-down predictions by controlling the precision of surprise signals that guide learning. Using functional MRI, we investigated whether patients with Parkinson's disease (PD) show impaired learning from prediction errors requiring either adaptation or stabilisation of current predictions. Moreover, we were interested in whether deficits in learning over a specific time scale would be accompanied by altered surprise responses in dopamine-related brain structures. To this end, twenty-one PD patients tested on and off dopaminergic medication and twenty-one healthy controls performed a digit prediction paradigm. During the task, violations of sequence-based predictions either signalled the need to update or to stabilise the current prediction and, thus, to react to them or ignore them, respectively. To investigate contextual adaptation to prediction errors, the probability (or its inverse, surprise) of the violations fluctuated across the experiment. When the probability of prediction errors over a specific time scale increased, healthy controls but not PD patients off medication became more flexible, i.e., error rates at violations requiring a motor response decreased in controls but increased in patients. On the neural level, this learning deficit in patients was accompanied by reduced signalling in the substantia nigra and the caudate nucleus. In contrast, differences between the groups regarding the probabilistic modulation of behaviour and neural responses were much less pronounced at prediction errors requiring only stabilisation but no adaptation. Interestingly, dopaminergic medication could neither improve learning from prediction errors nor restore the physiological, neurotypical pattern. Our findings point to a pivotal role of dysfunctions of the substantia nigra and caudate nucleus in deficits in learning from flexibility-demanding prediction errors in PD. Moreover, the data witness poor effects of dopaminergic medication on learning in PD.

Context is everything: How context shapes modulations of responses to unattended sound

The concept of perceptual inferences taking place over multiple timescales simultaneously raises questions about how the brain can balance the demands of remaining sensitive to local rarity while utilising more global longer-term predictability to modulate cortical responses. In the present study auditory evoked potentials to four presentations of the same sound sequence containing predictable structure on a local (milliseconds to seconds) and more global (many minutes) timescales were recorded. The results from 33 participants are used to demonstrate that predictions about both local (internal predictive models) and global (meta-models that define expected precisions associated with familiar internal model states) regularities are formed. The study exposes more local context-based modulations of the P1 but more global order-based modulations of the auditory evoked N2 components. The results are discussed in terms of theoretical links advocating that uncertainty at multiple timescales could lead to differential component modulations, and the importance of considering the broader learning context in auditory evoked potential studies.

Maladaptive consequences of repeated intermittent exposure to uncertainty

Recently we reported that nucleus accumbens (NAcc) dopamine (DA) tracks uncertainty during operant responding for non-caloric saccharin. We also showed that repeated intermittent exposure to this uncertainty, like exposure to drugs of abuse, leads to sensitization of the locomotor and NAcc DA effects of amphetamine and promotes the subsequent self-administration of the drug. Here we review these findings together with others showing that NAcc glutamate signaling is similarly affected by uncertainty. Extracellular levels of glutamate in this site also track uncertainty in a task in which nose poking for saccharin on an escalating variable ratio schedule of reinforcement is associated with progressively increasing variance between performance of the operant and payout. Furthermore, sensitized behavioral responding to and for amphetamine following exposure to uncertainty is accompanied by increased levels of Ca2+/calmodulin-dependent protein kinase II (CaMKII) and protein kinase C (PKC) phosphorylation as well as altered protein levels of the transcription factor ∆FosB (increased) and glutamate transporter 1 (GLT1; decreased) in NAcc tissues. Notably, phosphorylation by CaMKII and PKC regulates AMPA receptor trafficking and function in this site, is elevated following psychostimulant exposure, and is necessary for the expression of enhanced drug taking. Increased ∆FosB and decreased GLT1 levels are observed following psychostimulant exposure, are associated with increased drug taking and seeking, and are known to modulate AMPA receptors and extracellular glutamate levels respectively. These adaptations in glutamate transmission as well as those observed with DA following repeated intermittent exposure to uncertainty are similar to those produced by exposure to abused drugs. Together, they point to the recruitment of both DA and glutamate signaling pathways in the NAcc in both drug and behavioral addictions. As uncertainty is central to games of chance, these findings have particular relevance for gambling disorders known to exhibit comorbidity with drug abuse.

Exploitation of local and global information in predictive processing

While prediction errors have been established to instigate learning through model adaptation, recent studies have stressed the role of model-compliant events in predictive processing. Specifically, probabilistic information at critical points in time (so-called checkpoints) has been suggested to be sampled in order to evaluate the internal model, particularly in uncertain contexts. This way, initial model-based expectations are iteratively reaffirmed under uncertainty, even in the absence of prediction errors. Using electroencephalography (EEG), the present study aimed to investigate the interplay of such global uncertainty information and local adjustment cues prompting on-line adjustments of expectations. Within a stream of single digits, participants were to detect ordered sequences (i.e., 3-4-5-6-7) that had a regular length of five digits and were occasionally extended to seven digits. Over time, these extensions were either rare (low irreducible uncertainty) or frequent (high uncertainty) and could be unexpected or indicated by incidental colour cues. Accounting for cue information, an N400 component was revealed as the correlate of locally unexpected (vs expected) outcomes, reflecting effortful integration of incongruous information. As for model-compliant information, multivariate pattern decoding within the P3b time frame demonstrated effective exploitation of local (adjustment cues vs non-informative analogues) and global information (high vs low uncertainty regular endings) sampled from probabilistic events. Finally, superior fit of a global model (disregarding local adjustments) compared to a local model (including local adjustments) in a representational similarity analysis underscored the precedence of global reference frames in hierarchical predictive processing. Overall, results suggest that just like error-induced model adaptation, model evaluation is not limited to either local or global information. Following the hierarchical organisation of predictive processing, model evaluation too can occur at several levels of the processing hierarchy.

Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models

Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear-especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability-similar to decision 'noise' parameters in RL-and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a 'go/no-go' task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision 'noise' (P = 0.020), and this relationship with D2/3R availability was confirmed with a 'decision stochasticity' factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans.

Catecholaminergic modulation of meta-learning

The remarkable expedience of human learning is thought to be underpinned by meta-learning, whereby slow accumulative learning processes are rapidly adjusted to the current learning environment. To date, the neurobiological implementation of meta-learning remains unclear. A burgeoning literature argues for an important role for the catecholamines dopamine and noradrenaline in meta-learning. Here, we tested the hypothesis that enhancing catecholamine function modulates the ability to optimise a meta-learning parameter (learning rate) as a function of environmental volatility. 102 participants completed a task which required learning in stable phases, where the probability of reinforcement was constant, and volatile phases, where probabilities changed every 10-30 trials. The catecholamine transporter blocker methylphenidate enhanced participants' ability to adapt learning rate: Under methylphenidate, compared with placebo, participants exhibited higher learning rates in volatile relative to stable phases. Furthermore, this effect was significant only with respect to direct learning based on the participants' own experience, there was no significant effect on inferred-value learning where stimulus values had to be inferred. These data demonstrate a causal link between catecholaminergic modulation and the adjustment of the meta-learning parameter learning rate.

A Basal Forebrain-Cingulate Circuit in Macaques Decides It Is Time to Act

The medial frontal cortex has been linked to voluntary action, but an explanation of why decisions to act emerge at particular points in time has been lacking. We show that, in macaques, decisions about whether and when to act are predicted by a set of features defining the animal’s current and past context; for example, respectively, cues indicating the current average rate of reward and recent previous voluntary action decisions. We show that activity in two brain areas—the anterior cingulate cortex and basal forebrain—tracks these contextual factors and mediates their effects on behavior in distinct ways. We use focused transcranial ultrasound to selectively and effectively stimulate deep in the brain, even as deep as the basal forebrain, and demonstrate that alteration of activity in the two areas changes decisions about when to act.

•

Likelihood and timing of voluntary action in macaques can be partially predicted

•

Recent experience and present context influence when voluntary action occurs

•

A basal forebrain-cingulate circuit mediated effects of these factors on behavior

•

Stimulation of this circuit by ultrasound changed decisions about when to act

Perceptual awareness and active inference

Perceptual awareness depends upon the way in which we engage with our sensorium. This notion is central to active inference, a theoretical framework that treats perception and action as inferential processes. This variational perspective on cognition formalizes the notion of perception as hypothesis testing and treats actions as experiments that are designed (in part) to gather evidence for or against alternative hypotheses. The common treatment of perception and action affords a useful interpretation of certain perceptual phenomena whose active component is often not acknowledged. In this article, we start by considering Troxler fading - the dissipation of a peripheral percept during maintenance of fixation, and its recovery during free (saccadic) exploration. This offers an important example of the failure to maintain a percept without actively interrogating a visual scene. We argue that this may be understood in terms of the accumulation of uncertainty about a hypothesized stimulus when free exploration is disrupted by experimental instructions or pathology. Once we take this view, we can generalize the idea of using bodily (oculomotor) action to resolve uncertainty to include the use of mental (attentional) actions for the same purpose. This affords a useful way to think about binocular rivalry paradigms, in which perceptual changes need not be associated with an overt movement.

Pupil-linked phasic arousal evoked by violation but not emergence of regularity within rapid sound sequences

The ability to track the statistics of our surroundings is a key computational challenge. A prominent theory proposes that the brain monitors for unexpected uncertainty - events which deviate substantially from model predictions, indicating model failure. Norepinephrine is thought to play a key role in this process by serving as an interrupt signal, initiating model-resetting. However, evidence is from paradigms where participants actively monitored stimulus statistics. To determine whether Norepinephrine routinely reports the statistical structure of our surroundings, even when not behaviourally relevant, we used rapid tone-pip sequences that contained salient pattern-changes associated with abrupt structural violations vs. emergence of regular structure. Phasic pupil dilations (PDR) were monitored to assess Norepinephrine. We reveal a remarkable specificity: When not behaviourally relevant, only abrupt structural violations evoke a PDR. The results demonstrate that Norepinephrine tracks unexpected uncertainty on rapid time scales relevant to sensory signals.

The Misestimation of Uncertainty in Affective Disorders

Our knowledge about the state of the world is often incomplete, making it difficult to select the best course of action. One strategy that can be used to improve our ability to make decisions is to identify the causes of our ignorance (i.e., why an unexpected event might have occurred) and use estimates of the uncertainty induced by these causes to guide our learning. Here, we explain the logic behind this process and describe the evidence that human learners use estimates of uncertainty to sculpt their learning. Finally, we describe recent work suggesting that misestimation of uncertainty is involved in the development of anxiety and depression and describe how these ideas may be advanced.

The computational pharmacology of oculomotion

Many physiological and pathological changes in brain function manifest in eye-movement control. As such, assessment of oculomotion is an invaluable part of a clinical examination and affords a non-invasive window on several key aspects of neuronal computation. While oculomotion is often used to detect deficits of the sort associated with vascular or neoplastic events; subtler (e.g. pharmacological) effects on neuronal processing also induce oculomotor changes. We have previously framed oculomotor control as part of active vision, namely, a process of inference comprising two distinct but related challenges. The first is inferring where to look, and the second is inferring how to implement the selected action. In this paper, we draw from recent theoretical work on the neuromodulatory control of active inference. This allows us to simulate the sort of changes we would expect in oculomotor behaviour, following pharmacological enhancement or suppression of key neuromodulators-in terms of deciding where to look and the ensuing trajectory of the eye movement itself. We focus upon the influence of cholinergic and GABAergic agents on the speed of saccades, and consider dopaminergic and noradrenergic effects on more complex, memory-guided, behaviour. In principle, a computational approach to understanding the relationship between pharmacology and oculomotor behaviour affords the opportunity to estimate the influence of a given pharmaceutical upon neuronal function, and to use this to optimise therapeutic interventions on an individual basis.

With an eye on uncertainty: Modelling pupillary responses to environmental volatility

Living creatures must accurately infer the nature of their environments. They do this despite being confronted by stochastic and context sensitive contingencies—and so must constantly update their beliefs regarding their uncertainty about what might come next. In this work, we examine how we deal with uncertainty that evolves over time. This prospective uncertainty (or imprecision) is referred to as volatility and has previously been linked to noradrenergic signals that originate in the locus coeruleus. Using pupillary dilatation as a measure of central noradrenergic signalling, we tested the hypothesis that changes in pupil diameter reflect inferences humans make about environmental volatility. To do so, we collected pupillometry data from participants presented with a stream of numbers. We generated these numbers from a process with varying degrees of volatility. By measuring pupillary dilatation in response to these stimuli—and simulating the inferences made by an ideal Bayesian observer of the same stimuli—we demonstrate that humans update their beliefs about environmental contingencies in a Bayes optimal way. We show this by comparing general linear (convolution) models that formalised competing hypotheses about the causes of pupillary changes. We found greater evidence for models that included Bayes optimal estimates of volatility than those without. We additionally explore the interaction between different causes of pupil dilation and suggest a quantitative approach to characterising a person’s prior beliefs about volatility.

Humans are constantly confronted with surprising events. To navigate such a world, we must understand the chances of an unexpected event occurring at any given point in time. We do this by creating a model of the world around us, in which we allow for these unexpected events to occur by holding beliefs about how volatile our environment is. In this work we explore the way in which we update our beliefs, demonstrating that this updating relies on the number of unexpected events in relation to the expected number. We do this by examining the pupil diameter, since—in controlled environments—changes in pupil diameter reflect our response to unexpected observations. Finally, we show that our methodology is appropriate for assessing the individual participant’s prior expectations about the amount of uncertainty in their environment.

Catecholamines: Knowledge and understanding in the 1960s, now, and in the future

The late 1960s was a heyday for catecholamine research. Technological developments made it feasible to study the regulation of sympathetic neuronal transmission and to map the distribution of noradrenaline and dopamine in the brain. At last, it was possible to explain the mechanism of action of some important drugs that had been used in the clinic for more than a decade (e.g. the first generation of antidepressants) and to contemplate the rational development of new treatments (e.g. l-dihydroxyphenylalanine therapy, to compensate for the dopaminergic neuropathy in Parkinson’s disease, and β₁-adrenoceptor antagonists as antihypertensives). The fact that drug targeting noradrenergic and/or dopaminergic transmission are still the first-line treatments for many psychiatric disorders (e.g. depression, schizophrenia, and attention deficit hyperactivity disorder) is a testament to the importance of these neurotransmitters and the research that has helped us to understand the regulation of their function. This article celebrates some of the highlights of research at that time, pays tribute to some of the subsequent landmark studies, and appraises the options for where it could go next.