The neurocomputational link between defensive cardiac states and approach-avoidance arbitration under threat

Avoidance, a hallmark of anxiety-related psychopathology, often comes at a cost; avoiding threat may forgo the possibility of a reward. Theories predict that optimal approach-avoidance arbitration depends on threat-induced psychophysiological states, like freezing-related bradycardia. Here we used model-based fMRI analyses to investigate whether and how bradycardia states are linked to the neurocomputational underpinnings of approach-avoidance arbitration under varying reward and threat magnitudes. We show that bradycardia states are associated with increased threat-induced avoidance and more pronounced reward-threat value comparison (i.e., a stronger tendency to approach vs. avoid when expected reward outweighs threat). An amygdala-striatal-prefrontal circuit supports approach-avoidance arbitration under threat, with specific involvement of the amygdala and dorsal anterior cingulate (dACC) in integrating reward-threat value and bradycardia states. These findings highlight the role of human freezing states in value-based decision making, relevant for optimal threat coping. They point to a specific role for amygdala/dACC in state-value integration under threat.

Curiosity and the dynamics of optimal exploration

What drives our curiosity remains an elusive and hotly debated issue, with multiple hypotheses proposed but a cohesive account yet to be established. This review discusses traditional and emergent theories that frame curiosity as a desire to know and a drive to learn, respectively. We adopt a model-based approach that maps the temporal dynamics of various factors underlying curiosity-based exploration, such as uncertainty, information gain, and learning progress. In so doing, we identify the limitations of past theories and posit an integrated account that harnesses their strengths in describing curiosity as a tool for optimal environmental exploration. In our unified account, curiosity serves as a 'common currency' for exploration, which must be balanced with other drives such as safety and hunger to achieve efficient action.

Goal commitment is supported by vmPFC through selective attention

When striking a balance between commitment to a goal and flexibility in the face of better options, people often demonstrate strong goal perseveration. Here, using functional MRI (n = 30) and lesion patient (n = 26) studies, we argue that the ventromedial prefrontal cortex (vmPFC) drives goal commitment linked to changes in goal-directed selective attention. Participants performed an incremental goal pursuit task involving sequential decisions between persisting with a goal versus abandoning progress for better alternative options. Individuals with stronger goal perseveration showed higher goal-directed attention in an interleaved attention task. Increasing goal-directed attention also affected abandonment decisions: while pursuing a goal, people lost their sensitivity to valuable alternative goals while remaining more sensitive to changes in the current goal. In a healthy population, individual differences in both commitment biases and goal-oriented attention were predicted by baseline goal-related activity in the vmPFC. Among lesion patients, vmPFC damage reduced goal commitment, leading to a performance benefit.

The Computational and Neural Substrates of Ambiguity Avoidance in Anxiety

Theoretical accounts have linked anxiety to intolerance of ambiguity. However, this relationship has not been well operationalized empirically. Here, we used computational and neuro-imaging methods to characterize anxiety-related differences in aversive decision-making under ambiguity and associated patterns of cortical activity. Adult human participants chose between two urns on each trial. The ratio of tokens ('O's and 'X's) in each urn determined probability of electrical stimulation receipt. A number above each urn indicated the magnitude of stimulation that would be received if a shock was delivered. On ambiguous trials, one of the two urns had tokens occluded. By varying the number of tokens occluded, we manipulated the extent of missing information. At higher levels of missing information, there is greater second order uncertainty, i.e., more uncertainty as to the probability of pulling a given type of token from the urn. Adult human participants demonstrated avoidance of ambiguous options which increased with level of missing information. Extent of 'information-level dependent' ambiguity aversion was significantly positively correlated with trait anxiety. Activity in both the dorsal anterior cingulate cortex and inferior frontal sulcus during the decision-making period increased as a function of missing information. Greater engagement of these regions, on high missing information trials, was observed when participants went on to select the ambiguous option; this was especially apparent in high trait anxious individuals. These findings are consistent with individuals vulnerable to anxiety requiring greater activation of frontal regions supporting rational decision-making to overcome a predisposition to engage in ambiguity avoidance at high levels of missing information.

Medial Frontal Cortex Activity Predicts Information Sampling in Economic Choice

Decision-making not only requires agents to decide what to choose but also how much information to sample before committing to a choice. Previously established frameworks for economic choice argue for a deliberative process of evidence accumulation across time. These tacitly acknowledge a role of information sampling in that decisions are only made once sufficient evidence is acquired, yet few experiments have explicitly placed information sampling under the participant's control. Here, we use fMRI to investigate the neural basis of information sampling in economic choice by allowing participants (n = 30, sex not recorded) to actively sample information in a multistep decision task. We show that medial frontal cortex (MFC) activity is predictive of further information sampling before choice. Choice difficulty (inverse value difference, keeping sensory difficulty constant) was also encoded in MFC, but this effect was explained away by the inclusion of information sampling as a coregressor in the general linear model. A distributed network of regions across the prefrontal cortex encoded key features of the sampled information at the time it was presented. We propose that MFC is an important controller of the extent to which information is gathered before committing to an economic choice. This role may explain why MFC activity has been associated with evidence accumulation in previous studies in which information sampling was an implicit rather than explicit feature of the decision.SIGNIFICANCE STATEMENT The decisions we make are determined by the information we have sampled before committing to a choice. Accumulator frameworks of decision-making tacitly acknowledge the need to sample further information during the evidence accumulation process until a decision boundary is reached. However, relatively few studies explicitly place this decision to sample further information under the participant's control. In this fMRI study, we find that MFC activity is related to information sampling decisions in a multistep economic choice task. This suggests that an important role of evidence representations within MFC may be to guide adaptive sequential decisions to sample further information before committing to a final decision.

Neuronal Computation Underlying Inferential Reasoning in Humans and Mice

Every day we make decisions critical for adaptation and survival. We repeat actions with known consequences. But we also draw on loosely related events to infer and imagine the outcome of entirely novel choices. These inferential decisions are thought to engage a number of brain regions; however, the underlying neuronal computation remains unknown. Here, we use a multi-day cross-species approach in humans and mice to report the functional anatomy and neuronal computation underlying inferential decisions. We show that during successful inference, the mammalian brain uses a hippocampal prospective code to forecast temporally structured learned associations. Moreover, during resting behavior, coactivation of hippocampal cells in sharp-wave/ripples represent inferred relationships that include reward, thereby "joining-the-dots" between events that have not been observed together but lead to profitable outcomes. Computing mnemonic links in this manner may provide an important mechanism to build a cognitive map that stretches beyond direct experience, thus supporting flexible behavior.

Nine-month-old infants update their predictive models of a changing environment

Humans generate internal models of their environment to predict events in the world. As the environments change, our brains adjust to these changes by updating their internal models. Here, we investigated whether and how 9-month-old infants differentially update their models to represent a dynamic environment. Infants observed a predictable sequence of stimuli, which were interrupted by two types of cues. Following the update cue, the pattern was altered, thus, infants were expected to update their predictions for the upcoming stimuli. Because the pattern remained the same after the no-update cue, no subsequent updating was required. Infants showed an amplified negative central (Nc) response when the predictable sequence was interrupted. Late components such as the PSW were also evoked in response to unexpected stimuli; however, we found no evidence for a differential response to the informational value of surprising cues at later stages of processing. Infants rather learned that surprising cues always signal a change in the environment that requires updating. Interestingly, infants responded with an amplified neural response to the absence of an expected change, suggesting a top-down modulation of early sensory processing in infants. Our findings corroborate emerging evidence showing that infants build predictive models early in life.

Control of entropy in neural models of environmental state

Humans and animals construct internal models of their environment in order to select appropriate courses of action. The representation of uncertainty about the current state of the environment is a key feature of these models that controls the rate of learning as well as directly affecting choice behaviour. To maintain flexibility, given that uncertainty naturally decreases over time, most theoretical inference models include a dedicated mechanism to drive up model uncertainty. Here we probe the long-standing hypothesis that noradrenaline is involved in determining the uncertainty, or entropy, and thus flexibility, of neural models. Pupil diameter, which indexes neuromodulatory state including noradrenaline release, predicted increases (but not decreases) in entropy in a neural state model encoded in human medial orbitofrontal cortex, as measured using multivariate functional MRI. Activity in anterior cingulate cortex predicted pupil diameter. These results provide evidence for top-down, neuromodulatory control of entropy in neural state models.

Infants differentially update their internal models of a dynamic environment

Unexpected events provide us with opportunities for learning about what to expect from the world around us. Using a saccadic-planning paradigm, we investigated whether and how infants and adults represent the statistics of a changing environment (i.e. build an internal model of the environment). Participants observed differently colored bees that appeared at an unexpected location every few trials. The color cues indicated whether the subsequent bees would appear at this new location (i.e. update trials) or at the same location as previously (i.e. no-update trials). Infants learned the predictive value of the color cues and updated their internal models when necessary. Unlike infants, adults had a tendency to update their models each time they observed a change in the structure. We argue that infants are open to learning from current evidence due to being less influenced by their prior knowledge. This is an advantageous learning strategy to form accurate representations in dynamic environments, which is fundamental for successful adaptation.

State-change decisions and dorsomedial prefrontal cortex: the importance of time

Different kinds of decision making can be categorized by their differential effect on the agent’s current and future states as well as the computational challenges they pose. Here, we draw a distinction between within-state and state-change decision-making, and propose that a dedicated decision mechanism exists in dorsomedial prefrontal cortex (dmPFC) that is specialized for state-change decisions. We set out a formal framework in which state change decisions may be made on the basis of the integrated momentary reward rate, over the intended time to be spent in a state. A key feature of this framework is that reward rate is expressed as a function of continuous time. We argue that dmPFC is suited for this type of decision making partly due to its ability to track the passage of time. This proposed function of dmPFC is placed in contrast to other evaluative systems such as the orbitofrontal cortex, which is important for careful deliberation within a specific model-space or option-space and within a decision strategy.

Towards a neuro-computational account of prism adaptation

Prism adaptation has a long history as an experimental paradigm used to investigate the functional and neural processes that underlie sensorimotor control. In the neuropsychology literature, prism adaptation behaviour is typically explained by reference to a traditional cognitive psychology framework that distinguishes putative functions, such as 'strategic control' versus 'spatial realignment'. This theoretical framework lacks conceptual clarity, quantitative precision and explanatory power. Here, we advocate for an alternative computational framework that offers several advantages: 1) an algorithmic explanatory account of the computations and operations that drive behaviour; 2) expressed in quantitative mathematical terms; 3) embedded within a principled theoretical framework (Bayesian decision theory, state-space modelling); 4) that offers a means to generate and test quantitative behavioural predictions. This computational framework offers a route towards mechanistic neurocognitive explanations of prism adaptation behaviour. Thus it constitutes a conceptual advance compared to the traditional theoretical framework. In this paper, we illustrate how Bayesian decision theory and state-space models offer principled explanations for a range of behavioural phenomena in the field of prism adaptation (e.g. visual capture, magnitude of visual versus proprioceptive realignment, spontaneous recovery and dynamics of adaptation memory). We argue that this explanatory framework can advance understanding of the functional and neural mechanisms that implement prism adaptation behaviour, by enabling quantitative tests of hypotheses that go beyond merely descriptive mapping claims that ‘brain area X is (somehow) involved in psychological process Y’.

•

Traditional neuropsychological models of prism adaptation lack precision.

•

Computational models improve explanatory and predictive power.

•

A range of adaptation phenomena can be explained quantitatively.

•

Mathematics offers a bridge between neural mechanisms and behaviour.

•

A neuro-computational approach will advance neuropsychology.

Organizing conceptual knowledge in humans with a gridlike code

It has been hypothesized that the brain organizes concepts into a mental map, allowing conceptual relationships to be navigated in a manner similar to that of space. Grid cells use a hexagonally symmetric code to organize spatial representations and are the likely source of a precise hexagonal symmetry in the functional magnetic resonance imaging signal. Humans navigating conceptual two-dimensional knowledge showed the same hexagonal signal in a set of brain regions markedly similar to those activated during spatial navigation. This gridlike signal is consistent across sessions acquired within an hour and more than a week apart. Our findings suggest that global relational codes may be used to organize nonspatial conceptual representations and that these codes may have a hexagonal gridlike pattern when conceptual knowledge is laid out in two continuous dimensions.

Two Anatomically and Computationally Distinct Learning Signals Predict Changes to Stimulus-Outcome Associations in Hippocampus

Complex cognitive processes require sophisticated local processing but also interactions between distant brain regions. It is therefore critical to be able to study distant interactions between local computations and the neural representations they act on. Here we report two anatomically and computationally distinct learning signals in lateral orbitofrontal cortex (lOFC) and the dopaminergic ventral midbrain (VM) that predict trial-by-trial changes to a basic internal model in hippocampus. To measure local computations during learning and their interaction with neural representations, we coupled computational fMRI with trial-by-trial fMRI suppression. We find that suppression in a medial temporal lobe network changes trial-by-trial in proportion to stimulus-outcome associations. During interleaved choice trials, we identify learning signals that relate to outcome type in lOFC and to reward value in VM. These intervening choice feedback signals predicted the subsequent change to hippocampal suppression, suggesting a convergence of signals that update the flexible representation of stimulus-outcome associations.

Testing the inter-hemispheric competition account of visual extinction with combined TMS/fMRI

Theoretical models of visual neglect and extinction entail claims about the normal functioning of attention and parietal cortex in the healthy brain: (1) 'pseudoneglect', a commonly observed attentional bias towards left space, reflects the greater dominance of parietal cortex activity of the right versus left hemisphere; (2) the capacity to distribute attention bilaterally depends causally on the relative balance of parietal activity between the hemispheres; (3) disruption of the dominant right parietal cortex shifts this inter-hemispheric balance leftward, causing a rightward shift in attentional bias. We tested these claims using low-frequency offline transcranial magnetic stimulation (TMS) to transiently inhibit activity in the right angular gyrus/intra-parietal sulcus, followed by a visual detection task to assess changes in attentional bias, and functional magnetic resonance imaging (fMRI) to test for the predicted leftward shift in brain activity. The task required participants to covertly monitor both hemifields to detect and report the location of upcoming transient visual targets that appeared on the left, right or bilaterally. In the behavioural experiment, participants exhibited a leftward attentional bias ('pseudoneglect') at baseline, which was abolished by TMS. In the fMRI experiment, participants activated an expected network of visual, parietal and frontal cortex bilaterally during the period of covert bilateral attention. TMS shifted the relative hemispheric balance of parietal activity from right to left. The consistent direction of TMS-induced behavioural and functional change indicates a causal role for parietal inter-hemispheric balance in distributing visual attention across space.

Causal manipulation of functional connectivity in a specific neural pathway during behaviour and at rest

Correlations in brain activity between two areas (functional connectivity) have been shown to relate to their underlying structural connections. We examine the possibility that functional connectivity also reflects short-term changes in synaptic efficacy. We demonstrate that paired transcranial magnetic stimulation (TMS) near ventral premotor cortex (PMv) and primary motor cortex (M1) with a short 8-ms inter-pulse interval evoking synchronous pre- and post-synaptic activity and which strengthens interregional connectivity between the two areas in a pattern consistent with Hebbian plasticity, leads to increased functional connectivity between PMv and M1 as measured with functional magnetic resonance imaging (fMRI). Moreover, we show that strengthening connectivity between these nodes has effects on a wider network of areas, such as decreasing coupling in a parallel motor programming stream. A control experiment revealed that identical TMS pulses at identical frequencies caused no change in fMRI-measured functional connectivity when the inter-pulse-interval was too long for Hebbian-like plasticity.

A neural circuit covarying with social hierarchy in macaques

Despite widespread interest in social dominance, little is known of its neural correlates in primates. We hypothesized that social status in primates might be related to individual variation in subcortical brain regions implicated in other aspects of social and emotional behavior in other mammals. To examine this possibility we used magnetic resonance imaging (MRI), which affords the taking of quantitative measurements noninvasively, both of brain structure and of brain function, across many regions simultaneously. We carried out a series of tests of structural and functional MRI (fMRI) data in 25 group-living macaques. First, a deformation-based morphometric (DBM) approach was used to show that gray matter in the amygdala, brainstem in the vicinity of the raphe nucleus, and reticular formation, hypothalamus, and septum/striatum of the left hemisphere was correlated with social status. Second, similar correlations were found in the same areas in the other hemisphere. Third, similar correlations were found in a second data set acquired several months later from a subset of the same animals. Fourth, the strength of coupling between fMRI-measured activity in the same areas was correlated with social status. The network of subcortical areas, however, had no relationship with the sizes of individuals' social networks, suggesting the areas had a simple and direct relationship with social status. By contrast a second circuit in cortex, comprising the midsuperior temporal sulcus and anterior and dorsal prefrontal cortex, covaried with both individuals' social statuses and the social network sizes they experienced. This cortical circuit may be linked to the social cognitive processes that are taxed by life in more complex social networks and that must also be used if an animal is to achieve a high social status.

The organization of dorsal frontal cortex in humans and macaques

The human dorsal frontal cortex has been associated with the most sophisticated aspects of cognition, including those that are thought to be especially refined in humans. Here we used diffusion-weighted magnetic resonance imaging (DW-MRI) and functional MRI (fMRI) in humans and macaques to infer and compare the organization of dorsal frontal cortex in the two species. Using DW-MRI tractography-based parcellation, we identified 10 dorsal frontal regions lying between the human inferior frontal sulcus and cingulate cortex. Patterns of functional coupling between each area and the rest of the brain were then estimated with fMRI and compared with functional coupling patterns in macaques. Areas in human medial frontal cortex, including areas associated with high-level social cognitive processes such as theory of mind, showed a surprising degree of similarity in their functional coupling patterns with the frontal pole, medial prefrontal, and dorsal prefrontal convexity in the macaque. We failed to find evidence for "new" regions in human medial frontal cortex. On the lateral surface, comparison of functional coupling patterns suggested correspondences in anatomical organization distinct from those that are widely assumed. A human region sometimes referred to as lateral frontal pole more closely resembled area 46, rather than the frontal pole, of the macaque. Overall the pattern of results suggest important similarities in frontal cortex organization in humans and other primates, even in the case of regions thought to carry out uniquely human functions. The patterns of interspecies correspondences are not, however, always those that are widely assumed.

Making predictions in a changing world-inference, uncertainty, and learning

To function effectively, brains need to make predictions about their environment based on past experience, i.e., they need to learn about their environment. The algorithms by which learning occurs are of interest to neuroscientists, both in their own right (because they exist in the brain) and as a tool to model participants' incomplete knowledge of task parameters and hence, to better understand their behavior. This review focusses on a particular challenge for learning algorithms-how to match the rate at which they learn to the rate of change in the environment, so that they use as much observed data as possible whilst disregarding irrelevant, old observations. To do this algorithms must evaluate whether the environment is changing. We discuss the concepts of likelihood, priors and transition functions, and how these relate to change detection. We review expected and estimation uncertainty, and how these relate to change detection and learning rate. Finally, we consider the neural correlates of uncertainty and learning. We argue that the neural correlates of uncertainty bear a resemblance to neural systems that are active when agents actively explore their environments, suggesting that the mechanisms by which the rate of learning is set may be subject to top down control (in circumstances when agents actively seek new information) as well as bottom up control (by observations that imply change in the environment).

Tools of the trade: psychophysiological interactions and functional connectivity

Psychophysiological interactions (PPIs) analysis is a method for investigating task-specific changes in the relationship between activity in different brain areas, using functional magnetic resonance imaging (fMRI) data. Specifically, PPI analyses identify voxels in which activity is more related to activity in a seed region of interest (seed ROI) in a given psychological context, such as during attention or in the presence of emotive stimuli. In this tutorial, we aim to give a simple conceptual explanation of how PPI analysis works, in order to assist readers in planning and interpreting their own PPI experiments.

Social network size affects neural circuits in macaques

It has been suggested that variation in brain structure correlates with the sizes of individuals' social networks. Whether variation in social network size causes variation in brain structure, however, is unknown. To address this question, we neuroimaged 23 monkeys that had been living in social groups set to different sizes. Subject comparison revealed that living in larger groups caused increases in gray matter in mid-superior temporal sulcus and rostral prefrontal cortex and increased coupling of activity in frontal and temporal cortex. Social network size, therefore, contributes to changes both in brain structure and function. The changes have potential implications for an animal's success in a social context; gray matter differences in similar areas were also correlated with each animal's dominance within its social network.

Distinct and overlapping functional zones in the cerebellum defined by resting state functional connectivity

The cerebellum processes information from functionally diverse regions of the cerebral cortex. Cerebellar input and output nuclei have connections with prefrontal, parietal, and sensory cortex as well as motor and premotor cortex. However, the topography of the connections between the cerebellar and cerebral cortices remains largely unmapped, as it is relatively unamenable to anatomical methods. We used resting-state functional magnetic resonance imaging to define subregions within the cerebellar cortex based on their functional connectivity with the cerebral cortex. We mapped resting-state functional connectivity voxel-wise across the cerebellar cortex, for cerebral-cortical masks covering prefrontal, motor, somatosensory, posterior parietal, visual, and auditory cortices. We found that the cerebellum can be divided into at least 2 zones: 1) a primary sensorimotor zone (Lobules V, VI, and VIII), which contains overlapping functional connectivity maps for domain-specific motor, somatosensory, visual, and auditory cortices; and 2) a supramodal zone (Lobules VIIa, Crus I, and II), which contains overlapping functional connectivity maps for prefrontal and posterior-parietal cortex. The cortical connectivity of the supramodal zone was driven by regions of frontal and parietal cortex which are not directly involved in sensory or motor processing, including dorsolateral prefrontal cortex and the frontal pole, and the inferior parietal lobule.

Acquisition of the Temporal and Ordinal Structure of Movement Sequences in Incidental Learning

We investigated the acquisition and integration of temporal and ordinal sequence information in an incidental learning model of motor skill acquisition (the serial reaction time task). Human participants were exposed to a stimulus-response sequence that had temporal structure, ordinal structure, or both. By changing the temporal or ordinal structure, or both, we were able to ask two questions: first, does a regular temporal structure facilitate learning of an ordinal sequence and second, is a temporal sequence, presented in the context of a random ordinal sequence of finger movements, “picked up” through incidental learning? We found that a predictable temporal structure greatly facilitated the learning of an ordinal sequence but was not learned when presented in isolation. The results suggest that when motor skills are acquired under incidental learning conditions, timing is represented at a level specific to the ordinal sequence of movements rather than as an independent temporal template.

The cerebellum predicts the timing of perceptual events

Prospective (forward) temporal-spatial models are essential for both action and perception, but the literature on perceptual prediction has primarily been limited to the spatial domain. In this study we asked how the neural systems of perceptual prediction change, when change-over-time must be modeled. We used a naturalistic paradigm in which observers had to extrapolate the trajectory of an occluded moving object to make perceptual judgments based on the spatial (direction) or temporal-spatial (velocity) characteristics of object motion. Using functional magnetic resonance imaging we found that a region in posterior cerebellum (lobule VII crus 1) was engaged specifically when a temporal-spatial model was required (velocity judgment task), suggesting that circuitry involved in motor forward-modeling may also be engaged in perceptual prediction when a model of change-over-time is required. This cerebellar region appears to supply a temporal signal to cortical networks involved in spatial orienting: a frontal-parietal network associated with attentional orienting was engaged in both (spatial and temporal-spatial) tasks, but functional connectivity between these regions and the posterior cerebellum was enhanced in the temporal-spatial prediction task. In addition to the oculomotor spatial orienting network, regions involved in hand movements (aIP and PMv) were recruited in the temporal-spatial task, suggesting that the nature of perceptual prediction may bias the recruitment of sensory-motor networks in orienting. Finally, in temporal-spatial prediction, functional connectivity was enhanced between the cerebellum and the putamen, a structure which has been proposed to supply the brain's metric of time, in the temporal-spatial prediction task.