Value learning and arousal in the extinction of probabilistic rewards: the role of dopamine in a modified temporal difference model

Music Listening and Homeostatic Regulation: Surviving and Flourishing in a Sonic World

This paper argues for a biological conception of music listening as an evolutionary achievement that is related to a long history of cognitive and affective-emotional functions, which are grounded in basic homeostatic regulation. Starting from the three levels of description, the acoustic description of sounds, the neurological level of processing, and the psychological correlates of neural stimulation, it conceives of listeners as open systems that are in continuous interaction with the sonic world. By monitoring and altering their current state, they can try to stay within the limits of operating set points in the pursuit of a controlled state of dynamic equilibrium, which is fueled by interoceptive and exteroceptive sources of information. Listening, in this homeostatic view, can be adaptive and goal-directed with the aim of maintaining the internal physiology and directing behavior towards conditions that make it possible to thrive by seeking out stimuli that are valued as beneficial and worthy, or by attempting to avoid those that are annoying and harmful. This calls forth the mechanisms of pleasure and reward, the distinction between pleasure and enjoyment, the twin notions of valence and arousal, the affect-related consequences of music listening, the role of affective regulation and visceral reactions to the sounds, and the distinction between adaptive and maladaptive listening.

The Brainstem in Emotion: A Review

Emotions depend upon the integrated activity of neural networks that modulate arousal, autonomic function, motor control, and somatosensation. Brainstem nodes play critical roles in each of these networks, but prior studies of the neuroanatomic basis of emotion, particularly in the human neuropsychological literature, have mostly focused on the contributions of cortical rather than subcortical structures. Given the size and complexity of brainstem circuits, elucidating their structural and functional properties involves technical challenges. However, recent advances in neuroimaging have begun to accelerate research into the brainstem’s role in emotion. In this review, we provide a conceptual framework for neuroscience, psychology and behavioral science researchers to study brainstem involvement in human emotions. The “emotional brainstem” is comprised of three major networks – Ascending, Descending and Modulatory. The Ascending network is composed chiefly of the spinothalamic tracts and their projections to brainstem nuclei, which transmit sensory information from the body to rostral structures. The Descending motor network is subdivided into medial projections from the reticular formation that modulate the gain of inputs impacting emotional salience, and lateral projections from the periaqueductal gray, hypothalamus and amygdala that activate characteristic emotional behaviors. Finally, the brainstem is home to a group of modulatory neurotransmitter pathways, such as those arising from the raphe nuclei (serotonergic), ventral tegmental area (dopaminergic) and locus coeruleus (noradrenergic), which form a Modulatory network that coordinates interactions between the Ascending and Descending networks. Integration of signaling within these three networks occurs at all levels of the brainstem, with progressively more complex forms of integration occurring in the hypothalamus and thalamus. These intermediary structures, in turn, provide input for the most complex integrations, which occur in the frontal, insular, cingulate and other regions of the cerebral cortex. Phylogenetically older brainstem networks inform the functioning of evolutionarily newer rostral regions, which in turn regulate and modulate the older structures. Via these bidirectional interactions, the human brainstem contributes to the evaluation of sensory information and triggers fixed-action pattern responses that together constitute the finely differentiated spectrum of possible emotions.

Uncertainty-Dependent Extinction of Fear Memory in an Amygdala-mPFC Neural Circuit Model

Uncertainty of fear conditioning is crucial for the acquisition and extinction of fear memory. Fear memory acquired through partial pairings of a conditioned stimulus (CS) and an unconditioned stimulus (US) is more resistant to extinction than that acquired through full pairings; this effect is known as the partial reinforcement extinction effect (PREE). Although the PREE has been explained by psychological theories, the neural mechanisms underlying the PREE remain largely unclear. Here, we developed a neural circuit model based on three distinct types of neurons (fear, persistent and extinction neurons) in the amygdala and medial prefrontal cortex (mPFC). In the model, the fear, persistent and extinction neurons encode predictions of net severity, of unconditioned stimulus (US) intensity, and of net safety, respectively. Our simulation successfully reproduces the PREE. We revealed that unpredictability of the US during extinction was represented by the combined responses of the three types of neurons, which are critical for the PREE. In addition, we extended the model to include amygdala subregions and the mPFC to address a recent finding that the ventral mPFC (vmPFC) is required for consolidating extinction memory but not for memory retrieval. Furthermore, model simulations led us to propose a novel procedure to enhance extinction learning through re-conditioning with a stronger US; strengthened fear memory up-regulates the extinction neuron, which, in turn, further inhibits the fear neuron during re-extinction. Thus, our models increased the understanding of the functional roles of the amygdala and vmPFC in the processing of uncertainty in fear conditioning and extinction.

Animals live in environments that contain uncertainty. To adapt to uncertain situations, they flexibly learn to associate environmental cues with rewards and punishments. Understanding how the brain processes uncertainty has remained an important issue in neuroscience. To address this question, we focused on neural processing in the amygdala and mPFC during fear conditioning and extinction. We developed a neural circuit model that incorporates distinct neural populations in the amygdala and mPFC. Our model first successfully reproduced uncertainty-dependent resistance to the extinction of fear memory. An extension of the model provided a possible explanation for observations made during optogenetic manipulation of the ventral mPFC. Finally, we proposed a procedure to accelerate the efficacy of subsequent extinction based on our model.

The effects of methylphenidate on cerebral responses to conflict anticipation and unsigned prediction error in a stop-signal task

To adapt flexibly to a rapidly changing environment, humans must anticipate conflict and respond to surprising, unexpected events. To this end, the brain estimates upcoming conflict on the basis of prior experience and computes unsigned prediction error (UPE). Although much work implicates catecholamines in cognitive control, little is known about how pharmacological manipulation of catecholamines affects the neural processes underlying conflict anticipation and UPE computation. We addressed this issue by imaging 24 healthy young adults who received a 45 mg oral dose of methylphenidate (MPH) and 62 matched controls who did not receive MPH prior to performing the stop-signal task. We used a Bayesian Dynamic Belief Model to make trial-by-trial estimates of conflict and UPE during task performance. Replicating previous research, the control group showed anticipation-related activation in the presupplementary motor area and deactivation in the ventromedial prefrontal cortex and parahippocampal gyrus, as well as UPE-related activations in the dorsal anterior cingulate, insula, and inferior parietal lobule. In group comparison, MPH increased anticipation activity in the bilateral caudate head and decreased UPE activity in each of the aforementioned regions. These findings highlight distinct effects of catecholamines on the neural mechanisms underlying conflict anticipation and UPE, signals critical to learning and adaptive behavior.