Representation of reward feedback in primate auditory cortex

Neural correlates of auditory recognition memory in the primate dorsal temporal pole

Temporal pole (TP) cortex is associated with higher-order sensory perception and/or recognition memory, as human patients with damage in this region show impaired performance during some tasks requiring recognition memory (Olson et al. 2007). The underlying mechanisms of TP processing are largely based on examination of the visual nervous system in humans and monkeys, while little is known about neuronal activity patterns in the auditory portion of this region, dorsal TP (dTP; Poremba et al. 2003). The present study examines single-unit activity of dTP in rhesus monkeys performing a delayed matching-to-sample task utilizing auditory stimuli, wherein two sounds are determined to be the same or different. Neurons of dTP encode several task-relevant events during the delayed matching-to-sample task, and encoding of auditory cues in this region is associated with accurate recognition performance. Population activity in dTP shows a match suppression mechanism to identical, repeated sound stimuli similar to that observed in the visual object identification pathway located ventral to dTP (Desimone 1996; Nakamura and Kubota 1996). However, in contrast to sustained visual delay-related activity in nearby analogous regions, auditory delay-related activity in dTP is transient and limited. Neurons in dTP respond selectively to different sound stimuli and often change their sound response preferences between experimental contexts. Current findings suggest a significant role for dTP in auditory recognition memory similar in many respects to the visual nervous system, while delay memory firing patterns are not prominent, which may relate to monkeys' shorter forgetting thresholds for auditory vs. visual objects.

Set and setting: how behavioral state regulates sensory function and plasticity

Recently developed neuroimaging and electrophysiological techniques are allowing us to answer fundamental questions about how behavioral states regulate our perception of the external environment. Studies using these techniques have yielded surprising insights into how sensory processing is affected at the earliest stages by attention and motivation, and how new sensory information received during wakefulness (e.g., during learning) continues to affect sensory brain circuits (leading to plastic changes) during subsequent sleep. This review aims to describe how brain states affect sensory response properties among neurons in primary and secondary sensory cortices, and how this relates to psychophysical detection thresholds and performance on sensory discrimination tasks. This is not intended to serve as a comprehensive overview of all brain states, or all sensory systems, but instead as an illustrative description of how three specific state variables (attention, motivation, and vigilance [i.e., sleep vs. wakefulness]) affect sensory systems in which they have been best studied.

Rhythm sensitivity in macaque monkeys

This study provides evidence that monkeys are rhythm sensitive. We composed isochronous tone sequences consisting of repeating triplets of two short tones and one long tone which humans perceive as repeating triplets of two weak and one strong beat. This regular sequence was compared to an irregular sequence with the same number of randomly arranged short and long tones with no such beat structure. To search for indication of rhythm sensitivity we employed an oddball paradigm in which occasional duration deviants were introduced in the sequences. In a pilot study on humans we showed that subjects more easily detected these deviants when they occurred in a regular sequence. In the monkeys we searched for spontaneous behaviors the animals executed concomitant with the deviants. We found that monkeys more frequently exhibited changes of gaze and facial expressions to the deviants when they occurred in the regular sequence compared to the irregular sequence. In addition we recorded neuronal firing and local field potentials from 175 sites of the primary auditory cortex during sequence presentation. We found that both types of neuronal signals differentiated regular from irregular sequences. Both signals were stronger in regular sequences and occurred after the onset of the long tones, i.e., at the position of the strong beat. Local field potential responses were also significantly larger for the durational deviants in regular sequences, yet in a later time window. We speculate that these temporal pattern-selective mechanisms with a focus on strong beats and their deviants underlie the perception of rhythm in the chosen sequences.

Differences between primary auditory cortex and auditory belt related to encoding and choice for AM sounds

We recorded from middle-lateral (ML) and primary (A1) auditory cortex while macaques discriminated amplitude-modulated (AM) noise from unmodulated noise. Compared with A1, ML had a higher proportion of neurons that encoded increasing AM depth by decreasing their firing rates ("decreasing" neurons), particularly with responses that were not synchronized to the modulation. Choice probability (CP) analysis revealed that A1 and ML activity were different during the first half of the test stimulus. In A1, significant CP began before the test stimulus, remained relatively constant (or increased slightly) during the stimulus, and increased greatly within 200 ms of lever release. Neurons in ML behaved similarly, except that significant CP disappeared during the first half of the stimulus and reappeared during the second half and prerelease periods. CP differences between A1 and ML depend on neural response type. In ML (but not A1), when activity was lower during the first half of the stimulus in nonsynchronized, decreasing neurons, the monkey was more likely to report AM. Neurons that both increased firing rate with increasing modulation depth ("increasing" neurons) and synchronized their responses to AM had similar choice-related activity dynamics in ML and A1. These results suggest that, when ascending the auditory system, there is a transformation in coding AM from primarily synchronized increasing responses in A1 to nonsynchronized and dual (increasing/decreasing) coding in ML. This sensory transformation is accompanied by changes in the timing of activity related to choice, suggesting functional differences between A1 and ML related to attention and/or behavior.

Feedback that confirms reward expectation triggers auditory cortex activity

Associative learning studies have shown that the anticipation of reward and punishment shapes the representation of sensory stimuli, which is further modulated by dopamine. Less is known about whether and how reward delivery activates sensory cortices and the role of dopamine at that time point of learning. We used an appetitive instrumental learning task in which participants had to learn that a specific class of frequency-modulated tones predicted a monetary reward following fast and correct responses in a succeeding reaction time task. These fMRI data were previously analyzed regarding the effect of reward anticipation, but here we focused on neural activity to the reward outcome relative to the reward expectation and tested whether such activation in the reward reception phase is modulated by L-DOPA. We analyzed neural responses at the time point of reward outcome under three different conditions: 1) when a reward was expected and received, 2) when a reward was expected but not received, and 3) when a reward was not expected and not received. Neural activity in auditory cortex was enhanced during feedback delivery either when an expected reward was received or when the expectation of obtaining no reward was correct. This differential neural activity in auditory cortex was only seen in subjects who learned the reward association and not under dopaminergic modulation. Our data provide evidence that auditory cortices are active at the time point of reward outcome. However, responses are not dependent on the reward itself but on whether the outcome confirmed the subject's expectations.

Characterising reward outcome signals in sensory cortex

Reward outcome signalling in the sensory cortex is held as important for linking stimuli to their consequences and for modulating perceptual learning in response to incentives. Evidence for reward outcome signalling has been found in sensory regions including the visual, auditory and somatosensory cortices across a range of different paradigms, but it is unknown whether the population of neurons signalling rewarding outcomes are the same as those processing predictive stimuli. We addressed this question using a multivariate analysis of high-resolution functional magnetic resonance imaging (fMRI), in a task where subjects were engaged in instrumental learning with visual predictive cues and auditory signalled reward feedback. We found evidence that outcome signals in sensory regions localise to the same areas involved in stimulus processing. These outcome signals are non-specific and we show that the neuronal populations involved in stimulus representation are not their exclusive target, in keeping with theoretical models of value learning. Thus, our results reveal one likely mechanism through which rewarding outcomes are linked to predictive sensory stimuli, a link that may be key for both reward and perceptual learning.

•

Rewarding outcomes reactivate sensory regions involved in stimulus representation.

•

Within these regions reward is not directed specifically to stimulus-coding neurons.

•

This resembles a teaching signal predicted by reinforcement learning models.

Auditory cortex represents both pitch judgments and the corresponding acoustic cues

The neural processing of sensory stimuli involves a transformation of physical stimulus parameters into perceptual features, and elucidating where and how this transformation occurs is one of the ultimate aims of sensory neurophysiology. Recent studies have shown that the firing of neurons in early sensory cortex can be modulated by multisensory interactions [1-5], motor behavior [1, 3, 6, 7], and reward feedback [1, 8, 9], but it remains unclear whether neural activity is more closely tied to perception, as indicated by behavioral choice, or to the physical properties of the stimulus. We investigated which of these properties are predominantly represented in auditory cortex by recording local field potentials (LFPs) and multiunit spiking activity in ferrets while they discriminated the pitch of artificial vowels. We found that auditory cortical activity is informative both about the fundamental frequency (F0) of a target sound and also about the pitch that the animals appear to perceive given their behavioral responses. Surprisingly, although the stimulus F0 was well represented at the onset of the target sound, neural activity throughout auditory cortex frequently predicted the reported pitch better than the target F0.

Stable bottom-up processing during dynamic top-down modulations in monkey auditory cortex

It is unclear whether top-down processing in the auditory cortex (AC) interferes with its bottom-up analysis of sound. Recent studies indicated non-acoustic modulations of AC responses, and that attention changes a neuron's spectrotemporal tuning. As a result, the AC would seem ill-suited to represent a stable acoustic environment, which is deemed crucial for auditory perception. To assess whether top-down signals influence acoustic tuning in tasks without directed attention, we compared monkey single-unit AC responses to dynamic spectrotemporal sounds under different behavioral conditions. Recordings were mostly made from neurons located in primary fields (primary AC and area R of the AC) that were well tuned to pure tones, with short onset latencies. We demonstrated that responses in the AC were substantially modulated during an auditory detection task and that these modulations were systematically related to top-down processes. Importantly, despite these significant modulations, the spectrotemporal receptive fields of all neurons remained remarkably stable. Our results suggest multiplexed encoding of bottom-up acoustic and top-down task-related signals at single AC neurons. This mechanism preserves a stable representation of the acoustic environment despite strong non-acoustic modulations.

Features versus feelings: dissociable representations of the acoustic features and valence of aversive sounds

This study addresses the neuronal representation of aversive sounds that are perceived as unpleasant. Functional magnetic resonance imaging in humans demonstrated responses in the amygdala and auditory cortex to aversive sounds. We show that the amygdala encodes both the acoustic features of a stimulus and its valence (perceived unpleasantness). Dynamic causal modeling of this system revealed that evoked responses to sounds are relayed to the amygdala via auditory cortex. While acoustic features modulate effective connectivity from auditory cortex to the amygdala, the valence modulates the effective connectivity from amygdala to the auditory cortex. These results support a complex (recurrent) interaction between the auditory cortex and amygdala based on object-level analysis in the auditory cortex that portends the assignment of emotional valence in amygdala that in turn influences the representation of salient information in auditory cortex.

Effects of L-dopa during auditory instrumental learning in humans

The dopaminergic neurotransmitter system is critically involved in promoting plasticity in auditory cortex. We combined functional magnetic resonance imaging (fMRI) and a pharmacological manipulation to investigate dopaminergic modulation of neural activity in auditory cortex during instrumental learning. Volunteers either received 100 mg L-dopa (Madopar) or placebo in an appetitive, differential instrumental conditioning paradigm, which involved learning that a specific category of frequency modulated tones predicts a monetary reward when fast responses were made in a subsequent reaction time task. The other category of frequency modulated tones was not related to a reward. Our behavioral data provides evidence that dopaminergic stimulation differentially impacts on the speed of instrumental responding in rewarded and unrewarded trials. L-dopa increased neural BOLD activity in left auditory cortex to tones in rewarded and unrewarded trials. This increase was related to plasma L-dopa levels and learning rate. Our data thus provides evidence for dopaminergic modulation of neural activity in auditory cortex, which occurs for both auditory stimuli related to a later reward and those not related to a reward.

Harnessing plasticity to understand learning and treat disease

A large body of evidence suggests that neural plasticity contributes to learning and disease. Recent studies suggest that cortical map plasticity is typically a transient phase that improves learning by increasing the pool of task-relevant responses. Here, I discuss a new perspective on neural plasticity and suggest how plasticity might be targeted to reset dysfunctional circuits. Specifically, a new model is proposed in which map expansion provides a form of replication with variation that supports a Darwinian mechanism to select the most behaviorally useful circuits. Precisely targeted neural plasticity provides a new avenue for the treatment of neurological and psychiatric disorders and is a powerful tool to test the neural mechanisms of learning and memory.

Signal detection in animal psychoacoustics: analysis and simulation of sensory and decision-related influences

Signal detection theory (SDT) provides a framework for interpreting psychophysical experiments, separating the putative internal sensory representation and the decision process. SDT was used to analyse ferret behavioural responses in a (yes–no) tone-in-noise detection task. Instead of measuring the receiver-operating characteristic (ROC), we tested SDT by comparing responses collected using two common psychophysical data collection methods. These (Constant Stimuli, Limits) differ in the set of signal levels presented within and across behavioural sessions. The results support the use of SDT as a method of analysis: SDT sensory component was unchanged between the two methods, even though decisions depended on the stimuli presented within a behavioural session. Decision criterion varied trial-by-trial: a ‘yes’ response was more likely after a correct rejection trial than a hit trial. Simulation using an SDT model with several decision components reproduced the experimental observations accurately, leaving only ∼10% of the variance unaccounted for. The model also showed that trial-by-trial dependencies were unlikely to influence measured psychometric functions or thresholds. An additional model component suggested that inattention did not contribute substantially. Further analysis showed that ferrets were changing their decision criteria, almost optimally, to maximise the reward obtained in a session. The data suggest trial-by-trial reward-driven optimization of the decision process. Understanding the factors determining behavioural responses is important for correlating neural activity and behaviour. SDT provides a good account of animal psychoacoustics, and can be validated using standard psychophysical methods and computer simulations, without recourse to ROC measurements.

Learning-dependent plasticity in human auditory cortex during appetitive operant conditioning

Animal experiments provide evidence that learning to associate an auditory stimulus with a reward causes representational changes in auditory cortex. However, most studies did not investigate the temporal formation of learning-dependent plasticity during the task but rather compared auditory cortex receptive fields before and after conditioning. We here present a functional magnetic resonance imaging study on learning-related plasticity in the human auditory cortex during operant appetitive conditioning. Participants had to learn to associate a specific category of frequency-modulated tones with a reward. Only participants who learned this association developed learning-dependent plasticity in left auditory cortex over the course of the experiment. No differential responses to reward predicting and nonreward predicting tones were found in auditory cortex in nonlearners. In addition, learners showed similar learning-induced differential responses to reward-predicting and nonreward-predicting tones in the ventral tegmental area and the nucleus accumbens, two core regions of the dopaminergic neurotransmitter system. This may indicate a dopaminergic influence on the formation of learning-dependent plasticity in auditory cortex, as it has been suggested by previous animal studies.

Activity related to perceptual judgment and action in primary auditory cortex

Recent evidence is reshaping the view of primary auditory cortex (A1) from a unisensory area to one more involved in dynamically integrating multisensory- and task-related information. We found A1 single- (SU) and multiple-unit (MU) activity correlated with macaques' choices in an amplitude modulation (AM) discrimination task. Animals were trained to discriminate AM noise from unmodulated noise by releasing a lever for AM noise and holding down the lever for unmodulated noise. Activity for identical stimuli was compared between trials where the animals reported AM and trials where they did not. We found 47.4% of MUs and 22.8% of SUs significantly increased firing shortly before the animal's behavioral response to report AM when compared to the equivalent time period on trials where AM was not reported. Activity was also linked to lever release in a different task context, suggesting A1 modulation by somatosensory, or efference copy, input. When spikes were counted only during the stimulus, 19.6% of MUs and 13.8% of SUs increased firing rate when animals reported AM compared to when they did not, suggesting an attentional effect, or that A1 activity can be used by higher decision areas, or that such areas provide feedback to A1. Activity associated with AM reporting was correlated with a unit's AM sensitivity, suggesting AM sensitive neurons' involvement in task performance. A1 neurons' phase locking to AM correlated more weakly (compared to firing rate) with the animals' report of AM, suggesting a preferential role for rate-codes in A1 for this AM discrimination task.

Extinction reveals that primary sensory cortex predicts reinforcement outcome

Primary sensory cortices are traditionally regarded as stimulus analysers. However, studies of associative learning-induced plasticity in the primary auditory cortex (A1) indicate involvement in learning, memory and other cognitive processes. For example, the area of representation of a tone becomes larger for stronger auditory memories and the magnitude of area gain is proportional to the degree that a tone becomes behaviorally important. Here, we used extinction to investigate whether 'behavioral importance' specifically reflects a sound's ability to predict reinforcement (reward or punishment) vs. to predict any significant change in the meaning of a sound. If the former, then extinction should reverse area gains as the signal no longer predicts reinforcement. Rats (n = 11) were trained to bar-press to a signal tone (5.0 kHz) for water-rewards, to induce signal-specific area gains in A1. After subsequent withdrawal of reward, A1 was mapped to determine representational areas. Signal-specific area gains, estimated from a previously established brain-behavior quantitative function, were reversed, supporting the 'reinforcement prediction' hypothesis. Area loss was specific to the signal tone vs. test tones, further indicating that withdrawal of reinforcement, rather than unreinforced tone presentation per se, was responsible for area loss. Importantly, the amount of area loss was correlated with the amount of extinction (r = 0.82, P < 0.01). These findings show that primary sensory cortical representation can encode behavioral importance as a signal's value to predict reinforcement, and that the number of cells tuned to a stimulus can dictate its ability to command behavior.

Slow Modulation of Ongoing Discharge in the Auditory Cortex during an Interval-Discrimination Task

In this study, we recorded single unit activity from rat auditory cortex while the animals performed an interval-discrimination task. The animals had to decide whether two auditory stimuli were separated by either 150 or 300 ms, and go to the left or right nose poke accordingly. Spontaneous firing in between auditory responses was compared in the attentive versus non-attentive brain states. We describe the firing rate modulation detected during intervals while there was no auditory stimulation. Nearly 18% of neurons (n = 14) showed a prominent neuronal discharge during the interstimulus interval, in the form of an upward or downward ramp towards the second auditory stimulus. These patterns of spontaneous activity were often modulated in the attentive versus passive trials. Modulation of the spontaneous firing rate during the task was observed not only between auditory stimuli, but also in the interval preceding the stimulus. These slow modulatory components could be locally generated or the result of a top-down influence originated in higher associative association areas. Such a neuronal discharge may be related to the computation of the interval time and contribute to the perception of the auditory stimulus.